Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Proposal] Add Subtitle Support to Media Element #1899

Open
5 of 8 tasks
ne0rrmatrix opened this issue May 24, 2024 · 7 comments · May be fixed by #1918
Open
5 of 8 tasks

[Proposal] Add Subtitle Support to Media Element #1899

ne0rrmatrix opened this issue May 24, 2024 · 7 comments · May be fixed by #1918
Assignees
@ne0rrmatrix
Labels
approved This Proposal has been approved and is ready to be added to the Toolkit champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature 📽️ MediaElement Issue/PR that has to do with MediaElement proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail

Comments

@ne0rrmatrix
Copy link
Contributor

ne0rrmatrix commented May 24, 2024 •
edited
Loading

Feature name

Media Element Subtitles

Link to discussion

#1867

Progress tracker

  • Android Implementation
  • iOS Implementation
  • MacCatalyst Implementation
  • Windows Implementation
  • Tizen Implementation
  • Unit Tests
  • Samples
  • Documentation

Summary

Add support for Subtitles to Media Element. Initial support for Windows, Mac, iOS, and Android. Subtitle format allowed at launch include srt and sub.

Windows

windows-subtitle-normal

Windows-subtitle-fullscreen

Android

Android-normal-subtitles

Android-subtitles-fullscreen

IOS

2024-05-09.17-11-43.mp4

Motivation

I would like to help support people who have difficulty with hearing or need assistance in loud noisy environments where it is hard to hear the dialogue. Or people that want translations of foreign languages spoken that they do not know. Other people just like subtitles. This will add robust support that can be expanded later.

Detailed Design

API:

	/// <summary>
	/// Backing store for the <see cref="SubtitleFont"/> property.
	/// </summary>
	public static readonly BindableProperty SubtitleFontProperty = BindableProperty.Create(nameof(SubtitleFont), typeof(string), typeof(MediaElement), string.Empty);
	
	/// <summary>
	/// Backing store for the <see cref="SubtitleFontSize"/> property.
	/// </summary>
	public static readonly BindableProperty SubtitleFontSizeProperty = BindableProperty.Create(nameof(SubtitleFontSize), typeof(double), typeof(MediaElement), 16.0);

	/// <summary>
	/// Backing store for the <see cref="SubtitleUrl"/> property.
	/// </summary>
	public static readonly BindableProperty SubtitleProperty = BindableProperty.Create(nameof(SubtitleUrl), typeof(string), typeof(MediaElement), string.Empty);

Usage Syntax

XAML Usage:

 <toolkit:MediaElement
     x:Name="MediaElement"
     ShouldAutoPlay="True"
     SubtitleFont="monospace"
     SubtitleFontSize="20"
     SubtitleUrl="https://raw.githubusercontent.com/ne0rrmatrix/SampleVideo/main/SRT/WindowsVideo.srt"
     Source="https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"/>

C# Code behind example

MediaElement.SubtitleFont = "monospace";
MediaElement.SubtitleFontSize = 16;
MediaElement.SubtitleUrl = "https://raw.githubusercontent.com/ne0rrmatrix/SampleVideo/main/SRT/WindowsVideo.srt";
MediaElement.Source = MediaSource.FromResource("WindowsVideo.mp4");

Drawbacks

Currently it uses a custom class to read the subtitle files and convert them to strings that can be used at specific times in a text box appropriate for each OS. This is a new class that only supports plain text at the moment. Non standard files and RTF formatted files are not supported.

Alternatives

The custom class for reading subtitle files was written to allow iOS and Mac Catalyst support as the current methods are to use m3u8 files and do not support directly linking a subtitle file. ATM we have support for m3u8 files but we have no control over how or what is displayed at all. Android and Windows both have robust libraries that support subtitles. They do not support any common format of subtitles.

ATM if you want uniform support for multiple different formats on each and every device my custom class is the way to go.

Unresolved Questions

Do we want to use this custom implementation or go with native libraries? I have both ready for testing.
@ne0rrmatrix ne0rrmatrix added new proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail labels May 24, 2024
@brminnick brminnick added the needs discussion Discuss it on the next Monthly standup label May 24, 2024
@vhugogarcia vhugogarcia added the 📽️ MediaElement Issue/PR that has to do with MediaElement label May 27, 2024
@brminnick brminnick added champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature and removed new needs discussion Discuss it on the next Monthly standup labels Jun 6, 2024
@brminnick brminnick added the approved This Proposal has been approved and is ready to be added to the Toolkit label Jun 6, 2024
@ne0rrmatrix ne0rrmatrix linked a pull request Jun 7, 2024 that will close this issue
Draft
6 tasks
@ne0rrmatrix
Copy link
Contributor Author

All the work for this features current API is done. I have been refactoring code. I need to update the docs and get them approved. In about another week I will mark it as ready for review. I need to do device testing to see if there are any show stopper bugs or things that emulators missed.

@ne0rrmatrix ne0rrmatrix self-assigned this Aug 19, 2024
@imememani
Copy link

How's this looking? Feel like this could be a really great addition.

@JonathanWebsdale
Copy link

Any updates on availability? Currently investigating the use of bunny.net as a media provider and restricted to using their viewer hosted in an iFrame inside a WebView control (which brings its own problems), but they do have support for VVT and SRT format subtitle files. Would ideally want to test this within the MediaElement.

@bijington
Copy link
Contributor

I believe @ne0rrmatrix was really close with this. We have piled a lot of effort into .NET 9.0 support which we hope to merge in the next week or so and then we can start to revisit open PRs like the one for this proposal 👍

@ne0rrmatrix
Copy link
Contributor Author

The last unresolved issue was solved months ago. I will update that this morning to reflect that I fixed the font issue. The one thing I have done that will require input is a complete rewrite that implements all this by using built in classes and we need to decide whether we go with custom implementation for subtitles or use native methods. Using native libs removes the need to support it but with custom libs we can do anything we want like allow third party support for custom subtitle support. An example is you can insert your own implementation for parser with this version but native libs don't allow this.

@MuhammadTalhaahmed
Copy link

The last unresolved issue was solved months ago. I will update that this morning to reflect that I fixed the font issue. The one thing I have done that will require input is a complete rewrite that implements all this by using built in classes and we need to decide whether we go with custom implementation for subtitles or use native methods. Using native libs removes the need to support it but with custom libs we can do anything we want like allow third party support for custom subtitle support. An example is you can insert your own implementation for parser with this version but native libs don't allow this.

You mean Subtitle support is finally ready and is going to merge in main ?
Can you explain it more simply what you said about "Custom implementation for Subtitles" or "Native Methods".

@ne0rrmatrix
Copy link
Contributor Author

You will be able to add a custom parser and support any text based subtitles that media element does not itself natively support. ATM I have added support for srt and vtt files. Or if you find a need for say supporting html and do not want to use my text only srt or vtt you will be able to just your own implementation and integrate that with media element directly yourself. It has great versitility. The choice will be yours. Or we can go with native libs(I have both versions ready) which will have better UI support with native buttons on windows and android plus support mulitple languages in both but not in IOS or mac os. Both versions do support both vtt and srt in a single file.

The key differences are the native version supports a UI button and both versions let you select a variety of subtitle files that can be added as a simple list of URL's for both windows and android. This is not supported on apple devices natively. On the other hand you can write your own parser to fix any issues mine has plus you can add additional supported file types as you want with my custom implementation. I will let the community decide which version they want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ne0rrmatrix ne0rrmatrix

Labels
approved This Proposal has been approved and is ready to be added to the Toolkit champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature 📽️ MediaElement Issue/PR that has to do with MediaElement proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Subtitle Support for MediaElement ne0rrmatrix/MauiOld
7 participants
@vhugogarcia @ne0rrmatrix @brminnick @JonathanWebsdale @bijington @imememani @MuhammadTalhaahmed