Pluto Closed Captions

Channels DVR Server Troubleshooting
#23

Edit: I made some assumptions here that were wrong, as tmm1 points out below. Channels doesn’t use FFmpeg for HLS playback, nor do a lot of other projects.

————

I spent many hours in the last week trying to find solutions and every path I went down was a dead end.

From what information I have gathered, captions from Pluto do not work because FFmpeg, the library that Channels (and a lot of other video software) uses to handle multimedia does not support webvtt captions in HLS streams.

This issue has been known since at least 2013 and patches have been proposed but none have been accepted into the codebase.

The feature is supposedly behind an experimental flag, however, as lines 513-518 of this file show, the code to actually handle the webvtt stream does not exist.

To get on my soapbox a bit, I honestly find it unacceptable that the maintainers of FFmpeg have not made resolving this bug a priority. This is a library that is used by billions of people. HLS streams with WebVTT subtitles has become a de facto standard. I understand the resource limitations inherit to open source software projects, but projects like this have a social responsibility to make sure that their software works for the 430+ million deaf and hard-of-hearing people across the world, not to mention the countless neurodivergent people who depend on captions. I wish I understood how to fix the issue, but my technical acumen in this area is extremely limited.

@tmm1, I know this is an unlikely ask, but is it possible that patching the version of FFmpeg used in Channels with the code linked to in this GitHub comment would resolve the issue? I assume that you're already patching it for other things and that the effort might not make much financial sense. But I know a lot of people that use Channels would be greatly appreciative to get captions working from sources like Pluto and other live video streaming services.

#24

First let me say that I use CC most all the time, and I am unable to watch certain programs because of lack of CC. I have been writing code since before you ever knew what code was.

People who write code and give it away for free, owe the world nothing! If giving away code came with rules and responsibilities, people would stop giving away code. They write code that suits their personal needs, and sometimes add in some extras for other people.

Given enough effort (a lot), you could learn how to do this, but you would rather tell others to do the work for you. You call this a "bug". It is not. A "bug" is an unintended result. There was no intention to ever make this work.

There is a simple way to fix this. Pay a qualified person to write the code. You could pay for it yourself, or get some of those 430 million people to kick in a few bucks appease.

There is no such thing as a free lunch.

2 Likes
#25

Edit to add: I didn’t realize you were the OP of this thread. I spent hours trying to find a way to fix this issue for you and when I came back with my findings and advocated for you and others who are experiencing this issue, you decided to attack and insult me. Wow.

—

I'm not intending to start a flame war in the Channels community, so this will be my only reply to any criticism of my comment.

Starting off on a great foot here...

FFmpeg isn't just a small random hobby project. It is one of the most used FOSS projects in the world. Just look at their contributors list. They have engineers from Huawei, HBO, Kaltura, Twitch, Epic, Google, Vimeo, and other large companies contributing to the project.

When a project is used by that many companies for profit, then those companies have a social responsibility to ensure that that project is secure, well maintained, and accessible to people with disabilities.

I tried to find a way to solve it, but as I said above, my technical skills in this area are not adequate. So, I did what I am skilled in: research a technical issue, document what I've found, and publicly advocate that it be fixed for the sake of the disabled.

Don't get me wrong. I am very grateful and appreciative to people who write code for FOSS projects. It's a thankless job and my post was not intended to be a criticism at the work that they do. There are other ways to be a contributor, however, and one of those ways is to highlight areas where the project is failing. This is what security researchers do. How is what I did any different?

Except for the numerous tickets and proposed patches going back nearly a decade?

As far as I can tell, the code to fix it has been written. It just hasn't been accepted into the main codebase. I'd be more than happy to contribute money toward getting the issue resolved, but I don't see the point when there are already patches that the maintainers have not accepted.

#26

Almost no one uses ffmpeg for hls playback. Even OSS projects like VLC and Kodi don't use ffmpeg's hls code.

We are also no longer using ffmpeg for hls playback, because it simply does not work well.

Most companies are using ffmpeg on the server, to generate hls for consumption by players built into the OS or browser. Generating vs consuming hls in ffmpeg takes completely different code paths.

Anyway, this is on our radar and something I would like to implement, but its not very easy.

1 Like
#27

Thanks for providing more context. My knowledge in this area is pretty limited and I made some assumptions that I shouldn’t have. I’m sorry if I was out of line.

Is there anything that I and/or others can do to support getting this feature implemented?

#31

I have a very basic implementation available for testing. It may be buggy.

You will need the DVR server prerelease and the latest beta of the Channels app. With this combination, you should see subtitles on Pluto channels where they are available. New recordings from these Pluto channels will also include the subtitle data.

ADBTuner: A "channel tuning" application for networked Google TV / Android TV devices
#32

I dunno if this is related to subtitle/cc update, but ever since I loaded 2022.08.02.0430 this morning, many of my Pluto channels are showing video that is slowing/pausing and then rapid catching up all the while the audio remains consistent. This is happening on AppleTV 4k beta (8.2.438) and stable (5.4.1). The CC looks mostly fine, but sometimes choppy. The main issue is the video disruption. Diagnostics have been sent from the AppleTV client and the server.

Edit/Update: Also happening on FireTV 4.3.0. Also happening on some Plex custom channels.

#33

@Absenm Thanks, yes it was due to my changes. I reverted them in the next build, but I think have made some improvements now. With upcoming v2022.08.03.0007 subtitle support has been re-added and hopefully doesn't have the same video stuttering problems.

Please try it out and let me know how it works out for you.

#34

I'm still seeing the stuttering issue on Android mobile on the latest server pre-release and beta app while watching Pluto TV. I submitted player diagnostics twice a few minutes ago: once on hardware video decoder, once on software (same issue both times).

#35

Stuttering and Pausing when captions are on... Also sometimes that whole screen gets covered with Captions. Lipsync issues.

They repeat like below and sometimes fill whole screen

20220802_182113
20220802_1821131916×1437 124 KB

#36

Okay thanks reverting again

2 Likes
#37

Just curious if you’ve had any more luck with this or if it’s just technically impossible

#38

The few hours the experiment was operational it showed a lot of promise. I hope that it is still being worked on and that some day sooner than later we will finally have CC available on Pluto and other similarly affected Custom Channels sources. It was so disappointing when it got pulled so quickly. I completely understand why it got pulled, but was hoping that it would be tried again. Would be happy to alpha test (if that were a thing, lol).

3 Likes
#39

Damn, I was reading this thread last week and then I couldn't find it when I posted over in the "Pluto Holiday Channels added" thread.

Anyhow, here's my tidbit from there that I really meant to post here: Pluto Holiday Channels added - #15 by jkr4m3r

#40

It may be a good time to inquire as to if there has been any further development in making CC for Pluto happen. There have been changes to Channels and the clients over the past few months which I believe may be positive in implementing this feature (as referenced in a question I asked not to long ago).

EXPERIMENTAL: New MPEG-TS Rewriter for HLS (TVE)
#41

@eric @tmm1 is making this work possible now with the experimental rewriter or any other changes?

3 Likes
#42

Has there been anymore progress on getting this to work. I have been despratelty trying to get CC working on Pluto. Lately I have been experimenting with the HDMI for Channels to try to pull in channels from the Pluto app with unimpressive results getting some OSD that wont go away and getting the channels tuned in. I have also dabbled with the BETA: Chrome Capture for Channels which is too experimental to work with Pluto (but its less than a day old so keeping my flingers crossed).

Would love to see this working in ChannelsDVR tried and true Pluto methods.

#43

Channels newbie/non-technie here. I just implemented the Pluto channels nocords solution and it's amazing. Couldn't figure out how to turn on CC so came to see what I was doing wrong, and--there is no CC! Unfortunately, with hearing issues here it's a must. With TVE gradually/suddenly going away, the huge variety of Pluto channels would've gone a long way toward mitigating its loss. But when something seems too good to be true it often is. Whether it's a matter of resources/time or a complete impossibility IDK, but I'd still like to add a belated vote to the search for a solution. Thanks.

#44

A few of the Pluto channels do have closed captions, but yeah it sucks. Here’s a list I put together of sources that have captions. Some of the Pluto channels are on these other services that have captions. Worth trying out :slight_smile:

#45

As I recall, they work fine on the Pluto app.