Synchronized closed captions for live tv

[jay343]

I've been experimenting with adding real-time synchronized closed captions to news broadcasts. Whisper can do it, but it creates a .srt file which the Fire TV client doesn't support. The generated captions can be muxed back into the video file with ffmpeg, but it's only supported in mp4 and mkv files (at least reliably), so the database would need to be updated to make the mp4/mkv file appear in the UI instead of the mpg.

Is it possible to update the database on the fly after the new video file is created? As I recall, there is an API that should support that.

Is there an easier way to accomplish this? How about adding support for .srt files in the Fire TV Channels client?

BTW, I'm using @Paul.C's "cuckoo" program to launch whisper after commercial detection. Are you guys planning to support custom post processing soon?

[tmm1]

You can rename the mp4 to mpg and replace the original file.

[jay343]

Thanks, I actually tried that, but the video would not render properly (noise, dropped frames, etc.) and the captions were mostly gibberish. The playback terminated after about a minute. I was using CoPilot (ChatGPT5) to set this up. It suggested renaming the .mp4 to .mpg and then updating the metadata. That didn't work either. I tried the import function, and that worked captions and all.

[jay343]

I tried that on another recording, and it worked fine. Not sure what went wrong on the first attempt because the file would play fine with dragon player, and it would play fine when renamed to .mpg withe the Android Channels client. I'll keep testing...

[jay343]

I have this mostly working now, but I have run into a problem that I don't understand. Sometimes commercials are processed without calling comskip? I notice this on CNN programs, but not others. What is happening?

[chDVRuser]

If it's a TVE or M3U channel, the show was probably processed using the Smart Commercial Detection, instead of comskip

BETA: Smart Commercial Detection for TVE & Pluto

With v2021.12.22.0148 we are testing a new smarter commercial detector, which works on some streaming channels. Instead of using comskip to scan video, this new detector uses a more traditional approach similar to how Pi-hole works. We can detect patterns on some channels where the advertising content comes from a different place. I have tested this with some Discovery channels on TVE, and some of the channels on Pluto. Share your own experiences below. Log output for traditional comskip: 2…

[jay343]

Yup, I see that. So, I refactored the program to watch the Channels DVR log for entries like "[DVR] Processing file-83171: TV/The Situation Room/The Situation Room 2025-12-11-0800.mpg". It appears that those occur after the recording is written to disk, so it's safe to process. This is running in linux, so I'm using tail -F to match those lines, then call the captioning program. So far so good!