EXPERIMENTAL: New MPEG-TS Rewriter for HLS (TVE)

Beta Community DVR Beta
#1

Our TV Everywhere and M3u sources ingests HLS streams that require us to rewrite the timestamps of to normalize them to consistent timestamps.

We've released a new mechanism for doing this rewriting that we are in the initial testing stages of. We hope that it will better be able to handle some of the issues that users have experienced with Discovery network based TVE channels. This new system should give us more flexibility to handle more kinds of feeds that caused us trouble in the past.

If you'd like to test this out, you can enable the setting in the DVR:

CleanShot 2022-10-31 at 17.55.16@2x
CleanShot 2022-10-31 at 17.55.16@2x2090×636 158 KB

PLEASE NOTE: Enabling this new feature may improve your experience, but if there is a bug, it could render TVE recordings made while it was enabled un-watchable. We've done testing as the feature has been developed and don't expect any issues, but this is brand new code, so it could have problems.

6 Likes
Discovery channel droputs?
Issues recording TVE from TLC, Discovery, TRVL and Food
Discovery Channels Not Working
Recording shows as too short, server crash when generate logs
Issues With Recording Length
DVR Recording Issues with HGTV Shows
Recording shows as too short, server crash when generate logs
#2
#3

@eric Great news.

A few questions if I may.
When enabled is this used for HDHR or M3U sources, or is it limited to only TVE sources.
If M3U sources, is it used for MPEG-TS, HLS or both.
Is it used for Live viewing, recordings, or both.

Just looking for clarification before I try it.

1 Like
Discovery Channels Not Working
#4

Will smart detection be enabled again and is it expected to work with the new experimental rewriter

1 Like
#5

It is used for all HLS-based sources across M3U, TVE, Pluto for both live and recordings.

Yes, it should be active for as long as it's still using the old CDN. Smart detection should continue to work with the new rewriter. If it isn't working right, that is a bug.

4 Likes
The "Logs" folder is using 60% of my recording drive. Safe to remove the content?
Major slowdowns when scrubbing through TVE channels on Apple TV
#6

This may be getting greedy, but it was the first question that came to my mind. Would this rewritter in any way help with the HLS WebVTT subtitles from TVE and M3U sources that was attempted in August? I honestly have no idea if it is even slightly involved with that project. Figured it wouldn't hurt to ask.

2 Likes
Pluto Closed Captions
#7

Yes, this new strategy will make WebVTT subtitles easier in the future.

2 Likes
#8

Last question (for tonight :grin: )
If enabled is it applied to all HLS streams, or does it make a determination what qualifies to be rewritten.

1 Like
#9

All HLS streams have always been and always are rewritten to have consistent timestamps. We don't have a way to know ahead of time if a HLS stream would need to be, so we just do it for all of them.

1 Like
#10

OK. so while remuxing the incoming HLS to a TS stream it's always rewritten timestamps and this is basically just a new experimental version of remuxing that HLS stream into the MPEG-TS transport stream?

#11

HLS segments are already MPEG-TS streams that can basically be concatenated together. Because the segments don't always have contiguous timestamps, we take the incoming MPEG-TS segments and rewrite the timestamps to be contiguous so that playback and seeking can work. Toggling this setting switches between our existing mechanism for doing that rewriting and a new one that works at a slightly different level of our code and will hopefully handle certain kinds of unusual HLS streams better.

3 Likes
#12

Thanks for the explanation. And hopefully it can help with non contiguos HLS segments too.

1 Like
#13

Just curious does this have any impact on login for TVE or is it strictly going to impact recording of TVE streams?

#14

Stress testing this... :slight_smile:

Recording ch6140 for First Take until 12:01PM: buf=0% drop=0%

Recording ch6192 for Good Morning Football until 1:01PM: buf=0% drop=0%

Recording ch6750 for Press Your Luck until 10:31AM: buf=0% drop=0%

Recording ch6782 for Campus Insiders until 10:31AM: buf=0% drop=0%

Recording ch9006 for The Twelve Trees of Christmas (2013) until 12:01PM: buf=0% drop=0%

#15

Is this also used while just watching ?

#16

Yes

4 Likes
#17

Not sure if this is source or this new rewriter, but getting the below on ESPN Plus:

2022/11/01 11:02:54.438653 [HLS] Starting live stream for channel 7000 from 192.168.50.1
2022/11/01 11:02:55.270834 [HLS] ffmpeg: ch7000-dANY-ip192.168.50.1-remux:  [hls @ 0x146808e00] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2022/11/01 11:02:55.353158 [HLS] Probed live stream in 912.946584ms: h264 1280x720 progressive 1762131bps
2022/11/01 11:02:58.984468 [HLS] ffmpeg: ch7000-dANY-ip192.168.50.1-remux:  [hls @ 0x146808e00] Non-monotonous DTS in output stream 0:1; previous: 540149, current: 536585; changing to 540150. This may result in incorrect timestamps in the output file.
2022/11/01 11:02:58.984732 [HLS] ffmpeg: ch7000-dANY-ip192.168.50.1-remux:  [hls @ 0x146808e00] Non-monotonous DTS in output stream 0:1; previous: 540150, current: 538505; changing to 540151. This may result in incorrect timestamps in the output file.
2022/11/01 11:03:03.714910 [HLS] Stopping transcoder session ch7000-dANY-ip192.168.50.1 (out: 10.996722s, finished: false)
2022/11/01 11:03:04.022601 [TNR] Closed connection to M3U-ESPNPlus for ch7000 EPlusTV 7000
#18

7 posts were merged into an existing topic: Discovery Channels Not Working

#19

No impact on login.

5 Likes
#20

Thank you, gentlemen! I'll give it a try today!

1 Like