Commercial Detection for In-Process Recordings (Catch-up to live broadcast - especially useful for live sports/events)

I am still curious to see these files while recording. Please take a screenshot or copy/paste so I can see them. If the issue happens after 90 mins then capture it at that point, but before the recording is over.

The comskip.log file would also be useful.

Here's the Google Sheets where I imported and parsed the XML. There are 3 different tabs. One compare in-process SageTV EDL file vs. in-process Channels XML markers. The other tabs compare Channels in-process XML vs. post-process XML and post-process EDL.

There are two files created in the channels Log directory called video.edl and video.log

I need to see those files

FYI nothing in channels uses XML. The format you're looking at is called JSON

I need to know what's in the json and what's the video.edl file and I need a copy of the video.log file

Details about what sagetv does are not helpful

Sorry, it's been over a year since I captured that data and I've forgotten most of the the details. I'll pull the data and log files during the NFL games tonight.

Here are the log files, taken while the recording was in-process. The first 1 or 2 commercials were detected at the correct positions. All of the subsequent skips started and ended earlier vs. the actual commercial positions in the recording (FYI, the recording started late due to me not having Channels DVR running.)

1 Like

Okay the edl and json match. So the incorrect results are coming out of comskip.

I see some errors in the comskip log. Perhaps it's related.


Retry=0 at frame=10018, time=  335.10 seconds
Retry target pos=353133184, pts=4089579361
Retry t_pos=353133184, l_pos=350112776, t_pts=4089579361, l_pts=4089352354
Strange audio pts step of 332.64050 instead of 0.00000 at frame 10018
Strange video pts step of -2.13547 instead of 0.03337 at frame 10018
[mpeg2video @ 000000007faee640] Invalid mb type in P-frame at 9 44
[mpeg2video @ 000000007faee640] Warning MVs not available
[ac3 @ 000000007fc2e740] incomplete frame
[mpeg2video @ 000000007faee640] ac-tex damaged at 99 26
[mpeg2video @ 000000007faee640] Warning MVs not available

Retry=0 at frame=10155, time=  339.71 seconds
Retry target pos=357047344, pts=4089985441
Retry t_pos=357047344, l_pos=354579656, t_pts=4089985441, l_pts=4089757759
Strange audio pts step of 337.28050 instead of 0.00000 at frame 10155
Strange video pts step of -2.63597 instead of 0.03337 at frame 10155
[mpeg2video @ 000000007fd0de40] slice too small
[ac3 @ 000000002366cf00] incomplete frame

We use live_tv=1 in our comskip.ini

Maybe sagetv does something differently.

I guess you could try running sagetv with our comskip.exe to compare.

This is what I use on SageTV ...

live_tv=1 ; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4 ; change to 16 when using live_tv in BTV, used for mpeg PS and TS

Does the comskip log in sagetv show mpeg2video and ac3 errors?

There are no mpeg errors in my SageTV comskip log file. It’s from the same SiliconDust HDHR source device.

I hope there is something found to fix this issue.

I have often found that the in-process commercial accuracy is MUCH worse than the post-process detection, but this is just my observation playing recordings and I have no log details.

1 Like

Any update on why the in-process commercial detection has timestamp drift?

Just chiming in to say I am also eagerly awaiting this fix... :grimacing: :pray: :crossed_fingers:

1 Like

As the Olympics approach, is there anything I can do to help identify the source of this problem?

It's Football season again.

@tmm1 what can be done to identify the source of this "time stamp drift" bug? I think we've ruled-out comskip being the issue.

Ugh, this "timestamp drift" bug is so annoying. Time after time I'll be watching a game and just as a big play is happening the automatic commercial skip kicks in 30 seconds early and completely ruins the moment.

The fact that such an intrusive bug has been allowed to persist for so long just seems so antithetical to the sublime user experience that Channels has always been about. @tmm1 @maddox, this has been a known issue for at least two years (!), will this ever be fixed?

I resolved this issue. I turned it off and use the remote.

My feeling is that the developers gave this a try due to popular demand yet a fix is beyond their control so they should pull the feature.

1 Like

Just to be clear, commercial detection is third-party software not developed by the Fancy Bits team. It has been around long before Channels. While the Channels developers do make contributions to that project, it is not theirs.

(Even before live detection was added as an option to Channels it was noted that it was far less accurate and offered a less-than-great experience. I personally lobbied against adding the feature because I knew from personal experience how poor it is with other DVRs. While the Channels devs have made additions that have improved commercial detection, live detection is still alpha-quality.)

This is more avoiding the the issue than resolving it, but I get your point.

I conducted numerous A/B tests with SageTV and Channels recording the same show at the same time and both doing the same in-process commercial detection with the same ComSkip settings. I provided all of the results and log files. Every indication is that this is not a ComSkip problem, but rather an issue with the Channels playback positions. In-process detection worked as expected with SageTV (it occasionally missed a commercial, but all of the skips that were detected happened at the correct start/stop points.)

I think reduced accuracy of in-process commercial detection is a separate issue. It should not be confused with this playback position drift issue. This isn't about ComSkip detecting or not detecting all the commercials. It's about Channels not accurately jumping to the correct timestamps for the commercials that ComSkip did accurately detect.

For reference, in-process detection for SageTV didn't detect every commercial, but for the commercials it did detect, the skips happened at the correct times. For Channels, it detected the same commercials, but the timestamps in the comskip file do not align with where Channels is skipping from/to during playback. The results should be the same between two different DVRs recording the same show, from the same source, using the same ComSkip library (missed commercials and all should be the same.) This is unfortunately not the case, because the issue isn't about commercial detection. This a separate time drift issue with Channels.

Three hours into a 4 hour recording, I deleted the in-process ComSkip files and started in-process Comskip detection over, which should make the first ~2:45 minutes of the recording the same as post-processed ComSkip. It detected all of the commercials correctly, but the playback of the in-process recording still had playback position drift. Each skip happened earlier and earlier than when it should have as playback progresses through the recording.

I can't tell if it's an issue with how the Channels Server writes incremental chunks to the recording file that makes reading timestamps error-prone or if it's a problem with not accurately tracking the current playback position, or something else. Regardless, it seems to clearly be a Channels issue, not a ComSkip issue.

8 Likes

This is not a commercial detection problem, but rather a commercial skipping one.

The commercial start and end points are correctly identified by comskip as evidenced by the darker color on the playback progress bar, but Channels somehow gets confused and starts progressively skipping earlier and earlier the longer the recording goes. It is an issue with Channels. Please see @ChannelSurfer's post above

PS Big thanks to @ChannelSurfer for all the work identifying and documenting this :clap:

3 Likes

Sigh. Still no love.

1 Like