Commercial Detection for In-Process Recordings (Catch-up to live broadcast - especially useful for live sports/events)

Channels DVR Server Feature Requests
#134

Yeah, it's frustrating that the comskip time stamps drift for the in-process recording vs. post-processed recording. It's not that the commercials are completely missed or the wrong total duration, it's that the commercial start/end marker becomes progressively earlier as the recording progresses. It seems like there is an issue with how Channels appends the recording as it is being written to disk that causes comskip to view an incorrect timestamp as it scans the file.

I used the exact some comskip version and ini configuration with SageTV for the exact same in-process OTA recording, and SageTV had no issues getting the timestamps correct. There was no drift. There's something different about the Channels implementation that's causing this problem. If you playback the recording timestamps where the Channels in-process comskip markers are flagged, there is clearly nothing in the video feed at that time to trigger a detected commercial. No blank screen, no logo change, etc. Comskip is properly detecting the commercial in the in-process recording, but the time stamp it is associating with that detection position is incorrect.

@tmm1 Would you be willing to please look into this?

3 Likes
#135

And that is from years ago... And this is a forum where users of this service interact with other users and the devs from channels - so while I concur with your statement I don't get how it's germane here.

1 Like
#136

I do not think Channels DVR uses the EDL directly to comskip it's Recordings... I know SageTV does.

1 Like
#137

You're right, in-process Comskip isn't quite as accurate as post-process comskip and may completely miss commercials. However, that is different that then the timestamp drift issue being reported.

It's expected that in-process comskip may miss a commercial here or there (as it does with SageTV.) What's not expected that the time stamps drift for commercials that are detected and do not match the actual positions of the commercials that Comskip did detect. In-process timestamp drift is only an issue for Comskip when used with Channels. This is the issue I hope Channels developers can address.

1 Like
#138

Yes, you're correct, Channels stores the ComSkip markers in an XML file. In my very detailed post above providing data extracts documenting the problem, I pulled the in-process commercial markers from the Channels XML file while the recording was in-process.

#139

Nice analysis I missed that post thanks for pointing it out ...

I wonder what would happen if I run Comskip outside of channels on a recording when it starts creating the EDL.

I noticed Channels DVR does not use the EDL to comskip when recording so even though the EDL on my test contained markers Channels did not use it.

1 Like
#140

I am still curious to see these files while recording. Please take a screenshot or copy/paste so I can see them. If the issue happens after 90 mins then capture it at that point, but before the recording is over.

The comskip.log file would also be useful.

#141

Here's the Google Sheets where I imported and parsed the XML. There are 3 different tabs. One compare in-process SageTV EDL file vs. in-process Channels XML markers. The other tabs compare Channels in-process XML vs. post-process XML and post-process EDL.

#142

There are two files created in the channels Log directory called video.edl and video.log

I need to see those files

FYI nothing in channels uses XML. The format you're looking at is called JSON

I need to know what's in the json and what's the video.edl file and I need a copy of the video.log file

Details about what sagetv does are not helpful

#143

Sorry, it's been over a year since I captured that data and I've forgotten most of the the details. I'll pull the data and log files during the NFL games tonight.

#144

Here are the log files, taken while the recording was in-process. The first 1 or 2 commercials were detected at the correct positions. All of the subsequent skips started and ended earlier vs. the actual commercial positions in the recording (FYI, the recording started late due to me not having Channels DVR running.)

1 Like
#145

Okay the edl and json match. So the incorrect results are coming out of comskip.

I see some errors in the comskip log. Perhaps it's related.


Retry=0 at frame=10018, time=  335.10 seconds
Retry target pos=353133184, pts=4089579361
Retry t_pos=353133184, l_pos=350112776, t_pts=4089579361, l_pts=4089352354
Strange audio pts step of 332.64050 instead of 0.00000 at frame 10018
Strange video pts step of -2.13547 instead of 0.03337 at frame 10018
[mpeg2video @ 000000007faee640] Invalid mb type in P-frame at 9 44
[mpeg2video @ 000000007faee640] Warning MVs not available
[ac3 @ 000000007fc2e740] incomplete frame
[mpeg2video @ 000000007faee640] ac-tex damaged at 99 26
[mpeg2video @ 000000007faee640] Warning MVs not available

Retry=0 at frame=10155, time=  339.71 seconds
Retry target pos=357047344, pts=4089985441
Retry t_pos=357047344, l_pos=354579656, t_pts=4089985441, l_pts=4089757759
Strange audio pts step of 337.28050 instead of 0.00000 at frame 10155
Strange video pts step of -2.63597 instead of 0.03337 at frame 10155
[mpeg2video @ 000000007fd0de40] slice too small
[ac3 @ 000000002366cf00] incomplete frame

We use live_tv=1 in our comskip.ini

Maybe sagetv does something differently.

I guess you could try running sagetv with our comskip.exe to compare.

Commercial detection suddenly too slow to be useful
#146

This is what I use on SageTV ...

live_tv=1 ; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4 ; change to 16 when using live_tv in BTV, used for mpeg PS and TS

#147

Does the comskip log in sagetv show mpeg2video and ac3 errors?

#148

There are no mpeg errors in my SageTV comskip log file. It’s from the same SiliconDust HDHR source device.

#149

I hope there is something found to fix this issue.

I have often found that the in-process commercial accuracy is MUCH worse than the post-process detection, but this is just my observation playing recordings and I have no log details.

1 Like
#150

Any update on why the in-process commercial detection has timestamp drift?

#151

Just chiming in to say I am also eagerly awaiting this fix... :grimacing: :pray: :crossed_fingers:

1 Like
#152

As the Olympics approach, is there anything I can do to help identify the source of this problem?

#153

It's Football season again.

@tmm1 what can be done to identify the source of this "time stamp drift" bug? I think we've ruled-out comskip being the issue.