The markers come from comskip in all cases. You can compare the edl file in the Logs folder while live detection is happening. It should match the JSON output exactly.
I'm not sure why comskip is returning such incorrect markers in live mode, or how it can be told to do a better job.
I think it’s odd that the results change after the recording completes (without me manually rerunning comskip.) My best guess is that as the recording progresses, comskip is detecting the commercials correctly, but the timestamps/restart detection position is getting out of sync with the Channels Playback position. It must be related to the way comskip “restarts” as the recording progresses.
To be clear, comskip detects every commercial break, and detects the proper duration, but the skips are starting too early and ending too early. And the offset compounds over time.
In-process comskip never had this issue with SageTV, so it must have something to do with how the recording files are written or how the timestamps interpreted. I’m guessing maybe the incremental detection spot is not restarting at the correct location due to something unique with the file.
I tried switching from MOEG4 output using one HDHR to MPEG2 on the other HDHR, and there was no difference.
Okay I am still curious to see what the EDL vs JSON look like while the recording is still happening. That will tell us if its a comskip problem or channels problem.
I’ll make it happen. Thanks for the feedback.
I compared the video.edl file in the logs directory to the XML in the recording details during the recording. The time markers match exactly between the two.
I'm not sure if this is related, but the Streaming Index is showing "Out of date". I wonder if time stamps are becoming out-of-sync during muxing.
@tmm1
I did a very detailed comparison of in-process commercial detection last night (during Sunday Night Football.) I recorded the show using both SageTV and Channels DVR, and ran in-process commercial detection for both. Both recorded OTA from the same Silicon Dust HDHR Connect Quatro (different tuners.) I then re-watched the show and manually checked all commercial markers. The detailed results are below.
The quick summary is this: both performed about the same for the first 90 minutes. Neither was perfect at detecting commercial breaks, but all detected commercial skips occurred at the correct time. This is as expected for in-process recordings.
However, at about the 90 minute mark, Channels loses its mind. Channels started skipping commercials early and returning to video before the commercials were over. It's important to note that SageTV worked properly during this time, and continued to detect commercial breaks at the time they actually occurred.
What I'm seeing is that after the 90-minute mark, Channels Comskip is detecting the commercials, but the time stamps in the recording that Comskip is reading are not accurate. This causes Comskip to mark a commercial at a time that does not correspond the Channels playback time. After Channels re-runs Comskip on the completed recording, the offset timestamps for Commercials 14, 15. and 16 are corrected and match the SageTV timestamps.
Perhaps this is due to errors in the broadcast signal? Or the way segments are appended to the recording file? Whatever it is, the timestamps of the in-process Channels video file become corrupt, at least from a ComSkip perspective. Re-running ComSkip on the in-process recording results in the same offset commercial breaks mid-recording, which again points to some type of corruption of the in-process time stamps ComSkip is reading. The fact that SageTV does not have this problem when using the same source seems to narrow it down to a Channels issue.
I'm not sure what further information I can provide or anything else I can try. Please let me know if you think of anything.
| Com# | Channels Start | Channels End | SageTV Start | SageTV End | Start Difference | End Difference | Notes |
|---|---|---|---|---|---|---|---|
| 1 | 103.37 | 280.68 | 102.37 | 279.51 | -1 | -1.17 | Both correct |
| 2 | 1116.62 | 1238.3 | 1115.68 | 1236.54 | -0.94 | -1.76 | Both correct |
| 3 | 1655.65 | 1791.26 | 1654.55 | 1790.19 | -1.1 | -1.07 | Both correct |
| 4 | 2201 | 2231.83 | Sage Properly Detected, Channels Missed | ||||
| 5 | 2410.61 | 2506.84 | 2409.51 | 2505.67 | -1.1 | -1.17 | Both correct |
| 6 | 2866.06 | 2986.98 | 2865.16 | 2986.12 | -0.9 | -0.86 | Both correct |
| 7 | 3218.48 | 3338.94 | 3217.31 | 3337.9 | -1.17 | -1.04 | Both correct |
| 8 | 3803.6 | 3924.59 | 3802.67 | 3923.49 | -0.93 | -1.1 | Both correct |
| 9 | 4105 | 4225 | 4105 | 4225 | Both Missed | ||
| 10 | 4456.35 | 4591.09 | Channels Properly Detected, Sage Missed | ||||
| 11 | 5022.28 | 5176.44 | 5027.09 | 5182.51 | 4.81 | 6.07 | Channels Too Early, Sage is Correct |
| 12 | 5419.08 | 5752.68 | 5426.69 | 5467.09 | 7.61 | -285.59 | Channels Too Early and Missed Show, Sage is Correct |
| 13 | 5532.73 | 5708.54 | Channels Missed, Sage is Correct | ||||
| 14 | 6006.83 | 6139.1 | 6024.18 | 6159.95 | 17.35 | 20.85 | Channels Too Early, Sage is Correct |
| 15 | 6422.88 | 6559.02 | 6443.2 | 6509.64 | 20.32 | -49.38 | Channels Too Early, Sage is Correct |
| 16 | 6931.52 | 7066.29 | 6956.98 | 7092.65 | 25.46 | 26.36 | Channels Too Early, Sage is Correct |
I haven't used it in a while because of the issue but I'll note when Channels started to get off the mark for commercials it seemed progressively worse almost like it was a compounding issue (not sure if logarithmic is the right term in this context but it's coming to mind for some reason).
This is still a problem. From what I can tell, based on the info posted above, this isn’t a problem with Comskip itself or detecting commercial markers. It seems to be an issue with timestamp drift of the in-process recording itself, that compounds as the recording progresses and/or more commercials are detected.
This is 100% still a problem.
As someone who subs to channels DVR almost exclusively to record sports OTA and watch games delayed without commercials it's super frustrating how consistently bad live com skip is.
At this point when I watch something still in progress I have to disable even Skip Button because that thing is such a spoiler when the time drifts off, which it seems to do on every single game.
(a completed game is usually super solid, but in progress is wackadoo)
The first response in this thread was that Comskip in a live context is less accurate, and using it could lead to a poor user experience.
Yeah, it's frustrating that the comskip time stamps drift for the in-process recording vs. post-processed recording. It's not that the commercials are completely missed or the wrong total duration, it's that the commercial start/end marker becomes progressively earlier as the recording progresses. It seems like there is an issue with how Channels appends the recording as it is being written to disk that causes comskip to view an incorrect timestamp as it scans the file.
I used the exact some comskip version and ini configuration with SageTV for the exact same in-process OTA recording, and SageTV had no issues getting the timestamps correct. There was no drift. There's something different about the Channels implementation that's causing this problem. If you playback the recording timestamps where the Channels in-process comskip markers are flagged, there is clearly nothing in the video feed at that time to trigger a detected commercial. No blank screen, no logo change, etc. Comskip is properly detecting the commercial in the in-process recording, but the time stamp it is associating with that detection position is incorrect.
@tmm1 Would you be willing to please look into this?
And that is from years ago... And this is a forum where users of this service interact with other users and the devs from channels - so while I concur with your statement I don't get how it's germane here.
I do not think Channels DVR uses the EDL directly to comskip it's Recordings... I know SageTV does.
You're right, in-process Comskip isn't quite as accurate as post-process comskip and may completely miss commercials. However, that is different that then the timestamp drift issue being reported.
It's expected that in-process comskip may miss a commercial here or there (as it does with SageTV.) What's not expected that the time stamps drift for commercials that are detected and do not match the actual positions of the commercials that Comskip did detect. In-process timestamp drift is only an issue for Comskip when used with Channels. This is the issue I hope Channels developers can address.
Yes, you're correct, Channels stores the ComSkip markers in an XML file. In my very detailed post above providing data extracts documenting the problem, I pulled the in-process commercial markers from the Channels XML file while the recording was in-process.
Nice analysis I missed that post thanks for pointing it out ...
I wonder what would happen if I run Comskip outside of channels on a recording when it starts creating the EDL.
I noticed Channels DVR does not use the EDL to comskip when recording so even though the EDL on my test contained markers Channels did not use it.
I am still curious to see these files while recording. Please take a screenshot or copy/paste so I can see them. If the issue happens after 90 mins then capture it at that point, but before the recording is over.
The comskip.log file would also be useful.
Here's the Google Sheets where I imported and parsed the XML. There are 3 different tabs. One compare in-process SageTV EDL file vs. in-process Channels XML markers. The other tabs compare Channels in-process XML vs. post-process XML and post-process EDL.
There are two files created in the channels Log directory called video.edl and video.log
I need to see those files
FYI nothing in channels uses XML. The format you're looking at is called JSON
I need to know what's in the json and what's the video.edl file and I need a copy of the video.log file
Details about what sagetv does are not helpful
Sorry, it's been over a year since I captured that data and I've forgotten most of the the details. I'll pull the data and log files during the NFL games tonight.