In my experience, using this on in process nfl games from ESPN and fox, the commercial detection has no idea what’s going on for live events. I turned of auto commercial skip because it’s never right
I used it for in-progress recording of Saturday Night Live this weekend, and it worked pretty well. Not perfectly, but pretty close.
Yesterday I started watching the ALCS 30 minutes after the game started. The first two commercials were skipped correctly. After that, the playback would start a commercial skip while a player was still at bat. Had to back up manually and then skip the manually. I unchecked "Start live detection when watching in-progress recording " yet it continued to detect commercials for the rest of the recording which did not surprise me. I'm leaving live detection off till it's reported fixed and functioning correctly.
Thank you to the developers for trying to provide this feature.
@tmm1 I've found a bug with the implementation of this in-process detection feature. I've noticed the same pattern over the past several football games I've watched, and started tracking it much more closely yesterday and identified repeatable behaviors, and it seems to match the issues others are describing.
During playback of an in-process recording with in-process commercial detection, the first 3 or 4 commercials are skipped perfectly. Starting with about the 5th (or so) commercial, the commercial skip starts and finishes 10 seconds earlier than it should. Each subsequent commercial skip happens an additional 10 (or so) seconds too early. So the 6th commercial skip happens 20 seconds too early, the 7th commercial skip happens 30 seconds too early, the 8th commercial skip happens 40 seconds too early, etc. This basically makes the feature unusable, because significant portions of the game are skipped prior to commercial breaks, and commercial breaks are not skipped.
This has nothing to do with the accuracy of comskip in-process detection. I reran commercial detection after the recording completed and compared it to the original in-process .edl file. The commercial markers were exactly the same.
The problem seems to be with the app not properly calculating where it is in the playback progress. It seems to be getting the current playback position wrong after a few commercial skips, and then the issue keeps compounding.
This is happening with Windows DVR Server version 2022.10.03.2116 and the current Google Play client app version (running on an NVIDIA Shield.) The source is a HDHomeRun EXTEND OTA Antenna with the transcoding set to Heavy.
2 Likes
Interesting observation...
Can you compare the edl with the markers stored in the file json? Under View Details click the number under File ID
Could it be related to Initial video pts ?
OT: I never knew you could click on the file ID in Details and view the entire file object. Cool!
1 Like
Some recordings having regular comskip run post-recording show in log as using live commercial detection. I don't have that option enabled and I wasn't watching them during the recording.
Did a short recording from my HDHR Prime source to show the log entries (DVR v2022.10.24.2325)
2022/10/25 10:35:27.487901 [DVR] Starting job 1666717800-ch52 M*A*S*H on ch=[52]
2022/10/25 10:35:27.966714 [TNR] Opened connection to 1323AADB/0 for ch52 TVLANDP
2022/10/25 10:35:27.967038 [DVR] Recording for job 1666717800-ch52 from 1323AADB ch52 into "TV/MASH/MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg" for 9m32.511970397s
2022/10/25 10:35:28.405821 [IDX] Generating video index for job 1666717800-ch52
2022/10/25 10:38:46.698275 [ENC] Starting encoder for Quantum Leap S01E06 What a Disaster 2022-10-24-2159.mpg in /shares/dvr/Streaming/file5446-ip172.18.0.1-3339617002/encoder-0-264195017 at 0 (0.000000) (encoder=remux, resolution=, deinterlacer=hardware, bitrate=6573, segment_size=0.01)
2022/10/25 10:38:46.737751 [HLS] ffmpeg: file5446-ip172.18.0.1: [mpegts @ 0x8fde580] Dropped corrupted packet (stream = 2)
2022/10/25 10:38:46.737782 [HLS] ffmpeg: file5446-ip172.18.0.1: [mpegts @ 0x8fde580] Dropped corrupted packet (stream = 1)
2022/10/25 10:41:37.782141 [ENC] Stopped encoder for Quantum Leap S01E06 What a Disaster 2022-10-24-2159.mpg in /shares/dvr/Streaming/file5446-ip172.18.0.1-3339617002/encoder-0-264195017 after encoding 0 to 516
2022/10/25 10:45:00.002703 [TNR] Closed connection to 1323AADB/0 for ch52 TVLANDP
2022/10/25 10:45:00.096996 [SNR] Statistics for "TV/MASH/MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg": ss=95%-96% snq=100% seq=100% bps=1969870,374496-3960032 pps=189,62-351
2022/10/25 10:45:00.192559 [SNR] Buffer statistics for "TV/MASH/MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg": buf=0% drop=0%
2022/10/25 10:45:00.220986 [DVR] Finished job 1666717800-ch52 M*A*S*H
2022/10/25 10:45:00.369202 [DVR] Processing file-5447: TV/MASH/MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg
2022/10/25 10:45:00.561232 [DVR] Running live commercial detection on file 5447 (TV/MASH/MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg)
2022/10/25 10:45:24.805969 [DVR] Commercial detection for MASH S02E15 1973-12-22 Officers Only 2022-10-25-1035.mpg finished with 2 markers in 24.245062957s (live).
Settings from comskip log
[Live TV]
live_tv=0
live_tv_retries=6
require_div5=0
div5_tolerance=-1
incommercial_frames=1000
2 Likes
This was added a few days ago.
Thanks, should be fixed in next build.
4 Likes
I downloaded the latest server build and was able to compare the JSON and the .edl files. The time stamps matched the newer .eld file, and they were pretty close to the original in-process .edl file (generally within 2 seconds of one another.) They definitely were not off anywhere near the amount I was observing.
However...since I did re-detect the commercials to see if there was going to be a difference in the .edl files, maybe that corrected prior JSON errors. That perhaps could have overwritten the originally-observed discrepancy?
It is good to see the pattern detected. I look forward to a fix.
Thank you for all you do TMM1
I did a more thorough investigation of the in-process commercial detection while watching an NFL Football game. Once again, the commercial skips initiated earlier than they should, resumed too early (commercials still playing), and become progressively more offset with each commercial break.
I looked at the XML while the recording was happening and compared that to the XML after the recording complete. There was a substantial difference between the two, with the in-process time markers becoming progressively earlier. All of the data is below. Let me know if you need anything else.
Difference between in-process and post-process Commercials:
| XML (In-Process) | Difference (Post - InProcess) | XML (After Recording Complete) | EDL (After Recording Complete) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Commercial Start | Commercial End | Commercial Start | Commercial End | Commercial Start | Commercial End | Commercial Start | Commercial End | |||
| 4.07 | 75.41 | 4.07 | 11.68 | 0 | 63.73 | 0 | 63.73 | |||
| 707.87 | 827.63 | -0.04 | -2.14 | 707.91 | 829.77 | 707.91 | 829.77 | |||
| 1695.13 | 1815.11 | -17.48 | -4.14 | 1712.61 | 1819.25 | 1712.61 | 1819.25 | |||
| 2141.57 | 2270.9 | -5.85 | -8.61 | 2147.42 | 2279.51 | 2147.42 | 2279.51 | |||
| 2571.5 | 2671.7 | -9.21 | -10.38 | 2580.71 | 2682.08 | 2580.71 | 2682.08 | |||
| 2979.48 | 3120.25 | -12.08 | -13.12 | 2991.56 | 3133.37 | 2991.56 | 3133.37 | |||
| 3291.19 | 3426.56 | -44.51 | -16.12 | 3335.7 | 3442.68 | 3335.7 | 3442.68 | |||
| 3586.62 | 3695.32 | -16.52 | -19.63 | 3603.14 | 3714.95 | 3603.14 | 3714.95 | |||
| 3924.15 | 4044.47 | -19.13 | -20.69 | 3943.28 | 4065.16 | 3943.28 | 4065.16 | |||
| 4636.3 | 4756.38 | -54.46 | -25.93 | 4690.76 | 4782.31 | 4690.76 | 4782.31 | |||
| 4986.88 | 5115.51 | -26.9 | -30.17 | 5013.78 | 5145.68 | 5013.78 | 5145.68 | |||
| 5549.61 | 5671.43 | -25.8 | -32.77 | 5575.41 | 5704.2 | 5575.41 | 5704.2 | |||
| 6027.96 | 6279.64 | 6027.96 | 6279.64 | |||||||
| 6626.79 | 6758.56 | 6626.79 | 6758.56 | |||||||
| 7404.6 | 7526.32 | 7404.6 | 7526.32 | |||||||
| 8042.94 | 8174.8 | 8042.94 | 8174.8 | |||||||
| 8414.51 | 8555.38 | 8414.51 | 8555.38 | |||||||
| 8684.11 | 8805.97 | 8684.11 | 8805.97 | |||||||
| 8950.58 | 9087.92 | 8950.58 | 9087.92 | |||||||
| 9425.59 | 9547.34 | 9425.59 | 9547.34 | |||||||
| 10330.32 | 10436.86 | 10330.32 | 10436.86 | |||||||
| 11042.1 | 11198.86 | 11042.1 | 11198.86 | |||||||
| 12016.11 | 12168.16 | 12016.11 | 12168.16 |
XML (In-Process)
TV\NFL Football\NFL Football Pittsburgh Steelers at Philadelphia Eagles 2022-10-30-1300.mpg
"root":{
"ID":"2231"
"JobID":"1667149200-3"
"RuleID":"3"
"GroupID":"191277"
"Path":"TV\NFL Football\NFL Football Pittsburgh Steelers at Philadelphia Eagles 2022-10-30-1300.mpg"
"CreatedAt":1667149200
"PlaybackTime":5770
"Duration":0
"Commercials":[
0:
4.07
1:
75.41
2:
707.87
3:
827.63
4:
1695.13
5:
1815.11
6:
2141.57
7:
2270.9
8:
2571.5
9:
2671.7
10:
2979.48
11:
3120.25
12:
3291.19
13:
3426.56
14:
3586.62
15:
3695.32
16:
3924.15
17:
4044.47
18:
4636.3
19:
4756.38
20:
4986.88
21:
5115.51
22:
5549.61
23:
5671.43
]
"Airing":{
"Source":"tms"
"Channel":"4.1"
"Time":1667149200
"Duration":10800
"Title":"NFL Football"
"EventTitle":"Pittsburgh Steelers at Philadelphia Eagles"
"Summary":"The Eagles (6-0) come off their bye week to host the Keystone State rival Steelers (2-5)."
"FullSummary":"The undefeated Eagles come off their bye week to host the Keystone State rival Steelers. The last time Philadelphia started 6-0 was in 2004, a season capped with a trip to the Super Bowl. Pittsburgh (2-5) handed Philadelphia its first loss that year."
"Image":"https://tmsimg.fancybits.co/assets/p22189133_tb2_h9_aa.jpg?w=720&h=540"
"Categories":[
0:
"Sports"
1:
"Sports event"
]
"Genres":[
0:
"Football"
]
"Tags":[
0:
"CC"
1:
"HD 1080i"
2:
"HDTV"
3:
"Live"
4:
"New"
5:
"Stereo"
]
"SeriesID":"191277"
"ProgramID":"EP000031283437"
"TeamIDs":[
0:
"55"
1:
"54"
]
"Raw":{
"startTime":"2022-10-30T17:00Z"
"endTime":"2022-10-30T20:00Z"
"duration":180
"channels":[
0:
"29.1"
]
"stationId":"36147"
"qualifiers":[
0:
"CC"
1:
"HD 1080i"
2:
"HDTV"
3:
"Live"
4:
"New"
5:
"Stereo"
]
"program":{
"tmsId":"EP000031283437"
"rootId":"22189133"
"seriesId":"191277"
"entityType":"Sports"
"subType":"Sports event"
"title":"NFL Football"
"titleLang":"en"
"eventTitle":"Pittsburgh Steelers at Philadelphia Eagles"
"releaseYear":0
"releaseDate":""
"origAirDate":""
"descriptionLang":"en"
"shortDescription":"The Eagles (6-0) come off their bye week to host the Keystone State rival Steelers (2-5)."
"longDescription":"The undefeated Eagles come off their bye week to host the Keystone State rival Steelers. The last time Philadelphia started 6-0 was in 2004, a season capped with a trip to the Super Bowl. Pittsburgh (2-5) handed Philadelphia its first loss that year."
"topCast":NULL
"genres":[
0:
"Football"
]
"preferredImage":{
"uri":"https://tmsimg.fancybits.co/assets/p22189133_tb2_h9_aa.jpg?w=720&h=540"
"height":"540"
"width":"720"
"primary":"true"
"category":"Banner-L2"
"text":"yes"
"tier":"Team Event"
}
"organizationId":"20"
"sportsId":"111"
"teams":[
0:{
"teamBrandId":"55"
"name":"Pittsburgh Steelers"
}
1:{
"teamBrandId":"54"
"name":"Philadelphia Eagles"
"isHome":true
}
]
"gameDate":"2022-10-30"
"gameTime":"13:00:00-04:00"
"gameTimeZone":"Eastern Observing"
"season":{
"content":"2022-2023"
"type":"Reg"
}
}
}
}
"ChannelNumber":"4.1"
"DeviceID":"1051A8E6"
"PlayedAt":1667155020445
"UpdatedAt":1667155020445
"Version":6
"JobTime":1667149200
"JobDuration":12600
"CommercialsRequested":true
"CommercialDetectSource":"live"
}
XML (After Recording Completion)
TV\NFL Football\NFL Football Pittsburgh Steelers at Philadelphia Eagles 2022-10-30-1300.mpg
"root":{
"ID":"2231"
"JobID":"1667149200-3"
"RuleID":"3"
"GroupID":"191277"
"Path":"TV\NFL Football\NFL Football Pittsburgh Steelers at Philadelphia Eagles 2022-10-30-1300.mpg"
"CreatedAt":1667149200
"FileSize":20617337832
"PlaybackTime":11125
"Duration":12601.2046
"Commercials":[
0:
0
1:
63.730000000000004
2:
707.91
3:
829.77
4:
1712.6100000000001
5:
1819.25
6:
2147.42
7:
2279.51
8:
2580.71
9:
2682.08
10:
2991.56
11:
3133.37
12:
3335.7000000000003
13:
3442.6800000000003
14:
3603.14
15:
3714.9500000000003
16:
3943.28
17:
4065.1600000000003
18:
4690.76
19:
4782.31
20:
5013.78
21:
5145.68
22:
5575.41
23:
5704.2
24:
6027.96
25:
6279.64
26:
6626.79
27:
6758.56
28:
7404.6
29:
7526.32
30:
8042.9400000000005
31:
8174.8
32:
8414.51
33:
8555.380000000001
34:
8684.11
35:
8805.97
36:
8950.58
37:
9087.92
38:
9425.59
39:
9547.34
40:
10330.32
41:
10436.86
42:
11042.1
43:
11198.86
44:
12016.11
45:
12168.16
]
"Completed":true
"Processed":true
"Airing":{
"Source":"tms"
"Channel":"4.1"
"Time":1667149200
"Duration":10800
"Title":"NFL Football"
"EventTitle":"Pittsburgh Steelers at Philadelphia Eagles"
"Summary":"The Eagles (6-0) come off their bye week to host the Keystone State rival Steelers (2-5)."
"FullSummary":"The undefeated Eagles come off their bye week to host the Keystone State rival Steelers. The last time Philadelphia started 6-0 was in 2004, a season capped with a trip to the Super Bowl. Pittsburgh (2-5) handed Philadelphia its first loss that year."
"Image":"https://tmsimg.fancybits.co/assets/p22189133_tb2_h9_aa.jpg?w=720&h=540"
"Categories":[
0:
"Sports"
1:
"Sports event"
]
"Genres":[
0:
"Football"
]
"Tags":[
0:
"CC"
1:
"HD 1080i"
2:
"HDTV"
3:
"Live"
4:
"New"
5:
"Stereo"
]
"SeriesID":"191277"
"ProgramID":"EP000031283437"
"TeamIDs":[
0:
"55"
1:
"54"
]
}
"ChannelNumber":"4.1"
"DeviceID":"1051A8E6"
"PlayedAt":1667161690866
"UpdatedAt":1667161908372
"Version":6
"JobTime":1667149200
"JobDuration":12600
"SignalStats":{
"SS":{
"Initial":78
"Last":77
"Min":75
"Max":79
"Sum":481973
"GoodCount":6267
"BadCount":0
}
"SNQ":{
"Initial":85
"Last":90
"Min":81
"Max":90
"Sum":541869
"GoodCount":6267
"BadCount":0
}
"SEQ":{
"Initial":0
"Last":100
"Min":0
"Max":100
"Sum":626600
"GoodCount":6266
"BadCount":1
}
"BPS":{
"Initial":0
"Last":13480352
"Min":0
"Max":24888192
"Sum":82539464352
"GoodCount":6266
"BadCount":1
}
"PPS":{
"Initial":0
"Last":1153
"Min":0
"Max":1296
"Sum":7065705
"GoodCount":6266
"BadCount":1
}
"TSERR":{
"Initial":0
"Last":0
"Min":0
"Max":0
"Sum":0
"GoodCount":6267
"BadCount":0
}
"NETERR":{
"Initial":0
"Last":0
"Min":0
"Max":0
"Sum":0
"GoodCount":6267
"BadCount":0
}
"DVRBUF":{
"Initial":0
"Last":0
"Min":0
"Max":0
"Sum":0
"GoodCount":0
"BadCount":0
}
"DVRDROP":{
"Initial":0
"Last":0
"Min":0
"Max":0
"Sum":0
"GoodCount":0
"BadCount":0
}
"Grade":{
"GoodCount":6267
"BadCount":0
}
"NetworkGrade":{
"GoodCount":6267
"BadCount":0
}
"SignalGrade":{
"GoodCount":6267
"BadCount":0
}
}
"BufferStats":{
"BufferPct":{
"Initial":0
"Last":0
"Min":0
"Max":2
"Sum":4
"GoodCount":0
"BadCount":6288
}
"BufferDrop":{
"Initial":0
"Last":0
"Min":0
"Max":0
"Sum":0
"GoodCount":6288
"BadCount":0
}
}
"CommercialsAligned":true
"CommercialsRequested":true
"CommercialDetectSource":"live"
}
EDL File (After Recording Complete)
0 63.730000000000004 3
707.91 829.77 3
1712.6100000000001 1819.25 3
2147.42 2279.51 3
2580.71 2682.08 3
2991.56 3133.37 3
3335.7000000000003 3442.6800000000003 3
3603.14 3714.9500000000003 3
3943.28 4065.1600000000003 3
4690.76 4782.31 3
5013.78 5145.68 3
5575.41 5704.2 3
6027.96 6279.64 3
6626.79 6758.56 3
7404.6 7526.32 3
8042.9400000000005 8174.8 3
8414.51 8555.380000000001 3
8684.11 8805.97 3
8950.58 9087.92 3
9425.59 9547.34 3
10330.32 10436.86 3
11042.1 11198.86 3
12016.11 12168.16 3
2 Likes
Dumb question, but how do you get your comskip.ini file into Channels-DVR?
Check out this post earlier in the thread:
Any update on getting the in-process time stamps to align with the com skip time stamps?
@tmm1 I observed the same behavior with the commercial skips happening subsequently earlier as the recordin progressed. Let me know if there is anything more I can do to troubleshoot.
The in-process commercial markers in the XML being different than the after-recording markers and edl file.
The commercial were correct for about the first 70 minutes of the recording (watching while recording.) A little before halftime, the commercial skips starting happening progressively earlier as compared to the actual commercial positions.
I forced re-detection of commercials while the recording and the commercial skips still happened too early. I'm not sure if it's an issue with the markers being in the wrong position, or with payback timestamps not properly aligning. If there is anything else I can do to help identify the issue, please let me know.
The markers come from comskip in all cases. You can compare the edl file in the Logs folder while live detection is happening. It should match the JSON output exactly.
I'm not sure why comskip is returning such incorrect markers in live mode, or how it can be told to do a better job.
I think it’s odd that the results change after the recording completes (without me manually rerunning comskip.) My best guess is that as the recording progresses, comskip is detecting the commercials correctly, but the timestamps/restart detection position is getting out of sync with the Channels Playback position. It must be related to the way comskip “restarts” as the recording progresses.
To be clear, comskip detects every commercial break, and detects the proper duration, but the skips are starting too early and ending too early. And the offset compounds over time.
In-process comskip never had this issue with SageTV, so it must have something to do with how the recording files are written or how the timestamps interpreted. I’m guessing maybe the incremental detection spot is not restarting at the correct location due to something unique with the file.
I tried switching from MOEG4 output using one HDHR to MPEG2 on the other HDHR, and there was no difference.
Okay I am still curious to see what the EDL vs JSON look like while the recording is still happening. That will tell us if its a comskip problem or channels problem.