BETA: Smart Commercial Detection for TVE & Pluto

Can you give Fox News a shot?

Thanks!

@tmm1 I checked other episodes of cops and it looks like the commercial detection isn't working now. When I play the episode it shows no markers. The logs show some detected but the time line on the show looks like nothing was detected.

I just submitted server logs

I had a similar experience with a couple of random CNNGo shows last night (channel 6030 for me). Both showed no commercial markers, played right through the static "commercial" screens, and both said "Verified" when I tried to verify commercials. From the logs, it seems like it should have worked:

2021/12/21 22:00:01.438331 [MTS] Statistics for "TV/CNN Tonight/CNN Tonight S2021E12 2021-12-21 2021-12-21-2123.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=0 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=2232.997333
2021/12/21 22:00:01.438365 [ADS] Detected 4 commercials in file-7425 TV/CNN Tonight/CNN Tonight S2021E12 2021-12-21 2021-12-21-2123.mpg
2021/12/21 22:00:02.552849 [DVR] Finished job 1640149200-ch6030 CNN Tonight
2021/12/21 22:00:04.123173 [DVR] Waiting 20h59m55.876837596s until next job 1640228400-31 It's Always Sunny in Philadelphia
2021/12/21 22:00:04.200582 [DVR] Processing file-7425: TV/CNN Tonight/CNN Tonight S2021E12 2021-12-21 2021-12-21-2123.mpg
2021/12/21 22:00:04.746848 [TVE] stream timestamps: cnn: start_at=2021-12-21T21:49:53-08:00 current_at=2021-12-21T21:59:34-08:00 end_at=2021-12-21T21:59:46-08:00
2021/12/21 22:00:04.747633 [TNR] Opened connection to TVE-YouTubeTV for ch6030 CNN
2021/12/21 22:00:04.747954 [DVR] Recording for job 1640152800-ch6030 from TVE-YouTubeTV ch6030 into "TV/Don Lemon Tonight/Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg" for 59m59.572700306s
2021/12/21 22:00:08.570590 [DVR] Refreshing metadata for Don Lemon Tonight (19956970)
2021/12/21 22:00:12.526217 [IDX] Generating video index for job 1640152800-ch6030
2021/12/21 23:00:00.031707 [TNR] Closed connection to TVE-YouTubeTV for ch6030 CNN
2021/12/21 23:00:00.903844 [MTS] Statistics for "TV/Don Lemon Tonight/Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg": skipped=188 unhandled_packets=0 discontinuity_detected=0 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3608.987333
2021/12/21 23:00:00.903891 [ADS] Detected 4 commercials in file-7426 TV/Don Lemon Tonight/Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg
2021/12/21 23:00:01.723422 [DVR] Finished job 1640152800-ch6030 Don Lemon Tonight

Please submit diagnostics

Sure, f7b80254-dce5-4c31-8660-c683adf60e45. I ran "Redetect Commercials" (comskip) on it later and it failed:

21/12/21 23:00:03.293249 [DVR] Processing file-7426: TV/Don Lemon Tonight/Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg
2021/12/22 07:55:42.111397 [DVR] Running commercial detection on file 7426 (TV/Don Lemon Tonight/Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg)
2021/12/22 08:10:44.536145 [DVR] Commercial detection failed for Don Lemon Tonight 2021-12-21 2021-12-21-2200.mpg with exit status 1

I can't see what happened with that recording since you forced comskip on it.

What about the CNN Tonight one? It seems to have commercial markers set. They're not showing up in the player?

Wow! Can't beat this new ad detect time, it's instant.
Tested with a recording from DIY.
I verified the accuracy by plugging the adskip times from the recorded file json into my video editor.
It missed the static image blocks (Commercial break in progress. Your live stream will resume shortly.), but otherwise nailed the rest of the ads within a few video frames.

Since I rely on comskip creating a project file for my video editor, I'm glad Redetect Commercials is still available.

1 Like

Great!

I plan to improve things to catch the static image blocks as well.

2 Likes

Mine looks like it is detecting OK but the player does not show any markers. I just submitted diags in case its of any help

2021/12/22 12:30:00.008668 [DVR] Recording for job 1640194200-63 from M3U-PlutoTV ch9115 into "TV/COPS/COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg" for 30m29.991545899s
2021/12/22 12:30:00.059082 [IDX] Generating video index for job 1640194200-63
2021/12/22 13:00:30.023509 [MTS] Statistics for "TV/COPS/COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=481 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=30758.674311
2021/12/22 13:00:30.023521 [ADS] Detected 4 commercials in file-7472 TV/COPS/COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg

Considering I put in this feature request nearly 2 years ago, I'm delighted this is possible now!

@tmm1 I haven't had time to check myself, did you activate this on HLN and CNN International too?

2 Likes

Will running a redirect change to the new detection? Thanks.

"Redetect Commercials" will run comskip

"Regenerate Video Index" will run the smart detector again, but it only works on recordings made with the new prereleases.

1 Like

I recorded something off CNN with this morning's build and all the commercials were marked and showed up on the timeline on iOS.

2021/12/22 11:01:00.107498 [MTS] Statistics for "TV/CNN Newsroom With Ana Cabrera/CNN Newsroom With Ana Cabrera 2021-12-22-1016.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=0 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=2695.248133
2021/12/22 11:01:00.125013 [ADS] Detected 10 commercial markers in file-5829: TV/CNN Newsroom With Ana Cabrera/CNN Newsroom With Ana Cabrera 2021-12-22-1016.mpg
2021/12/22 11:01:00.251110 [DVR] Finished job 1640196000-ch6030 CNN Newsroom With Ana Cabrera
1 Like

Thanks, I see what's happening in the diagnostics.

Can you upload that recording too?

Then, after, try doing a "Regenerate Video Index" to see if it fixes it.

If not, then try "Fix Video Timestamps"

sure COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg - Google Drive

@tmm1 -- uh oh. I sent diags from the server as well. No change on the client after doing both of those tasks

2021/12/22 16:06:16.804897 [IDX] Generating video index for file-7472: COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg
2021/12/22 16:06:18.818843 [IDX] Finished video index generation for file-7472 in 2s
2021/12/22 16:07:02.720830 [MTS] Rewriting MPEG-TS timestamps for file-7472: COPS S32E17 2019-10-07 Cracking the Code 2021-12-22-1230.mpg


2021/12/22 16:07:03 [Recovery] 2021/12/22 - 16:07:03 panic recovered:
runtime error: index out of range [0] with length 0
runtime/panic.go:90 (0x433a74)
github.com/fancybits/channels-server/mpegts/stream_description.go:512 (0x7f06c8)
github.com/fancybits/channels-server/mpegts/rewriter.go:449 (0x7e8b75)
github.com/fancybits/channels-server/mpegts/rewriter.go:225 (0x7e7a64)
github.com/fancybits/channels-server/mpegts/rewriter.go:204 (0x122a164)
github.com/fancybits/channels-server/dvr/file.go:604 (0x122a152)
github.com/fancybits/channels-server/dvr/file.go:639 (0x122a909)
github.com/fancybits/channels-server/http_dvr_file.go:265 (0x1335b71)
github.com/gin-gonic/[email protected]/context.go:165 (0x130d503)
github.com/fancybits/channels-server/http_dvr_file.go:34 (0x130d437)
github.com/gin-gonic/[email protected]/context.go:165 (0x130630b)
github.com/fancybits/channels-server/http.go:235 (0x1306077)
github.com/gin-gonic/[email protected]/context.go:165 (0x1305fea)
github.com/fancybits/channels-server/http.go:215 (0x1305fcf)
github.com/gin-gonic/[email protected]/context.go:165 (0x1305e51)
github.com/fancybits/channels-server/http.go:207 (0x130567d)
github.com/gin-gonic/[email protected]/context.go:165 (0x9b7a41)
github.com/gin-gonic/[email protected]/recovery.go:106 (0x9b7a2c)
github.com/gin-gonic/[email protected]/context.go:165 (0x9b6ca6)
github.com/gin-gonic/[email protected]/logger.go:241 (0x9b6c89)
github.com/gin-gonic/[email protected]/context.go:165 (0x12b836c)
github.com/gin-contrib/[email protected]/sessions.go:65 (0x12b8358)
github.com/gin-gonic/[email protected]/context.go:165 (0x12b2569)
github.com/gin-contrib/[email protected]/gzip.go:47 (0x12b253f)
github.com/gin-gonic/[email protected]/context.go:165 (0x131ed21)
github.com/fancybits/channels-server/http.go:277 (0x131ed0b)
github.com/gin-gonic/[email protected]/context.go:165 (0x9b61dd)
github.com/gin-gonic/[email protected]/gin.go:420 (0x9b5e65)
github.com/gin-gonic/[email protected]/gin.go:376 (0x9b59c4)
net/http/server.go:2879 (0x6cf8da)
net/http/server.go:1930 (0x6caf87)
runtime/asm_amd64.s:1581 (0x46a4c0)

Thanks!

Fix uploading now. After upgrading, click Regenerate Video Index and it should make the commercials work on that COPS recording.

1 Like

I recorded another hour of CNN this morning and it showed up with commercial markers and properly skipped them. I then went back to the "CNN Tonight" recording from last night and now it has markers and is skipping the commercial breaks. It looks like it's working now!

1 Like

@tmm1 no joy. I did the regenerate video index and they still don't show. I did notice that the end of time line shrank slightly when I went back into the show. Almost like it recognized the end of the show but the middle is still missing all markers

Hmm. Maybe that panic earlier broke something. Try running Rewrite Video Timestamps again first, then do the Regenerate Index.

2 Likes