BETA: Smart Commercial Detection for TVE & Pluto

[speedingcheetah]

oops.

2021/12/22 20:05:00.034805 [MTS] Statistics for "TV/The Incredible Dr. Pol/The Incredible Dr. Pol Three Sled Dogs Gliding 2021-12-22-1859.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=11 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3967.267689

[tmm1]

Not necessary. It will work automatically

[chDVRuser]

Works even with Commercial Detection disabled.

BETA: Smart Commercial Detection for TVE & Pluto

Don't know if this is by design, but Smart Commercial Detection is happening even when the option [ ] Enable Commercial Detection is unchecked. Not like it's taking any resources, but just saying.

[speedingcheetah]

btw, goes with out saying....but....this new Smart Detection, taking no resources, is flipping awesome for those of us not using powerhouse hardware to run the server on. the RPi is a bit weak in that area. If this new thing becomes the norm, those who spent alot of $ on a powerful server, may be a bit peeved. lol. at least when it comes to commercial detection processes, transcodeing, who other thing.

is there some documentation on how this works vs comskip?

[chDVRuser]

It doesn't work for HDHR sources and not all TVE channels (yet).
I'm sure the devs aren't going to give away their secret sauce

[slampman]

@tmm1
I updated the server and then did the video indexing again. After that I noticed that moonshiners is now shown as 6min total length on appletv. The show info shows that as well.
It was fine until I did the regenerate index thing. The show info from the server shows the file size as 5gb so the data is there, it is just hidden

I submitted server diags.

[slampman]

@tmm1
This can be repeated on demand with other shows. This show was 60min but after doing the 2 tasks below the show now says 3 min for total length

2021/12/23 02:38:01.994484 [IDX] Generating video index for file-7449: Gold Rush White Water S05E07 2021-12-21 Gamblin Man 2021-12-22-0101.mpg
2021/12/23 02:38:14.074347 [IDX] Finished video index generation for file-7449 in 12s
2021/12/23 02:38:34.199345 [MTS] Rewriting MPEG-TS timestamps for file-7449: Gold Rush White Water S05E07 2021-12-21 Gamblin Man 2021-12-22-0101.mpg
2021/12/23 02:38:43.972966 [MTS] Statistics for #7449 "Gold Rush White Water S05E07 2021-12-21 Gamblin Man 2021-12-22-0101.mpg": skipped=12 unhandled_packets=0 discontinuity_detected=132 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=728.203744
2021/12/23 02:38:50.727500 [MTS] Finished video index generation for file-7449 in 16s
2021/12/23 02:38:50.782003 [IDX] Generating video index for file-7449: Gold Rush White Water S05E07 2021-12-21 Gamblin Man 2021-12-22-0101.mpg
2021/12/23 02:38:57.313314 [IDX] Finished video index generation for file-7449 in 6s

[Morris_Altman]

I'm thrilled to see this. Updated last night to v 2021.12.23.0210.

Recorded Red from TVE-Verizon for ch6055 SYFYP. Original commercial detection ran. It was perfect except for missing the last commercial break. I regenerated the video index and the result was the same though it ran much faster.

Log entries:
2021/12/23 00:00:00.011301 [DVR] Starting job 1640235600-ch6055 Red (2010) on ch=[6055]

2021/12/23 02:33:00.137423 [MTS] Statistics for "Movies\Red (2010) 2021-12-23-0000.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=1 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=9194.149900

2021/12/23 02:33:00.158516 [TNR] Closed connection to TVE-Verizon for ch6055 SYFYP

2021/12/23 02:33:00.195546 [DVR] Finished job 1640235600-ch6055 Red (2010)

2021/12/23 02:33:00.225829 [DVR] Processing file-765: Movies\Red (2010) 2021-12-23-0000.mpg

2021/12/23 02:33:01.403668 [DVR] Running commercial detection on file 765 (Movies\Red (2010) 2021-12-23-0000.mpg)

2021/12/23 02:46:20.983070 [DVR] Commercial detection for Red (2010) 2021-12-23-0000.mpg finished with 20 markers in 13m19.5874674s.

2021/12/23 10:01:55.380332 [IDX] Generating video index for file-765: Red (2010) 2021-12-23-0000.mpg

2021/12/23 10:03:05.220924 [IDX] Finished video index generation for file-765 in 69s

Thank you,

Morris

[chDVRuser]

@Morris_Altman

Recorded Red from TVE-Verizon for ch6055 SYFYP. Original commercial detection ran. It was perfect except for missing the last commercial break. I regenerated the video index and the result was the same though it ran much faster.

Your recording didn't "qualify" for Smart Commercial Detection, so Channels DVR used the old comskip method. Not all recordings will use the new method.

Regenerating the video index won't help in your case, as that only reruns the new Smart Commercial Detection which couldn't be used on your recording.

BETA: Smart Commercial Detection for TVE & Pluto

"Redetect Commercials" will run comskip "Regenerate Video Index" will run the smart detector again, but it only works on recordings made with the new prereleases.

[chDVRuser]

I would recommend only using Fix Video Timestamps as a last resort or when instructed to by Support. If you're going to run it on a recording, I would save off a copy of the recording file first.

As part of the process, Fix Video Timestamps automatically runs Regenerate Video Index after it rewrites timestamps, so no need to run it again.

[Morris_Altman]

@tmm1

new prereleases.

The recording was made with the new release. I don't understand why it would not qualify as it is recorded from TVE.

[tmm1]

Not all TVE channels are supported yet.

@tmm1

some streaming channels

@tmm1

tested this with some Discovery channels on TVE

@tmm1

The detector is not based on any channel number. It is part of the streaming code and will kick in whenever any of the ad-detect patterns match.

@tmm1

should catch the placeholders on many TVE channels, including all the Discovery and Viacom ones

@chDVRuser

not all TVE channels (yet)

@tmm1

I also found another technique that might work on all the nbc.com/live channels, and perhaps others as well. Will experiment more with that tomorrow.

[tmm1]

Send me an affected recording

[chDVRuser]

Whatever you did for Pluto in v2021.12.23.0416 is too aggressive.
It's now marking show segments as ads.
Too much work for me to get the Channels DVR markers from the recorded file json and put them in my video editor project file to keep checking test recordings. I can cut all ads manually much faster with my video editor.
Best I can do is use my video editor to get the correct show & ad timestamps.

[slampman]

@tmm1 here you go

Gold Rush White Water S05E07 2021-12-21 Gamblin Man 2021-12-22-0101.mpg (video)

[Channels2021]

Somehow this update effected my recording although I have always had commercial detection disabled. My playback device is Kodi with the Channels add-on. It shows red seek areas where it thinks the commercials are but they aren't correct. This is on a local FOX channel using TVE.

[chDVRuser]

Are you using .EDL files generated by Channels DVR for Kodi?
The new Smart Commercial Detection does generate .EDL files and works even if you have commercial detection disabled in your DVR settings.

[Channels2021]

I don't see an .edl file in the folder of the effected recording.

[Channels2021]

It was the 6 o'clock news and during the sports, it jumped forward and Kodi showed a commercial notification. I skiped back to the sports and was able to continue watching.

[chDVRuser]

I was able to figure out a simple way to use the Channels DVR generated EDL files from the new Smart Commercial Detection to create a VideoRedo VPrj project file to see the commercial markers in VideoRedo. Saves me a lot of time vs. copy and paste json and cleanup and multiplying timestamps and copy and paste again. Just a spreadsheet that accepts a paste of the EDL file contents and creates the VPrj file contents. Example here Convert Channels DVR .edl file to VideoRedo .VPrj file

Long story short - I don't need to run comskip on these to have it create the VPrj file.