Deleting commercials from recordings on OSX

[pfile]

Hi all

i've been messing around with HTPC stuff for probably 15-20 years now, starting with MythTV and PCI tuner cards. a while back i switched to channelsDVR because i've been using an AppleTV since they first appeared some years ago.

does anyone have a flow (under OSX) to use the channels-generated .edl files to directly edit the mpeg2 transport streams from an HD Homerun?

if one were to delete the commercials in this way, would Channels DVR be upset that the file length is different after editing and would the .edl file hanging around confuse the client when it tries to draw the commercial start/end points on the timeline scrubber?

short of transcoding shows and moving them to the library, how do people handle this? i have a lot of wasted space taken up by commercials in my saved recordings...

thanks

[racameron]

Here's link to brief script I created using the ffsplit files. With a little extra work, and possibly some awk, I'm sure you could easily use EDLs.

Recording HBOMax movie into Channels DVR?

Here's a script I put together in the past using the ffsplit files in the comskip log directories: #!/bin/sh ffmpeg -i video.mpg $(cat video.ffsplit) for _f in segment*.ts do echo "file ${_f}" >> edit.txt done ffmpeg -f concat -i edit.txt -c copy edit.mpg _file=$(readlink video.mpg) mv ${_file} ${_file}.orig mv edit.mpg ${_file}

[pfile]

thanks for that, i will give that a try. i didn't realize ffmpeg could cut mpegts files. way back in the day i either used ProjectX or EyeTV for this. at that time all i ever knew how to use was mencoder and IIRC it couldn't cut mpeg2.

i'm sure i can massage the edl files into the mencoder edit list format one way or another.

i see in the comments for that thread that indeed channels does get confused with the shorter file. do you think that if you told channels DVR to redetect commercials in the in-place edited file that it might fix up the database's idea of the length of the file? do you know if channels is taking the commercial timeline info from the edl, or is it keeping it in the database? meaning if you deleted or cleared out the edl file that might at least take care of bogus commercial markers in the client without having to redetect commercials.

[racameron]

It might, but that's not really a supported configuration. But if you decide to try that, it might help others if you report back your experiences.

[pfile]

i'm trying this now. for simplicity i just went into the comskip directory for the recording i'm experimenting on and executed the relevant parts of your script manually, thus running on the autodetected commercials which were not quite right.

i then moved the original recording and its edl out of the way and symlinked the video file from the comskip area.

the scrubber still shows the commercials but of course the recording has none. this must mean that the edl is kept somewhere in a database rather than on disk. however, updating the edl in the channels interface did result in changes to the .edl sidecar file. so it's not the source for the edl info, but it does mirror what's in the database. i should have tried fast forwarding off the end of the shorter file to see what would happen.

now i'm regenerating the video index for the file and see what channels does with that. after that i'll try redetecting commercials.

[racameron]

@pfile

this must mean that the edl is kept somewhere in a database

Correct.

The best way to bring an edited file back into Channels is through a local content import. That way Channels won't try to apply the now incorrect information from the recording (file size, duration, commercials), but will use your new file, and put that in its library.

[pfile]

right but how is the metadata handled then? one thing i want to avoid is having to do Subler-style searches for all the metadata that is already in Channels' database. if i'm going to have to do that i might as well also transcode the files to hevc or something.

anyway: the update is that "regenerate video index" didn't seem to do anything and as expected if i try to scrub past the new end of the file the channels client just unceremoniously drops back to the menu. a pretty normal response to a short file!

comskip is currently running again. that will take a while as i foolishly chose a long sporting event as a test case!

[chDVRuser]

The commercial markers are held in the recorded file json for the recording.

IIRC, going from memory when I did it once before.
Edit commercials in the Channels UI, making everything program material (lots of clicking) and Save changes.
Then move your recording that was cut back over the original in Channels.
Then Refresh metadata in Channels which picks up the new cut duration.

Too much work for me, that's why I only did it once.
If there's a way to curl POST or PATCH the recorded file json to change the Commercials entry to null, that would be much easier.

[pfile]

ahh - ok - it's "refresh metadata" that does it. i didn't notice that menu option!

where is this json file that you're referring to stored?

now Channels thinks the recording is of the proper length. i had run comskip and it found a bunch of "commercials" which in the end i went in and marked as show. i'm watching something else right now but after that i'll go in to the show on the appleTV and check out what the scrubber looks like.

overall yes this process is a bit of a pain but at least it does appear to be possible. yes it would be nice to automate it but in the end i think i have to manually double-check comskip's work on every show, and that can't be avoided.

[chDVRuser]

@pfile

where is this json file that you're referring to stored?

The data is stored in the Channels DVR database.
You can read it from a web browser.
Say your recording File ID is 3639 (you can see this on the recordings View Details view)
Use your browser and go to
http://127.0.0.1:8089/dvr/files/3639
to display the recorded file json data

Of course you'll need to change 127.0.0.1 to the IP addresss of your DVR server.

[chDVRuser]

I've never used a MAC.

Comskip is not 100% accurate. Some shows yes, others are way off. It's VERY RARE for Channels DVR comskip to mark Program material as an AD, but it does happen. TVE recordings are the worst because of the way ads are inserted. More so if they're interrupted.

For the shows/movies I want to keep, I spend the time cutting ads and saving them outside of Channels DVR. I've been using my Video Editor for so long that it's become muscle memory to cut ads. I can open the raw recording from Channels DVR as soon as it's done recording and cut the ads way before comskip is done processing it. I've done this with 4 back-back recordings of a marathon series before comskip is done with the first one.

Personally, I wouldn't trust the comskip ad markers in an automated ad-cut flow...

[pfile]

i agree, if i wanted to automate this i would probably only do it for something like a game show or other throwaway content. otherwise i'll check what comskip has for me and fix it up.

it looks like ffmpeg is totally cool with mpeg2 ts files, so it's really just a matter of massaging a comskip-style edl file into a set of ffmpeg "start-duration" commands.

[beyo77]

I made a script for that works Well but afterwards you have to remember to do regenerate video index.
It is always good to check manually the ads after detection and mark them if something is omitted.

You could use wondershare converter pro. I have some old version but it has a problems with converting to mp4 preserving quality ( the highest option for resolution I see is 1280/720) - I reported bug to them and waiting for workaround.
Ffmpeg is good because if you mark precisely and use -c copy option it will cut the movie and join without reencoding HOWEVER

for some reason TVE streams in USA some have audio in HE-AAC. Ffmpeg cannot handle that audio with mpeg container just because.
No solution exists so I was forced to convert it to ac3 during split and join it does not take much time.