Using LosslessCut LLC file to update CDVR commercial skip markers

#1

Happy Holidays CDVR people!

I'm hoping to be able to use LosslessCut LLC files to fix commercial skip points for files that CDVR's built-in commercial editor can't fix. It's fairly straightforward to create an LLC that only contains chapter markers for commercial breaks -- very similar to an EDL file.

They look like this:

{
  version: 1,
  mediaFileName: 'The Closer S01E09 2005-08-08 Good Housekeeping 2023-11-23-1900.mpg',
  cutSegments: [
    {
      start: 0,
      end: 23.52,
      name: '',
    },
    {
      start: 627.693733,
      end: 823.823,
      name: '',
    },
    {
      start: 1378.610567,
      end: 1542.774567,
      name: '',
    },
    {
      start: 1905.57,
      end: 2160.86,
      name: '',
    },
    {
      start: 2435.47,
      end: 2646.94,
      name: '',
    },
    {
      start: 2968.47,
      end: 3268.87,
      name: '',
    },
  ],
}

The above represents a fixed set of skip segments. The original EDL file for the same episode, for reference, looks like this:

0	23.52	3
1905.57	2160.86	3
2435.47	2646.94	3
2968.47	3268.87	3

So, I can clear all of the Comskip data for a given CDVR FileID using:
curl -XPOST "http://media-server6:8089/dvr/files/$fileID/comskip/edit?source=local" --data-raw "[]"
Is there something similar I can do to replace the current skip points, and would anything else be necessary besides that?

I have a basic script worked up to be able to turn the LLC into an EDL, which I imagine I'd need to do to keep the embedded skip points matched up with the EDL.

#2

Simple. Change the raw data your POSTing "[]" to the start and end values you want
"[start,end,start,end,...]"

"[0,23.52,627.693733,823.823,1378.610567,1542.774567,1905.57,2160.86,2435.47,2646.94,2968.47,3268.87]"

If you have enabled EDL Export Integration, this will also update your EDL file next to the recording

2 Likes
#3

This makes me wonder, if there is any data you could grab from Plex or Emby - for intro or credit markers for TV shows and Movies, and put them into CDVR as ‘commercials’.

I’m sure someone clever could make a script to automate that for all the files that match.

#4

Wow does this ever work nicely! Credit to @chDVRuser for pointing me in the right direction.

I've now added this capability to OliveTin-for-Channels, and what this does is allow you to fix commercial skipping issues that can't be dealt with via the excellent CDVR "Edit Commercials" feature. It requires that you have EDL export integration enabled in Settings.

Here are some details:

Say you have one or more recorded shows, that have the "Interrupted" flag on them, but appear to be fine otherwise. This can result in issues with commercial skipping, that can't be fixed with a simple check or uncheck in the CDVR editor. And they might look something like this:

screenshot-media-server6_8089-2023.12.26-01_35_50
screenshot-media-server6_8089-2023.12.26-01_35_501920×1050 102 KB

As you can see, there are commercials buried in blocks, with no way to fix them in this editor. LosslessCut to the rescue! Without actually making any changes to the video file itself, you can open the video and import the companion EDL file and modify the commercial breaks -- working with them as "chapters":

Screenshot 2023-12-26 010343
Screenshot 2023-12-26 0103431920×1200 145 KB

At this point, the main thing you want to check are the commercial breaks as they exist currently. So, click on the timeline within those colored chapters and make sure that each is actually all commercial. I like to click near the beginning, middle and end of this size chapter. Once you've done that, you want to "invert" the colored segments, using this dropdown:

Screenshot 2023-12-26 010420
Screenshot 2023-12-26 0104201920×1200 153 KB

After that, the show segments will become the colored blocks. Now you can start clicking on various points in the timeline, to determine where undetected commercials are located. There are forward and back keyframe buttons, and also one frame at a time buttons when you're closing in on the exact spot for the beginning of a commercial.

When you find the beginning of a commercial, use the "Split segment at cursor" button to set the beginning of a new segement. Do the same for the end of the commercial. Once the new segment is created, make sure it's the selected segment in the right-hand "Segments to export" area, and click the minus sign at the bottom of that area.

Screenshot 2023-12-26 010936
Screenshot 2023-12-26 0109361920×1200 118 KB

Once you have the segments correctly defined, you'll need invert the segments one last time so it's once again the commercials that are the colored blocks. From there you can close LosslessCut, and there's no need to save anything as LLC automatically updates the file containing the segment information whenever you make changes.

Next you'll need to run the new OliveTin Action "Update Commercials Metadata from LosslessCut LLC File", and for that you'll need the "File ID" for the recording you just worked on. The File ID can be found multiple places, but for this project right in the "View Details" option from that same dropdown you used to get to "Edit Commercials" is a good place to find it. It's also shown at the top of the commercial editing dialog.

Run the OliveTin Action using that ID, and then check the Log for details. The companion EDL file will be updated automatically too.

screenshot-htpc6-2023.12.26-05_12_00

Lastly, you need to go back into the CDVR commercial editor, admire your handiwork, and then save at the bottom:

screenshot-media-server6_8089-2023.12.26-01_58_13
screenshot-media-server6_8089-2023.12.26-01_58_131920×1050 97.3 KB

No video files were harmed during the making of this write-up. :slight_smile:

2 Likes
OliveTin for Channels: An Interface for Misc Channels DVR Scripts & Tricks
Issue with trimming a recorded tv program
#5

:joy: Great line!

And great job, @bnhf, with help from @chDVRuser.

I have had EDL export enabled like forever but I have never done anything with them... Until now.
This is cool, I was always hoping for a way to be able to edit the cut points without having to actually touch the videos.

I will start doing this soon.

Thank you for sharing!

#6

Script seems to get confused where things are on my *ix config but the llc is where the recording is:

  • recordingName=TV/CBS News Texas at 500 AM/CBS News Texas at 500 AM 2023-12-27-0500.mpg
  • recordingLLC=TV/CBS News Texas at 500 AM/CBS News Texas at 500 AM 2023-12-27-0500-proj.llc
  • cat /mnt/192.168.0.12-8089/TV/CBS News Texas at 500 AM/CBS News Texas at 500 AM 2023-12-27-0500-proj.llc
    cat: '/mnt/192.168.0.12-8089/TV/CBS News Texas at 500 AM/CBS News Texas at 500 AM 2023-12-27-0500-proj.llc': No such file or directory
#7

The most likely thing is an issue with the way that you've bound your shared "dvr" directory to the OliveTin container.

Are you binding an NFS or SMB share that you setup through Portainer-Volumes, or is this a directory that's available on your Docker host and you're able to bind it directly? If you could post the bottom section of your compose here including the bindings, and any volumes you're defining, that would be helpful.

EDIT: If it's a directory that's available on your Docker host (as is the case for me using Proxmox) then you bind that directory to the container. Otherwise, if it's a network share, you need to create a Docker Volume using Portainer-Volumes which looks like this in Portainer:

screenshot-htpc6-2023.12.27-12_44_57
screenshot-htpc6-2023.12.27-12_44_571920×1050 80.9 KB

And using this style of mount, the name you choose for the volume gets bound to the container. So, if your Docker Volume is called channels-dvr, then uncomment the lines in the very bottom "volumes" section. And the binding itself would look like this:

    volumes:
      - ${HOST_DIR}/olivetin:/config # Add the parent directory on your Docker you'd like to use
      - ${DVR_SHARE}:/mnt/192.168.0.12-8089 # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR network share
    restart: unless-stopped
volumes: # use this section if you've setup a docker volume named channels-dvr, with CIFS or NFS, to bind to /mnt/dvr inside the container
  channels-dvr:
    external: true

and the environment variable for DVR_SHARE would be:

DVR_SHARE=channels-dvr

Or, if it's a directory that's available directly on your Docker host -- for example, /media/dvr, then your compose would look like this:

    volumes:
      - ${HOST_DIR}/olivetin:/config # Add the parent directory on your Docker you'd like to use
      - ${DVR_SHARE}:/mnt/192.168.0.12-8089 # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR network share
    restart: unless-stopped
#volumes: # use this section if you've setup a docker volume named channels-dvr, with CIFS or NFS, to bind to /mnt/dvr inside the container
  #channels-dvr:
    #external: true

and the environment variable for DVR_SHARE would be:

DVR_SHARE=/media/dvr
OliveTin for Channels: An Interface for Misc Channels DVR Scripts & Tricks
OliveTin for Channels: An Interface for Misc Channels DVR Scripts & Tricks
#8

Ahh, yes - my main instance of channels is not running in a container. Lots of other stuff is but not the original instance of cDVR.

#9

That's fine -- it doesn't need to be. What we're talking about here is where your recordings can be found.

#10

Thanks - fixed it up just fine. Interesting find the LosslessCut LLC approach.

#11

The fact that this last step is manual bothers me. :grimacing:

I will add this to my list of things to try to automate. :grin:

Basically, check what happens in the background when you save. :thinking:

If I think that there is a way to automate it, I will report back. :slightly_smiling_face:

So I looked into it, @bnhf, and I am confused.
When you click to save in the CDVR commercial editor, it sends this request to the server:

POST http://127.0.0.1:8089/dvr/files/57560/comskip/edit?source=local

with a JSON payload that contains [start,end,start,end,...] as explained by @chDVRuser in this post.

So I think this step is already done by OliveTin earlier and it is not actually needed at the end.
Once you are at this step, the change has already been done and there is nothing to save.

Can you confirm?

#12

Taking a quick look at the script, combined with a dash of recollection :slight_smile:, I believe that last step updates an associated .edl file and any other files enabled in CDVR "Integrations".

#13

Interesting. Based on my new observation, isn't it something that could be automated by sending the POST request that I mentioned?
I thought this step was done by the OliveTin action already but I have not seen the script. :grin:

#14

It's pretty simple, like most OliveTin scripts I've written:

#/bin/bash

set -x

dvr=$1
channelsHost=$(echo $dvr | awk -F: '{print $1}')
channelsPort=$(echo $dvr | awk -F: '{print $2}')
logFile=/config/"$channelsHost"-"$channelsPort"_llc2metadata_latest.log
fileID="$2"
dvrDir="$channelsHost-$channelsPort"

echo "Retrieving JSON data for FileID $fileID...\n"
recordingName=$(curl -s "http://$dvr/dvr/files/$fileID" | jq -r '.Path' | sed 's|\\|/|g')
recordingLLC="${recordingName%.mpg}-proj.llc"
llcChapters=$(cat "/mnt/$dvrDir/$recordingLLC")
llcCommercials=$(echo "$llcChapters" | awk -F'(start: | end: )' '{print $2}' | tr -d '\n' | sed 's/,$//')

echo "\n\nRetrieving LosslessCut chapter markers for $recordingName...\n" > "$logFile"
echo "$llcChapters\n" >> "$logFile"

echo "POSTing updated commercial skip markers to $recordingName...\n" >> "$logFile"
echo "[$llcCommercials]\n" >> "$logFile"

curl -XPOST "http://$dvr/dvr/files/$fileID/comskip/edit?source=local" --data-raw "[$llcCommercials]"

echo "Regenerating video index...\n" >> $logFile
curl -XPUT "http://$dvr/dvr/files/$fileID/m3u8"

cat "$logFile"
#15

Yes, this looks exactly like the request that is sent from the CDVR commercial editor.

Hence, my confusion.
It seems redundant to redo it manually at the end.

But maybe I'm missing something. Maybe I'm not thinking straight; it's been a long day for me, I've been up since 3:30 AM.

I'll probably give it another look over the weekend after I get some rest. :slight_smile:

Thank you for posting your script, by the way. It's short, simple, and clean. :+1:

#16

If that POST is all that's done by the CDVR editor, then as you say, saving it is redundant. It should be easy enough to confirm one-way-or-another, the next time I have chance to use it.

1 Like
#17

I ran some tests and found that there is nothing extra to be done after you run the script, everything gets updated (internal data in CDVR + EDL file). :slight_smile:

Note: I don't have OliveTin so I ended up converting your shell script to Python in order for me to call it manually.

#18

Great, thanks for doing that. Glad to know there's one less step needed in the process.

Let me know if there's anything I can do to help there. :slight_smile:

#19

I'll have to give it another try at some point. Maybe during the Christmas holidays.
I will let you know. :wink: