OliveTin for Channels: An Interface for Misc Channels DVR Scripts & Tricks

@kcshield

A bit of a trip to crazy town -- but sorted now. That particular expansion of wildcard arguments is only partially supported in OliveTin, with the presence or absence of quotes being even more important than usual. Particularly when combined, as in this case, with filenames that can contain spaces.

Functioning build pushed under the :test.debian tag. Working well in my testing. Pretty convenient to do a *.mp4, or even an *S1E10* for a single episode shorthand.

Reworked playonedl.sh:

#!/bin/bash
#     Chapter #0:0: start 0.000000, end 1290.013333
#       first   _     _     start    _     end
set -x

dvr=$1
channelsHost=$(echo $dvr | awk -F: '{print $1}')
channelsPort=$(echo $dvr | awk -F: '{print $2}')
dvrDir="$channelsHost-$channelsPort"
playonPath="$2"
playonSingleMP4="$3"
playonMultipleMP4s=$3

wildcardArgumentHandler() {
if [[ "$playonSingleMP4" == *'?'* || "$playonSingleMP4" == *'*'* ]]; then
  playonMP4s=()
  for playonMP4 in $playonMultipleMP4s; do
    playonMP4s+=("$playonMP4")
  done
else
  playonMP4s=("$playonSingleMP4")
fi
}

commercialChapters2EDL() {
for playonMP4 in "${playonMP4s[@]}"; do
  ffmpeg -i "$playonMP4" 2> /tmp/playonedl.tmp
  playonEDL="${playonMP4%.mp4}.edl"  # Remove .mp4 from the filename and add .edl
  [ -f "$playonEDL" ] && rm "$playonEDL"

    while read -r first _ _ start _ end; do
      if [[ $first = Chapter ]]; then
        read  # discard line with Metadata:
        read _ _ chapter
        if [ "$chapter" = "Advertisement" ]; then
          echo -e "${start%?}\t$end\t3" >> "$playonEDL"
        fi
      fi
    done </tmp/playonedl.tmp

  rm /tmp/playonedl.tmp
  echo -e "\n$playonEDL created for:"
  echo -e "$playonMP4 with the following contents:\n"
  cat "$playonEDL"
done
}

main() {
  cd "/mnt/$dvrDir/PlayOn/$playonPath"
  wildcardArgumentHandler
  commercialChapters2EDL
  cd /
}

main

New Debian-based version of OliveTin-for-Channels pushed today as :latest. OliveTin Actions available include:

  • Comskip on/off by channel
  • List channels with Comskip off
  • Manually add recordings
  • Find Gracenote station IDs
  • Fix YouTube thumbnails
  • Download TwiT.tv guide data
  • Generate channels list in CSV format
  • Generate movie list in CSV format
  • Scan/Prune local content
  • Mark an episode for re-recording
  • Channel lineup change notifications
  • Generate filtered Channels DVR log
  • List sources for a single channel
  • Delete Channels DVR recording log files
  • Send message to defined Channels clients
  • Docker-Compose Examples for Channels & Related Extensions
  • Remove Comskip markers from a recording
  • Restart or shutdown a Channels DVR server
  • Ping Channels DVR server
  • Generate a Channels DVR M3U playlist
  • Remove commercials based on an EDL file
  • Create EDL file from PlayOn recording chapters
  • Create subtitles (.srt) file from Closed Captions

A number of these Actions include the ability to have them execute on a recurring schedule.

Happy Holidays Channels Community!

2 Likes

I pulled the latest image and I'm getting an error in List Sources For a Single Channel.


It's complaining about function gensub not being defined.

@cyoungers OK, thanks. It seems Fedora uses the GNU version of awk by default and Debian does not. So, I added gawk to the olivetin image and changed awk to gawk in the script. Appears fixed here, let me know if you see the same after pulling the :latest again.

That was a good fix. Works now.

1 Like

OliveTin-for-Channels :latest and :2023.12.26 updated to support fixing commercial skip points based on data from LosslessCut. This supplements the built-in CDVR commercial editor, if you have situations where commercials are buried within blocks. See here for more details:

screenshot-htpc6-2023.12.26-05_12_00

Here's the latest annotated docker-compose with sample env vars. I've added some clarification regarding domain search and the binding of directories or volumes from multiple DVRs:

version: '3.9'
services:
  olivetin:
    image: bnhf/olivetin:${TAG} # Add the tag like latest or test to the environment variables below
    container_name: olivetin
    dns_search: ${DOMAIN} # For Tailscale users using Magic DNS, add your Tailnet (tailxxxxx.ts.net) to use hostnames for remote nodes, otherwise use your local domain name
    ports:
      - 1337:1337
    environment:
      - CHANNELS_DVR=${CHANNELS_DVR} # Add your Channels DVR server in the form hostname:port or ip:port
      - CHANNELS_DVR_ALTERNATES=${CHANNELS_DVR_ALTERNATES} # Space separated list of alternate Channels DVR servers to choose from in the form hostname:port or ip:port
      - CHANNELS_CLIENTS=${CHANNELS_CLIENTS} # Space separated list of Channels DVR clients you'd like notifications sent to in the form hostname or IP
      - UPDATE_YAMLS=${UPDATE_YAMLS} # Set this to true to update config.yaml
      - UPDATE_SCRIPTS=${UPDATE_SCRIPTS} # Set this to true to update all included scripts
      - TZ=${TZ} # Add your local timezone in standard linux format. E.G. US/Eastern, US/Central, US/Mountain, US/Pacific, etc
    volumes:
      - ${HOST_DIR}/olivetin:/config # Add the parent directory on your Docker you'd like to use
      - ${DVR_SHARE}:/mnt/local-server6-8089 # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR network share
      - ${DVR2_SHARE}:/mnt/remote-server-8089 # Note that these volume mounts should always be to /mnt/hostname-port or /mnt/ip-port (dash rather than a colon between)
    restart: unless-stopped
#volumes: # use this section if you've setup a docker volume named channels-dvr, with CIFS or NFS, to bind to /mnt/dvr inside the container
  #channels-dvr:
    #external: true

Sample environment variables:

TAG=latest
DOMAIN=tailxxxxx.ts.net
CHANNELS_DVR=local-server:8089
CHANNELS_DVR_ALTERNATES=remote-server:8089
CHANNELS_CLIENTS=appletv4k-den firestick-bedroom
UPDATE_YAMLS=true
UPDATE_SCRIPTS=true
TZ=US/Mountain
HOST_DIR=/data
DVR_SHARE=/mnt/dvr
DVR2_SHARE=/mnt/dvr2
1 Like

Trying this out and of course I'm stumbling a little bit. I did the following to get it installed:

user@host: mkdir /etc/OliveTin/
user@host: # ie: Your config file is /etc/OliveTin/config.yaml on the host machine. We'll create this in the post-installation step.
user@host: docker pull jamesread/olivetin
user@host: docker create --name olivetin -p 1337:1337 -v /etc/OliveTin/:/config:ro docker.io/jamesread/olivetin
user@host: cd /etc/OliveTin/
user@host: curl -O https://raw.githubusercontent.com/OliveTin/OliveTin/main/config.yaml
user@host: docker start olivetin

However, it won't start and this is showing in the logs:

date	stream	content
2023/12/27 15:22:05	stderr	level="error" msg="Config file error at startup. Config File \"config.yaml\" Not Found in \"[/ /config /etc/OliveTin]\""
2023/12/27 15:22:05	stderr	level="debug" msg="Value of -configdir flag" value="."
2023/12/27 15:22:05	stderr	level="info" msg="OliveTin initializing" commit="6b0e414" date="2023-12-21T01:26:17Z" version="2023.12.21"

Here's where I have coinfig.yaml placed:
image

Any clue what I may have done wrong?

TIA

I'm supporting this project strictly via Portainer, which is super-easy to install:

docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 --name portainer \
    --restart=always \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v portainer_data:/data \
    cr.portainer.io/portainer/portainer-ce:latest

Then access the Portainer WebUI on port 9000 for http or 9443 for https. Setup your login name and password, and then in Portainer-Stacks you can use the docker-compose and environment variables as described in the post just above yours. You'll want to remove the jamesread/olivetin container, as this project uses a custom version of it.

You should edit the first post with all the install details on all your threads. Not easy going post by post when things change.

Thanks.

1 Like

That's a good idea -- I'll do that.

Also you'd probably find that many projects, including this one, have their current recommended docker-compose (or command line/or both) posted on Docker Hub:

Done.

1 Like

Might be worth adding the portainer install too, for those who haven't used it before.

Done deal. Thanks.

1 Like

Tonight I updated my Channels DVR server, an M1 Mac mini, from macOS Ventura to the latest, macOS Sonoma 14.2.1. Everything went well and all functions appear to be working. The only issue I seem to be having are with the two "green" OliveTin buttons I previously had running:

-Fix YouTube Thumbnails
-Update TWiT.tv Guide

Looks like I'm able to run them manually but they don't stay running and the fields I'm entering (IP address of my server, and 48h interval) aren't being saved. Those fields revert when I go back and refresh. Here are the error logs, I am running this in portainer.io, what else should I check to get these working again? Looks like a permission issue but I'm not sure how to fix this with portainer. Any help appreciated. Thanks!

Generate TWiT.tv Guide Data

stdout

Background script initiated, with 48h between runs for 10.0.1.21:8089
Downloading latest iCal data for 10.0.1.21:8089...
Background ical_2_xmltv process running for 10.0.1.21:8089
foreground.sh is exiting for ical_2_xmltv with exit code 0

stderr

+ dvr=10.0.1.21:8089
++ echo 10.0.1.21:8089
++ awk -F: '{print $1}'
+ channelsHost=10.0.1.21
++ echo 10.0.1.21:8089
++ awk -F: '{print $2}'
+ channelsPort=8089
+ backgroundScript=ical_2_xmltv
+ firstChar=i
+ runInterval=48h
+ healthchecksIO=https://hc-ping.com/your_custom_uuid
+ [[ https://hc-ping.com/your_custom_uuid == \h\t\t\p\s\:\/\/\h\c\-\p\i\n\g\.\c\o\m\/\y\o\u\r\_\c\u\s\t\o\m\_\u\u\i\d ]]
+ healthchecksIO=
+ spinUp=
++ ps -ef
++ grep '[i]cal_2_xmltv.* 10.0.1.21:8089'
++ awk '{print $2}'
+ runningScriptPID=
+ [[ -n '' ]]
+ backgroundArguments='10.0.1.21:8089 48h '
+ greenIcon='"icons/channels.png"'
+ purpleIcon='"https://community-assets.getchannels.com/original/2X/5/55232547f7e8f243069080b6aec0c71872f0f537.png"'
+ logFile=/config/10.0.1.21-8089_ical_2_xmltv_latest.log
+ runFile=/tmp/10.0.1.21-8089_ical_2_xmltv.run
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ rm /tmp/10.0.1.21-8089_ical_2_xmltv.run
+ [[ -n '' ]]
+ trap finish EXIT
+ main
+ scriptRun
+ case "$runInterval" in
+ [[ -n '' ]]
+ echo 'Background script initiated, with 48h between runs for 10.0.1.21:8089'
+ grep -q '(.*) #ical_2_xmltv' /config/config.yaml
+ nohup /config/ical_2_xmltv.sh 10.0.1.21:8089 48h
+ [[ 1 == \1 ]]
++ date +%d%b%y_%H:%M
+ sed -i '/#ical_2_xmltv title/s/#/(28Dec23_20:01) #/' /config/config.yaml
sed: couldn't open temporary file /config/sedrcfYQX: Permission denied
+ sed -i '/#ical_2_xmltv interval default/s/default: .* #/default: 48h #/' /config/config.yaml
sed: couldn't open temporary file /config/sedZpvI7C: Permission denied
+ [[ -n '' ]]
+ echo '10.0.1.21:8089 ical_2_xmltv 48h '
+ sleep 2
+ runningScripts
+ servers=($CHANNELS_DVR $CHANNELS_DVR_ALTERNATES)
+ for server in "${servers[@]}"
++ echo 10.0.1.21:8089
++ awk -F: '{print $1}'
+ serverHost=10.0.1.21
++ echo 10.0.1.21:8089
++ awk -F: '{print $2}'
+ serverPort=8089
++ ps -ef
++ grep '[i]cal_2_xmltv.* 10.0.1.21:8089'
++ awk '{print $2}'
+ activeProcess=563
+ [[ -n 563 ]]
+ echo 'Background ical_2_xmltv process running for 10.0.1.21:8089'
+ exit 0
+ finish
+ echo -e 'foreground.sh is exiting for ical_2_xmltv with exit code 0'
+ cp /config/config.yaml /config/temp.yaml
+ cp /config/temp.yaml /config/config.yaml
+ backgroundWait=0
+ maxWait=30
+ '[' 0 -lt 30 ']'
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ (( backgroundWait++ ))
+ sleep 1
+ '[' 1 -lt 30 ']'
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ (( backgroundWait++ ))
+ sleep 1
+ '[' 2 -lt 30 ']'
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ (( backgroundWait++ ))
+ sleep 1
+ '[' 3 -lt 30 ']'
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ (( backgroundWait++ ))
+ sleep 1
+ '[' 4 -lt 30 ']'
+ [[ -f /tmp/10.0.1.21-8089_ical_2_xmltv.run ]]
+ break
+ '[' -f /config/10.0.1.21-8089_ical_2_xmltv_latest.log ']'
+ cat /config/10.0.1.21-8089_ical_2_xmltv_latest.log```

Interesting convergence of the latest version of the MacOS and sed running in a container.

"sed -i" which I've used extensively in several scripts, does an inline editing of files -- but behind the scenes creates a temporary file, apparently in the same directory as the file it's modifying. In this case the directory on your host you have bound to /config.

This temporary file carries no permissions (something of a bug in sed I believe), but this hasn't been an issue until Docker Desktop for Mac starting using the VirtioFS (which was an experimental opt-in I believe, but now may be an opt-out).

Could you try disabling VirtioFS in Docker Desktop, which could be via tick-box or possibly switching from VirtioFS to gRPC, and see if that "fixes" the issue. Apparently sed 4.8 also addresses this problem, so I'm looking into options for upgrading that.

I could also re-work my 28 sed commands that use the -i option, but that's not my first choice. :upside_down_face:

@Fofer Even though I love "sed -i" like a brother, after looking more closely at the 28 times I used it in my scripts, reworking them was trivial.

So, I changed all of the scripts that use it save one (the delete old recording logs script), and pushed it with the :test tag. I believe it will sort out this issue, even if you're using VirtioFS in Docker Desktop. Let me know if it works, and I'll update that last script and push it as :latest.

Thank you so much! I left this setting as-is, in Docker Desktop:

But in portainer,I did just redeploy the container using the duplicate/edit command and the :test: tag, and now those two OliveTin buttons are saving my parameters and remaining green. I appreciate it!

P.S. along the way I found this interesting discussion, which seems applicable. Most of it went over my head though :slight_smile:

New bnhf/olivetin:latest pushed with fix for "Docker Desktop for Mac VirtioFS permissions" issue. No pressing need to update to this container unless your OliveTin Docker host is a Mac.

what would I use to connect via a machine on the same network?

DVR_SHARE=/mnt/dvr
DVR2_SHARE=/mnt/dvr2