Using OliveTin with TubeArchivist plus CDVR/TA Processing Script

[bnhf]

I've created an OliveTin-for-Channels Action for @Phillip_Berryman's TubeArchivist Processing Script. In addition I've added a Project One-Click Action to install TubeArchivist itself:

It's a fairly easy setup if you already have OliveTin-for-Channels installed. but I thought I'd create a separate thread specifically covering getting this up-and-running.

Installing TubeArchivist:

HOST_PORT, TA_HOST, TA_PASSWORD, TZ and HOST_DIR are the most likely fields you'll need to personalize.

TA uses port 8000 by default, which is used by Portainer, so I've defaulted it to 8010. The other ports are only exposed within the stack, so you can generally leave those as is.

TA_HOST is the hostname (must be resolvable on your LAN) of system running TA, and in this case it's the same as your Docker Host -- you can also use your IP address.

TA_PASSWORD is the initial password you'd like to use for TA, so if you set this now you'll be good-to-go.

TZ is in the usual Linux format, with Continent/City the desired construction.

HOST_DIR can be a little tricky to understand, but if you use the same value you used for DVR_SHARE when you setup OliveTin (and have tested it's correct with the OliveTin Post-Install Healthcheck), you'll be golden. Your downloaded TA videos will then be saved in the same location as your CDVR recordings, under a folder named tubearchivist, all set to have metadata downloaded, be renamed, and moved to Imports/Videos by the TA Processing script.

At this point you'll be able to subscribe to YouTube Channels, and download desired videos:

Next, you'll want to get the OliveTin Action running:

If you're not already aware, Actions that are running on a recurring basis are shown in green. My TubeArchivist Processing Script Action is already running.

dvr, frequency and youtube_api_key are all required. You'll need to setup an API key, if you don't already have one. To check if you have one setup, go to the Google Developer's Console. I had one from years ago I setup for Kodi, and it still works.

When TA Processing Script runs, it'll rename your videos (stored by TA using YT video ID in Channel folders), and move them from tubearchivist to Imports/Videos. They'll be renamed based on their metadata, and placed in folders by subscribed channel. NFO files will also be created.

Once that's complete, the videos should be organized with readable names and available for viewing in your CDVR client:

[Jean0987654321]

Ok...this may be problematic for me as my youtube vids are in an external drive away from the channels directory. Is there a way to specify a certain directory? I have far more space on my external drive for these vids so yeah...

[bnhf]

@Jean0987654321

Ok...this may be problematic for me as my youtube vids are in an external drive away from the channels directory. Is there a way to specify a certain directory? I have far more space on my external drive for these vids so yeah...

We'll have to get a discussion going with @Phillip_Berryman about that. I originally set things up assuming the videos could be stored anywhere, and then they'd be moved wherever. However the processing script uses a rename, which means the source and destination need to be on the same filesystem.

Since the ultimate goal, I assume, is for everything to appear for viewing in Channels I adopted this approach. The Docker container can be on any host, but the destination for the downloads, and the ultimate destination following the rename need to be on the same filesystem.

My initial goal was to get the processing script running in a Docker container (with support for multiple DVRs), and have them show-up in Channels DVR. With that accomplished, it shouldn't be difficult to make modifications -- if the script supports approaches I haven't thought of...

[bnhf]

@Jean0987654321

my youtube vids are in an external drive away from the channels directory

Can you give more detail on your setup?

Would your initial TA download directory, and the destination CDVR Personal Media directory be on the same filesystem, just not the same one as your DVR recordings? If so, I believe I see how I could make that work.

If you could layout your current, or desired flow, that would help...

[Jean0987654321]

Yeah

My DVR recordings is in /mediabox/dvr
My Youtube vids are in an external drive in /media/jean/2895-809F/tv/youtube
I intend to have Tubearchivist placed in /media/jean/2895-809F/tv/tubearchivsit

[bnhf]

@Jean0987654321

My DVR recordings is in /mediabox/dvr
My Youtube vids are in an external drive in /media/jean/2895-809F/tv/youtube
I intend to have Tubearchivist placed in /media/jean/2895-809F/tv/tubearchivsit

That setup should work with the modifications I made today. A new OliveTin-for-Channels Docker Compose is required (version dated 2025.01.21), as this requires binding a unique TubeArchivist storage location into the OliveTin container. Directory bindings or Docker Volumes are supported:

Latest OliveTin-for-Channels Docker Compose for use with TubeArchivist Action:

version: '3.9'
services:
  olivetin: # This docker-compose requires little or no editing. Set the Environment variables section of Portainer.
    # 2025.01.21
    # GitHub home for this project: https://github.com/bnhf/OliveTin.
    # Docker container home for this project with setup instructions: https://hub.docker.com/repository/docker/bnhf/olivetin.
    image: bnhf/olivetin:${TAG} # Add the tag like latest or test to the environment variables below.
    container_name: olivetin
    hostname: olivetin
    dns_search: ${DOMAIN} # For Tailscale users using Magic DNS, add your Tailnet (tailxxxxx.ts.net) to use hostnames for remote nodes, otherwise use your local domain name.
    ports:
      - ${HOST_PORT}:1337
    environment:
      - CHANNELS_DVR=${CHANNELS_DVR_HOST}:${CHANNELS_DVR_PORT} # Add your Channels DVR server in the form CHANNELS_DVR_HOST=<hostname or ip> and CHANNELS_DVR_PORT=<port>.
      #- CHANNELS_DVR_ALTERNATES=${CHANNELS_DVR2_HOST}:${CHANNELS_DVR2_PORT} # Space separated list of alternate Channels DVR servers to choose from in the form hostname:port or ip:port.
      - CHANNELS_CLIENTS=${CHANNELS_CLIENTS} # Space separated list of Channels DVR clients you'd like notifications sent to in the form hostname or IP.
      - ALERT_SMTP_SERVER=${ALERT_SMTP_SERVER} # SMTP server to use for sending alert e-mails. smtp.gmail.com:587 for example.
      - ALERT_EMAIL_FROM=${ALERT_EMAIL_FROM} # Sender address for alert e-mails.
      - ALERT_EMAIL_PASS=${ALERT_EMAIL_PASS} # SMTP "app" password established through GMail or Yahoo Mail. Do not use your everyday e-mail address.
      - ALERT_EMAIL_TO=${ALERT_EMAIL_TO} # Recipient address for alert e-mails.
      - UPDATE_YAMLS=${UPDATE_YAMLS} # Set this to true to update config.yaml.
      - UPDATE_SCRIPTS=${UPDATE_SCRIPTS} # Set this to true to update all included scripts.
      - TZ=${TZ} # Add your local timezone in standard linux format. E.G. US/Eastern, US/Central, US/Mountain, US/Pacific, etc.
      - PORTAINER_TOKEN=${PORTAINER_TOKEN} # Generate via <username> dropdown (upper right of WebUI), "My account", API tokens.
      - PORTAINER_HOST=${PORTAINER_HOST} # Hostname or IP of the Docker host you're running Portainer on.
    volumes:
      - ${HOST_DIR}/olivetin:/config # Add the parent directory on your Docker you'd like to use.
      - ${DVR_SHARE}:/mnt/${CHANNELS_DVR_HOST}-${CHANNELS_DVR_PORT} # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR network share.
      - ${LOGS_SHARE}:/mnt/${CHANNELS_DVR_HOST}-${CHANNELS_DVR_PORT}_logs # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR logs network share.
      - ${TUBEARCHIVIST_SHARE}:/mnt/${CHANNELS_DVR_HOST}-${CHANNELS_DVR_PORT}_ta # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your TubeArchivist videos network share.
      #- ${DVR2_SHARE}:/mnt/${CHANNELS_DVR2_HOST}-${CHANNELS_DVR2_PORT} # Note that these volume mounts should always be to /mnt/hostname-port or /mnt/ip-port (dash rather than a colon between).
      #- ${LOGS2_SHARE}:/mnt/${CHANNELS_DVR2_HOST}-${CHANNELS_DVR2_PORT}_logs # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your Channels DVR logs network share.
      #- ${TUBEARCHIVIST2_SHARE}:/mnt/${CHANNELS_DVR2_HOST}-${CHANNELS_DVR2_PORT}_ta # This can either be a Docker volume or a host directory that's connected via Samba or NFS to your TubeArchivist videos network share.
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

  static-file-server:
    image: halverneus/static-file-server:latest
    container_name: static-file-server
    dns_search: ${DOMAIN}
    ports:
      - ${HOST_SFS_PORT}:8080
    environment:
      - FOLDER=${FOLDER}
    volumes:
      - ${HOST_DIR}/olivetin/data:${FOLDER}
    restart: unless-stopped

#volumes: # use this section if you've setup a docker volume named channels-dvr, with CIFS or NFS, to bind to /mnt/dvr inside the container. Set ${HOST_DIR} to channels-dvr (HOST_DIR=channels_dvr) in that example.
  #channels-dvr:
    #external: true
  #channels-dvr-logs:
    #external: true
  #tubearchivist:
    #external: true

And, the sample environment variables:

TAG=latest
DOMAIN=tailxxxxx.ts.net
HOST_PORT=1337
CHANNELS_DVR_HOST=local-server
CHANNELS_DVR_PORT=8089
CHANNELS_DVR2_HOST=another-server
CHANNELS_DVR2_PORT=8089
CHANNELS_CLIENTS=appletv4k-den firestick-bedroom
ALERT_SMTP_SERVER=smtp.gmail.com:587
ALERT_EMAIL_FROM=username@gmail.com
ALERT_EMAIL_PASS=xxxxxxxxxxxxxxxx
ALERT_EMAIL_TO=username@gmail.com
UPDATE_YAMLS=true
UPDATE_SCRIPTS=true
TZ=US/Mountain
HOST_DIR=/data
DVR_SHARE=/mnt/dvr
LOGS_SHARE=/mnt/channelsdvr
TUBEARCHIVIST_SHARE=/mnt/dvr
HOST_SFS_PORT=8080
FOLDER=/web
PORTAINER_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
PORTAINER_HOST=docker-host

The new TUBEARCHIVIST_SHARE value should be set to a parent directory value that contains both your source downloaded videos AND the destination processed videos. The destination location needs to have been added to your CDVR Sources for Personal Media, if it's not the default Imports/Videos.

The child subdirectories under the TUBEARCHIVIST_SHARE parent will be set in the OliveTin Action, using two new fields added for that purpose.

If you're using the approach outlined in my previous post, where TubeArchivist stores its videos in the same location as your DVR recordings, your DVR_SHARE and TUBEARCHIVIST_SHARE values will be identical, and the child directories should be set to tubearchivist and Imports/Videos (which are the defaults).

The updated version of the Action looks like this:

[Jean0987654321]

That looks pretty good. I'll give it a proper go later in the week when I'm not busy

[bnhf]

@Phillip_Berryman

Notifications via Apprise:

• Receive processing summaries through your favorite notification platforms (e.g., Pushover, Discord, Slack, Email).

Took me a bit to get Apprise e-mail notifications working with a Gmail App password, but it is possible -- and does work. Here's the syntax to send a notification e-mail to yourself:

mailtos://<your gmail username>:<your gmail app password>@gmail.com@smtp.gmail.com

And, to send a notification to another e-mail address:

mailtos://<your gmail username>:<your gmail app password>@gmail.com@smtp.gmail.com?to=<e-mail address to notify>

EDIT: It turns out you don't even need to specify the SMTP server using Gmail:

mailtos://<your gmail username>:<your gmail app password>@gmail.com

[bnhf]

New bnhf/olivetin:latest (aka bnhf/olivetin:2025.01.31) pushed this morning with a fix for apprise_url handling for making any value you use the new default value. This now matches the way other similar fields are handled for processes setup to run continuously.

There's also a new option to use a value of olivetin:// in that apprise_url field. When that option is used, an Apprise e-mail notification is created using the $ALERT_EMAIL_FROM, $ALERT_EMAIL_PASS and $ALERT_EMAIL_TO values specified in your OliveTin env vars:

EDIT: Also, this format works for sending your Apprise notifications to one of your Channels DVR clients:

json://<your streaming stick hostname or IP>:57000/api/notify