BETA: Channels DVR Server for Raspberry Pi 4 (USB BOOT IMAGE)

We're working on an official RPi4 image for Channels DVR Server!

You can flash this image directly on a USB3 drive, then attach that to an RPi4 and it will boot up into a fully functional DVR. This is made possible by RPi4's new USB boot support, which means flaky SD cards are no longer required and you end up with a much faster and more reliable Pi-based DVR.

Our RPI4 image is based on an ARM64 embedded linux distribution, and has been optimized to fully take advantage of the Pi4 CPU and GPU including transcoding for remote viewing. Thanks to our friends at home-assistant.io and their HassOS for the inspiration.

We're looking for beta testers to try this out and provide feedback!


Requirements

Installation

  1. Download the Raspberry Pi Imager for your OS

  2. Upgrade the EEPROM bootloader on your Pi4 to enable USB booting

    • in the Raspberry Pi Imager: select CHOOSE OS > Misc Utility Images > Pi 4 EEPROM boot recovery
    • click Choose SD and select your microSD card
    • click Write and wait for the process to finish
    • insert the microSD the Pi and apply power
    • wait until the green light on the front starts flashing green rapidly
    • remove the SD card and put it away
  3. Prepare Channels DVR Server USB drive

    • download ChannelsDVRServer_PI4.img.gz
    • in the Raspberry Pi Imager: select CHOOSE OS > Use custom > ChannelsDVRServer_PI4.img.gz
    • click Choose SD and pick your USB3 drive
    • click Write and wait for the process to finish
  4. Boot up the new Pi

    • plug the USB drive into one of the BLUE USB3 ports on the Pi
    • plug the ethernet port into your router
    • plug in the power
  5. Wait a few minutes, then visit http://dvr-server.local:8089 to set up your new DVR

Features

  • Your RPi4 DVR USB3 drive can be attached to a PC or Mac for direct access your recording files. They are stored in an ExFAT partition

  • You can attach additional USB drives to the RPi. They will appear in the settings UI automatically and can be used with the Local Content and Additional Storage features

  • Hardware transcoding is supported using the RPi GPU

Troubleshooting

If you're having trouble getting things working (or simply want to peek under the hood and play around), you will need the following:

  • microHDMI to HDMI adapter
  • small USB flash drive
  • USB keyboard

Verifying EEPROM update

Plug the RPi4 into a TV or monitor using the microHDMI to HDMI adapter and an HDMI cable.

Power on the device, and you should see a red Raspberry Pi logo on the screen along with the text "bootloader: c305221a Sep 3 2020"

If you don't see the logo or the date is older than September 2020, follow the steps in (2) above to upgrade your bootloader.

Terminal access

If you plug the Pi into a TV and plug in a USB keyboard, you can login with the user root and access the system internals. Note that there is a very barebones linux install without many programs, and the entire root filesystem is read-only. Modifications are not possible or recommended.

SSH access

If you would like SSH access (for instance, to SFTP recordings), format a small USB disk as FAT/NTFS with the label "CONFIG" and place an authorized_keys file on it. Plug this into the RPi. You should be able to use ssh [email protected]<ip> -p 22222 to access the system.

7 Likes

Oh man. This will be a very tough call on whether or not i want to rebuild everything on this image - my pi is almost exclusively Channels DVR, and runs well.

I know this is duplicated effort - but any chance a changelog, or something for a DIY install, could be produced? Might save me having to transfer a ton of stuff back and forth, etc.

I can't get it to boot into a working state. I installed with the Raspberry Pi Imager for macOS on a 4TB NAS HDD in a USB3 enclosure. It boots and I get the initial splash screen, but then it goes to a black screen and I never get anything after that. I do get a response from ping for ~64 seconds, then it stops responding to ping. That whole time I get no SYN-ACK from port 8089 or 2222.

Here's the initial boot splash screen showing the correct boot loader is installed:

Is this a bus-powered enclosure, or does it have its own PSU? RPis have often had issues with bus-powered USB devices causing issues because of their increased power draw.

Own PSU :stuck_out_tongue:

@timstephens24 Can you try plugging into one of the USB2 (black) ports to see if there is any difference? What brand enclosure is it?

@hancox Depending on your setup, another option would be to pick up an small/inexpensive SSD and flash that with our img. Then use the SSD to boot the Pi, but still attach and reuse your existing DVR drive like before. After the install you would initiate a restore using the backup from your previous setup.

No difference using a USB2 port. The brand for the enclosure is Sabrent (https://www.amazon.com/dp/B00LS5NFQ2/ref=cm_sw_em_r_mt_dp_QfjGFbBE793K8).

Sabrent branded USB adapters are known not to work. See some discussion on https://jamesachambers.com/raspberry-pi-4-usb-boot-config-guide-for-ssd-flash-drives/

Can you pull the device/product ID from dmesg or lsusb on working Linux system?

You can try mounting the boot VFAT and editing cmdline.txt to add the quirk, or send it to me and I will add to our image.

Not sure if the quirk will fix it in this case. I thought using USB2 was equivalent to the quirk workaround but I am not positive.

Adding the quirks didn't help but just for info lsusb gives me:
ID 174c:1153 ASMedia Technology Inc. ASM1153 SATA 3Gb/s bridge

I did get it going with a 128GB USB thumb drive, but channels didn't start at boot. systemctl status showed "Dependency failed for Channels DVR." I did run it from the data/latest folder and it executed successfully, so I killed that instance and ran 'systemctl start channels-dvr.service' and it executed successfully.

I rebooted and then everything started up normal, so not sure what happened on the first boot.

Interesting..

I'll add that ID to the default quirks list. I did see someone else mention using that quirk.

For the DVR, the only dependencies are network-online and time-sync. I wonder which of those failed. journalctl -u channels-dvr might show more details, or systemctl list-jobs.

I'll upload a new image shortly with some fixes.

What a fabulous idea.

I decided to sacrifice my cold standby, a second Pi4 in a CanaKit PS and enclosure with a spare 750Gb HD in a Sabrent USB3 HD Enclosure. I updated its EEPROM bootloader and imaged the HD without first nuking an existing /media/DVR partition, apparently a mistake. The server failed to start on the first power up. On the second try, it entered the storage path dialog, offering what may have been that preexisting partition, but throwing a permissions error. That made me long for the comfort of a terminal session, but I realize the whole point is not to need one.

I used my laptop to delete all partitions on that HD then re-imaged it. The Pi4 again failed to start the server on the third try, then on the fourth, it came up, having recognized my HDHomeRun and ready to download the guide. It has started every time on the fifth thru seventh boots so far. Now waiting to see it it will record and playback tonight's PBS Newshour.

I suggest warning Beta testers to start with an unpartitioned drive and that two initial boots might be needed. I also wouldn't mind having a local terminal session up in case I find bugs to report.

One question: why so many small partitions created on the HD?

Again, Great Idea!

Appreciate the tests and feedback!

I fixed some bugs causing the the /media/DVR mount not to come up on the first boot. It should be better with the latest image.

I'm also seeing an issue where sometimes the DVR won't come up after reboot because of an issue with the systemd-time-sync-wait.service. I'm trying to resolve that.

The small partitions hold the bootloader, kernel, rootfs, and an overlayfs for files which can be modified. There are two sets of kernel/rootfs partitions plus a final bootstate partition, which enables an A/B boot scheme and will allow us to add automatic OS updates in the near future.


EDIT: I'm seeing the same failure here on 2020.1009.2002, and it is indeed related to the time-sync bug:

# systemctl list-jobs
JOB UNIT                           TYPE  STATE
 22 time-sync.target               start waiting
 53 systemd-time-wait-sync.service start running
 77 e2scrub_all.timer              start waiting
 75 timers.target                  start waiting

4 jobs listed.
# systemctl status channels-dvr
● channels-dvr.service - Channels DVR
     Loaded: loaded (/usr/lib/systemd/system/channels-dvr.service; enabled; vendor preset: enabled)
     Active: inactive (dead)

Feb 07 15:50:56 server systemd[1]: Dependency failed for Channels DVR.
Feb 07 15:50:56 server systemd[1]: channels-dvr.service: Job channels-dvr.service/start failed with result 'dependency'.

It looks like time-sync eventually corrected the date from Feb to Oct, but channels-dvr was already in a failed state by that point.

I do have a console display on it for the time being, and its throwing repeated errors. 'Dont know if they mean anything.

IPV6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
brcmfmac: brcmf_cfg80211_set_power_mgmt:powewr save enabled

The Pi4 is on wired Ethernet to my network. The wlan interface isn't configured.

I see those too, I believe they're pretty harmless. We'll be adding network configuration options directly to the DVR web UI in the future.

Alright, I have a new image uploaded that comes up cleanly on the first boot and subsequent reboots.

It also includes all the common USB quirks I could find for popular adapters.

My beta 2020.1009.0020 (Raspberry Pi 4) (kernel: 4.19.127-v8)
was failing all record tasks. The log showed repeated errors, though I could mount the DVR partition on a Mint 20 machine then copy to and read files back from it.

2020/10/09 22:59:38.204318 [TNR] Opened connection to 10770EA5/2 for ch20.1 WFYI 1
2020/10/09 22:59:38.204786 [TNR] Closed connection to 10770EA5/2 for ch20.1 WFYI 1
2020/10/09 22:59:38.204869 [DVR] Error running job 1602280800-ch20.1 PBS NewsHour: stat /media/DVR: no such file or directory

I just re-imaged it to 2020.1009.2307 (Raspberry Pi 4) (kernel: 4.19.127-v8) That booted up on the first try and is now recording and playing back perfectly with the iPad client. The web client on the same iPad is playing the audio, but displaying a black frame or a still image even after adequate time for buffering. The web client running with chrome v85 on a laptop is displaying both, but stuttering. Admin page shows the Pi4 hardware transcoder falling short of real time at any bitrate. I've seen a Pi4 transcode in real time only once and have not been able to duplicate it since. Not a big deficit since we use only the FireTV or AppleTV clients that don't need it.

I couldn't get it to work with any of my 3.5" HDD enclosures, even WD and Seagate. Right now I'm just using the thumbdrive with another thumbdrive for ssh keys and then a script to ssh in and mounting a folder from my Unraid.

I also tried to transcode something from the web interface and all I got was a black screen. Remux plays fine. (logs has extra stuff like adding TVE and importing TV, so maybe I'm trying to do too much at once?)

2020/10/10 00:49:23.494836 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:24.163864 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E12 - Evicted! (Bluray-1080p).mkv
2020/10/10 00:49:25.501966 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E13 - City of Thieves (Bluray-1080p).mkv
2020/10/10 00:49:26.711996 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:26.893699 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:28.037717 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:28.518471 [TVE] action=request type=Document method=GET url=https://www.sciencechannel.com/watch/science
2020/10/10 00:49:30.110984 [TVE] action=authed
2020/10/10 00:49:30.115952 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:30.120082 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:30.837301 [TVE] Channel scan 83/209 SCIENCE successful
2020/10/10 00:49:34.545129 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:34.554342 [TVE] action=mock
2020/10/10 00:49:34.556325 [TVE] action=set_cookies domains=1
2020/10/10 00:49:34.557590 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=ahc&redirect_uri=https%3A%2F%2Fwww.ahctv.com%2Fwatch%2Fahc&response_flow=code&response_type=adobe&state=nonce%2CQiTgoXd3floiE3bJNYxeYNl0egOzK9Fc
2020/10/10 00:49:34.560546 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:34.910784 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:35.091716 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:35.278426 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:35.617461 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:35.617632 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:36.229519 [TVE] action=wait_for_page
2020/10/10 00:49:36.276937 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:39.505741 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:39.687101 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:40.865696 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:41.340631 [TVE] action=request type=Document method=GET url=https://www.ahctv.com/watch/ahc
2020/10/10 00:49:41.505741 [TNR] Opened connection to 131F8A20/1 for ch1006 WJBFDT
2020/10/10 00:49:41.862469 [HLS] Starting transcoder for channel 1006 from 192.168.24.13 (encoder=remux, resolution=, deinterlacer=, bitrate=0)
2020/10/10 00:49:42.452879 [HLS] Probed live stream in 584.576067ms: h264 1280x720 progressive 1764409bps
2020/10/10 00:49:42.580225 [TVE] action=authed
2020/10/10 00:49:42.580411 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:42.592511 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:43.196427 [TVE] Channel scan 84/209 AHC successful
2020/10/10 00:49:43.384130 [HLS] Session ch1006-dANY-ip192.168.24.13 started in 2.469176949s
2020/10/10 00:49:43.412419 [ENC] Starting encoder for ch1006 in /media/data/Media/ChannelsDVR/Streaming/ch1006-dANY-ip192.168.24.13-249634983/encoder-1-691530714 at 1 (1.323911) (encoder=h264_v4l2m2m, resolution=576, deinterlacer=blend, bitrate=1000 segment_size=0.01)
2020/10/10 00:49:43.488023 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264 @ 0xa761840] Increasing reorder buffer to 2
2020/10/10 00:49:44.579228 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:44.588119 [TVE] action=mock
2020/10/10 00:49:44.589932 [TVE] action=set_cookies domains=1
2020/10/10 00:49:44.591090 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=dam&redirect_uri=https%3A%2F%2Fwww.destinationamerica.com%2Fwatch%2Fdestination-america&response_flow=code&response_type=adobe&state=nonce%2CrOIXKZTy78CPQkfqoZXpZITWnB0iIBoq
2020/10/10 00:49:44.593855 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:44.938669 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:45.119223 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:45.316525 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:45.634476 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set number of B-frames: Invalid argument
2020/10/10 00:49:45.634541 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set number of B-frames
2020/10/10 00:49:45.634612 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set header mode: Invalid argument
2020/10/10 00:49:45.634655 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set gop size: Invalid argument
2020/10/10 00:49:45.634686 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Encoder adjusted: qmin (0), qmax (51)
2020/10/10 00:49:45.634739 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set minimum video quantizer scale: Invalid argument
2020/10/10 00:49:45.634770 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set maximum video quantizer scale: Invalid argument
2020/10/10 00:49:45.728672 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:45.728753 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:46.301871 [TVE] action=wait_for_page
2020/10/10 00:49:46.389137 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:49.227918 [ENC] Segment 4 has unexpected duration: inputs=6-7 expected=0.950633 actual=1.001 expected_pts=4.376967-5.311233 actual_pts=4.348244-5.332567
2020/10/10 00:49:49.739941 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:49.878470 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:51.161213 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:51.577083 [TVE] action=request type=Document method=GET url=https://www.destinationamerica.com/watch/destination-america
2020/10/10 00:49:52.885950 [TVE] action=authed
2020/10/10 00:49:52.886238 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:52.899978 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:53.474970 [TVE] Channel scan 85/209 DESTINATION-AMERICA successful
2020/10/10 00:49:53.894731 [TVE] Channel scan 86/209 DISCOVERY-LIFE failed: notAuthorized: This network is not included in your current television subscription. Please contact your TV provider to subscribe.
https://my.xfinity.com/upgrade
2020/10/10 00:49:55.239228 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:55.256928 [TVE] action=mock
2020/10/10 00:49:55.259520 [TVE] action=set_cookies domains=1
2020/10/10 00:49:55.268233 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=vel&redirect_uri=https%3A%2F%2Fwatch.motortrend.com%2Fwatch%2Fmotortrend&response_flow=code&response_type=adobe&state=nonce%2CtfVeobkyo72ebBhH9uhcEqdMFn5luDCv
2020/10/10 00:49:55.276680 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:55.660444 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:55.828965 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:56.008210 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:56.256737 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:56.256831 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:56.582679 [TVE] action=wait_for_page
2020/10/10 00:49:56.675398 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:57.404152 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [mpegts @ 0xa7ae7a0] H.264 bitstream error, startcode missing, size 0
2020/10/10 00:49:58.858489 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E14 - The Witch's Garden (Bluray-1080p).mkv
2020/10/10 00:49:59.958499 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:50:00.092431 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:50:00.986499 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:50:01.433982 [TVE] action=request type=Document method=GET url=https://watch.motortrend.com/watch/motortrend
2020/10/10 00:50:01.572526 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [mpegts @ 0xa7ae7a0] H.264 bitstream error, startcode missing, size 0
2020/10/10 00:50:02.189787 [TVE] action=authed
2020/10/10 00:50:02.189939 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:50:02.233127 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:50:06.479558 [HLS] Stopping transcoder session ch1006-dANY-ip192.168.24.13 (out: 25.431344s, finished: false)
2020/10/10 00:50:06.517533 [ENC] Stopped encoder for ch1006 in /media/data/Media/ChannelsDVR/Streaming/ch1006-dANY-ip192.168.24.13-249634983/encoder-1-691530714 after encoding 1 to 18
2020/10/10 00:50:06.517740 [TNR] Closed connection to 131F8A20/1 for ch1006 WJBFDT
2020/10/10 00:50:06.528422 [SNR] Statistics for ch1006 WJBFDT: ss=77% snq=91%,86%-93% seq=100% bps=4405090,3104256-5522688 pps=422,297-527

I've tried a couple 2.5" drives and adapters that all worked. Will have to find a 3.5" to see why that's any different.

Curious what command you're using to mount. I would like to make both network mounting and network file access easier to setup from the web UI.

I will check out the transcoding issues.