BETA: Channels DVR Server for Raspberry Pi 4 (USB BOOT IMAGE)

dlpoole · October 10, 2020, 12:24am

My beta 2020.1009.0020 (Raspberry Pi 4) (kernel: 4.19.127-v8)
was failing all record tasks. The log showed repeated errors, though I could mount the DVR partition on a Mint 20 machine then copy to and read files back from it.

2020/10/09 22:59:38.204318 [TNR] Opened connection to 10770EA5/2 for ch20.1 WFYI 1
2020/10/09 22:59:38.204786 [TNR] Closed connection to 10770EA5/2 for ch20.1 WFYI 1
2020/10/09 22:59:38.204869 [DVR] Error running job 1602280800-ch20.1 PBS NewsHour: stat /media/DVR: no such file or directory

I just re-imaged it to 2020.1009.2307 (Raspberry Pi 4) (kernel: 4.19.127-v8) That booted up on the first try and is now recording and playing back perfectly with the iPad client. The web client on the same iPad is playing the audio, but displaying a black frame or a still image even after adequate time for buffering. The web client running with chrome v85 on a laptop is displaying both, but stuttering. Admin page shows the Pi4 hardware transcoder falling short of real time at any bitrate. I've seen a Pi4 transcode in real time only once and have not been able to duplicate it since. Not a big deficit since we use only the FireTV or AppleTV clients that don't need it.

timstephens24 · October 10, 2020, 12:52am

I couldn't get it to work with any of my 3.5" HDD enclosures, even WD and Seagate. Right now I'm just using the thumbdrive with another thumbdrive for ssh keys and then a script to ssh in and mounting a folder from my Unraid.

I also tried to transcode something from the web interface and all I got was a black screen. Remux plays fine. (logs has extra stuff like adding TVE and importing TV, so maybe I'm trying to do too much at once?)

2020/10/10 00:49:23.494836 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:24.163864 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E12 - Evicted! (Bluray-1080p).mkv
2020/10/10 00:49:25.501966 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E13 - City of Thieves (Bluray-1080p).mkv
2020/10/10 00:49:26.711996 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:26.893699 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:28.037717 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:28.518471 [TVE] action=request type=Document method=GET url=https://www.sciencechannel.com/watch/science
2020/10/10 00:49:30.110984 [TVE] action=authed
2020/10/10 00:49:30.115952 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:30.120082 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:30.837301 [TVE] Channel scan 83/209 SCIENCE successful
2020/10/10 00:49:34.545129 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:34.554342 [TVE] action=mock
2020/10/10 00:49:34.556325 [TVE] action=set_cookies domains=1
2020/10/10 00:49:34.557590 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=ahc&redirect_uri=https%3A%2F%2Fwww.ahctv.com%2Fwatch%2Fahc&response_flow=code&response_type=adobe&state=nonce%2CQiTgoXd3floiE3bJNYxeYNl0egOzK9Fc
2020/10/10 00:49:34.560546 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:34.910784 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:35.091716 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:35.278426 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:35.617461 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:35.617632 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:36.229519 [TVE] action=wait_for_page
2020/10/10 00:49:36.276937 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:39.505741 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:39.687101 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:40.865696 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:41.340631 [TVE] action=request type=Document method=GET url=https://www.ahctv.com/watch/ahc
2020/10/10 00:49:41.505741 [TNR] Opened connection to 131F8A20/1 for ch1006 WJBFDT
2020/10/10 00:49:41.862469 [HLS] Starting transcoder for channel 1006 from 192.168.24.13 (encoder=remux, resolution=, deinterlacer=, bitrate=0)
2020/10/10 00:49:42.452879 [HLS] Probed live stream in 584.576067ms: h264 1280x720 progressive 1764409bps
2020/10/10 00:49:42.580225 [TVE] action=authed
2020/10/10 00:49:42.580411 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:42.592511 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:43.196427 [TVE] Channel scan 84/209 AHC successful
2020/10/10 00:49:43.384130 [HLS] Session ch1006-dANY-ip192.168.24.13 started in 2.469176949s
2020/10/10 00:49:43.412419 [ENC] Starting encoder for ch1006 in /media/data/Media/ChannelsDVR/Streaming/ch1006-dANY-ip192.168.24.13-249634983/encoder-1-691530714 at 1 (1.323911) (encoder=h264_v4l2m2m, resolution=576, deinterlacer=blend, bitrate=1000 segment_size=0.01)
2020/10/10 00:49:43.488023 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264 @ 0xa761840] Increasing reorder buffer to 2
2020/10/10 00:49:44.579228 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:44.588119 [TVE] action=mock
2020/10/10 00:49:44.589932 [TVE] action=set_cookies domains=1
2020/10/10 00:49:44.591090 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=dam&redirect_uri=https%3A%2F%2Fwww.destinationamerica.com%2Fwatch%2Fdestination-america&response_flow=code&response_type=adobe&state=nonce%2CrOIXKZTy78CPQkfqoZXpZITWnB0iIBoq
2020/10/10 00:49:44.593855 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:44.938669 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:45.119223 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:45.316525 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:45.634476 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set number of B-frames: Invalid argument
2020/10/10 00:49:45.634541 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set number of B-frames
2020/10/10 00:49:45.634612 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set header mode: Invalid argument
2020/10/10 00:49:45.634655 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set gop size: Invalid argument
2020/10/10 00:49:45.634686 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Encoder adjusted: qmin (0), qmax (51)
2020/10/10 00:49:45.634739 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set minimum video quantizer scale: Invalid argument
2020/10/10 00:49:45.634770 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [h264_v4l2m2m @ 0xa802b10] Failed to set maximum video quantizer scale: Invalid argument
2020/10/10 00:49:45.728672 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:45.728753 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:46.301871 [TVE] action=wait_for_page
2020/10/10 00:49:46.389137 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:49.227918 [ENC] Segment 4 has unexpected duration: inputs=6-7 expected=0.950633 actual=1.001 expected_pts=4.376967-5.311233 actual_pts=4.348244-5.332567
2020/10/10 00:49:49.739941 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:49.878470 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:49:51.161213 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:49:51.577083 [TVE] action=request type=Document method=GET url=https://www.destinationamerica.com/watch/destination-america
2020/10/10 00:49:52.885950 [TVE] action=authed
2020/10/10 00:49:52.886238 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:49:52.899978 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:49:53.474970 [TVE] Channel scan 85/209 DESTINATION-AMERICA successful
2020/10/10 00:49:53.894731 [TVE] Channel scan 86/209 DISCOVERY-LIFE failed: notAuthorized: This network is not included in your current television subscription. Please contact your TV provider to subscribe.
https://my.xfinity.com/upgrade
2020/10/10 00:49:55.239228 [TVE] action=set_cookies domains=1 cookies=9
2020/10/10 00:49:55.256928 [TVE] action=mock
2020/10/10 00:49:55.259520 [TVE] action=set_cookies domains=1
2020/10/10 00:49:55.268233 [TVE] action=navigate url=https://login.discovery.com/v1/oauth2/authorize?affiliate_id=57a9eb716b66d14b2ff8aedb&asset_authz=true&auth_client_id=Comcast_SSO&auth_client_url=&auth_ttl=1209600&client_id=3020a40c2356a645b4b4&network_requestors=true&product.code=vel&redirect_uri=https%3A%2F%2Fwatch.motortrend.com%2Fwatch%2Fmotortrend&response_flow=code&response_type=adobe&state=nonce%2CtfVeobkyo72ebBhH9uhcEqdMFn5luDCv
2020/10/10 00:49:55.276680 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize
2020/10/10 00:49:55.660444 [TVE] action=request type=Document method=GET url=https://api.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:55.828965 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/api/v1/authenticate
2020/10/10 00:49:56.008210 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:49:56.256737 [TVE] action=request type=Document method=GET url=https://login.xfinity.com/login
2020/10/10 00:49:56.256831 [TVE] action=auth_domain domain=login.xfinity.com
2020/10/10 00:49:56.582679 [TVE] action=wait_for_page
2020/10/10 00:49:56.675398 [TVE] action=error_response type=Script error=net::ERR_CONNECTION_REFUSED
2020/10/10 00:49:57.404152 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [mpegts @ 0xa7ae7a0] H.264 bitstream error, startcode missing, size 0
2020/10/10 00:49:58.858489 [SCN] Imported TV show Adventure Time/Season 1/Adventure Time - S01E14 - The Witch's Garden (Bluray-1080p).mkv
2020/10/10 00:49:59.958499 [TVE] action=request type=Document method=GET url=https://oauth.xfinity.com/oauth/authorize
2020/10/10 00:50:00.092431 [TVE] action=request type=Document method=GET url=https://sp.auth.adobe.com/adobe-services/oauth2
2020/10/10 00:50:00.986499 [TVE] action=request type=Document method=GET url=https://login.discovery.com/v1/oauth2/authorize/callback/adobe
2020/10/10 00:50:01.433982 [TVE] action=request type=Document method=GET url=https://watch.motortrend.com/watch/motortrend
2020/10/10 00:50:01.572526 [HLS] ffmpeg: ch1006-dANY-ip192.168.24.13-1----1000-0-576-0-0---false-false-0.01:  [mpegts @ 0xa7ae7a0] H.264 bitstream error, startcode missing, size 0
2020/10/10 00:50:02.189787 [TVE] action=authed
2020/10/10 00:50:02.189939 [TVE] action=wait_for_page done=true reason=auth_finished
2020/10/10 00:50:02.233127 [TVE] action=cookies num_domains=3 num_cookies=14
2020/10/10 00:50:06.479558 [HLS] Stopping transcoder session ch1006-dANY-ip192.168.24.13 (out: 25.431344s, finished: false)
2020/10/10 00:50:06.517533 [ENC] Stopped encoder for ch1006 in /media/data/Media/ChannelsDVR/Streaming/ch1006-dANY-ip192.168.24.13-249634983/encoder-1-691530714 after encoding 1 to 18
2020/10/10 00:50:06.517740 [TNR] Closed connection to 131F8A20/1 for ch1006 WJBFDT
2020/10/10 00:50:06.528422 [SNR] Statistics for ch1006 WJBFDT: ss=77% snq=91%,86%-93% seq=100% bps=4405090,3104256-5522688 pps=422,297-527

tmm1 · October 10, 2020, 8:02am

I've tried a couple 2.5" drives and adapters that all worked. Will have to find a 3.5" to see why that's any different.

Curious what command you're using to mount. I would like to make both network mounting and network file access easier to setup from the web UI.

I will check out the transcoding issues.

timstephens24 · October 10, 2020, 9:07am

Here’s what I’m using:
mount -t cifs -o username=redacted,password=redacted,uid=501,gid=501,iocharset=utf8 //192.168.1.2/sharename /media/sharename

hancox · October 10, 2020, 11:40am

Screw it. Been looking for an excuse to get a proper USB3 hard drive for the pi anyway (using a repurposed old USB2 now, without too much fuss).

I'll go for it on Monday, most likely.

Bob_Prendergast · October 11, 2020, 4:52pm

how long till out of beta? I'm interested in switching my whole setup to a Raspberry Pi

jdts · October 13, 2020, 4:27am

How does transcoding/comskip performance compare to the Rasbian DVR build? ARM64 presumably improves it somewhat, but my PI does many other things...

hancox · October 13, 2020, 12:02pm

struggling with this question too.

For those with existing installs, it would also be helpful to install to the SD card. Any reason the image can't be applied there?

dlpoole · October 13, 2020, 2:03pm

Why wait? The DVR server functions perfectly over Raspian on a Pi for use with the streaming client apps. I've had one running for months, first on a Pi Model B, now on a Pi4. Transcoding for remote access and web browser clients hasn't worked yet for me even on the Pi4 but maybe the devs can overcome that yet. Faster transcoding, if it ultimately works, would be the only advantage to someone unafraid of installing Raspian, then the existing server. You could always repurpose a Raspian-based Pi server with the new one at the inconvenience of losing any recorded programs on the repurposed hard drive.

racameron · October 13, 2020, 2:04pm

No, but USB3 is faster and much more reliable than SD cards.

hancox · October 13, 2020, 2:45pm

...but the "DVR" directory would still be sourced from USB3, in my setup. The only missing piece is the OS, which i would wonder if there would be much benefit, anyway. It would also negate the need to move stuff on/off the USB drive that's in use now.

tmm1 · October 13, 2020, 3:20pm

We have decided not to support SD cards with this release because they all eventually die. Also USB3 is simply much faster and results in faster boots and a snappier system in general.

If you have a working Raspbian setup, just keep using that. Right now there's no advantage to switching. (But note that if you installed the dvr software to the SD card, it means that the guide databases are being downloaded to the SD card every morning and will wear it out eventually.)

jdts · October 13, 2020, 3:43pm

Thanks. I thought I recalled that the armv8 architecture would provide access to more CPU optimizations? The Raspbian folks are in fact moving towards releasing a parallel 64bit version.

I put both the Channels data drive and the channels-dvr directory together on an external USB drive, so that the SD card is barely touched. I think this would also work fine without requiring USB boot.

tmm1 · October 13, 2020, 3:52pm

This is what we recommend, but have found most users are unable to make it work because they don't know how to reformat the drive to use a file system that supports symlinks.

The goal of our dedicated image is to make this all much easier for new users who are not as technical, and simply want an affordable dvr box they can setup with limited expertise and time.

Bob_Prendergast · October 13, 2020, 4:20pm

I have a RP-4 kit coming today.. which install do you recommend? I’m looking to move everything off my iMac and have a dedicated box for channels DVR

tmm1 · October 13, 2020, 4:23pm

Currently you're using the armv7l builds of the DVR, but they specifically have NEON enabled. I don't know offhand if armv8l or aarch64 provide any additional benefits, but my guess is the 64bit binaries will provide a nice boost. I plan to do some benchmarks once things have stabilized a bit.

On the transcoding front, OMX/MMAL are only available on 32bit so this image is the first to use V4L2. I imagine that too will provide some small performance gains compared to OMX, but we have to work through the bugs first since it's not as stable as the raspbian h264_omx currently being used.

tmm1 · October 13, 2020, 4:30pm

I would use the instructions above for all new/dedicated DVR Pis

VTTom · October 13, 2020, 4:45pm

Seems to me this gets you -> <- this close to having a low-cost turn-key DVR hardware product. I wonder if there's a business case to be made here?

briankmurphy · October 13, 2020, 5:39pm

Any possibility of being able to load the linux version of Kodi on here as well that with the Channels plug-in could provide a dedicated client to go along with the server piece?

tmm1 · October 13, 2020, 6:24pm

No chance of that here. We are keeping this tightly scoped to a headless server you plug into your router next to the hdhomerun.

Technically speaking, decoding and playback over HDMI is an entirely different beast and adds a huge amount of complexity. If you really wanted an all-in-one, you might be better off loading Android TV OS onto the RPi4.

Our focus for playback remains on the Channels app for Apple TV and Android TV. These devices are readily available, often on sale, and give you access to the Channels app right next to Netflix, HBO Max, Peacock, etc.