ChannelsDVR on unRaid, dead after reboot [Solved]

SquelchOil · April 4, 2017, 4:55pm

Overall my channels DVR is working great. However, I’ve had a minor issue since I started using it. When I reboot the unRaid server, channels shows under settings as being in a “running” state, but the web page is unavailable. Simply changing running to stopped and back to running fixes the problem until the next reboot. I don’t reboot very often, so the only reason this is actually a concern to me is that if we have power outage or unplanned reboot while I’m away…there is no way in hell my wife is going to remember how to fix this!

So, if anyone knows of a workaround, I’d be very interested, and if the dev’s want any debug info, just let me know what would be helpful and I’ll get it posted. I can recreate the condition on demand.

tmm1 · April 4, 2017, 5:23pm

Can you check the Log from around the time of the reboot and see if there’s anything in there?

SquelchOil · April 4, 2017, 5:35pm

Here is the log covering the time of reboot:
2017/04/04 11:04:35 [SYS] Shutting down…
2017/04/04 11:04:35 [DVR] Recording engine stopped.
2017/04/04 11:25:48 [SYS] Starting Channels DVR v2017.04.04.0009 (linux-x86_64) in /mnt/user/channels-dvr/data
2017/04/04 11:25:48 [HDR] Found 1 devices
2017/04/04 11:25:49 [SYS] Started HTTP Server
2017/04/04 11:25:51 [DVR] Recording engine started in /mnt/user/Recordings
2017/04/04 11:25:51 [ERR] bonjour: no suitable IPv6 interface: listen udp6 [ff02::]:5353: socket: address family not supported by protocol
2017/04/04 11:25:51 [SYS] Bonjour service running for dvr-tower.local. [192.168.86.166 192.168.122.1]
2017/04/04 11:25:51 [DVR] Waiting 1h34m8.264622173s until next job 1491328800-2
2017/04/04 11:25:51 [SYS] Created database snapshot: backup-20170404.112551
2017/04/04 11:25:52 [SYS] Removing old backup backup-20170324.140550
2017/04/04 11:26:01 [IDX] Pruned 13 expired airings from USA-OTA72758 in 206.56891ms.

The entry just prior to this list has a 9am time stamp, so it’s several hours earlier. Looking at the logs time stamps it seems like unRaid is showing the channels DVR in a running state under settings after reboot, but there isn’t a matching Channels log entry showing that it was actually started. Only once I “stopped” and “started” it does the “starting” entry appear at 11:25.

tmm1 · April 4, 2017, 5:39pm

Okay, I suspect I know where the issue is, but would be great to confirm so I can put in a fix.

Can you go ahead and reboot the box, and then when it comes up SSH in and run some commands:

ls -alh /var/run/channels-dvr.pid
cat /var/run/channels-dvr.pid
ps $(cat /var/run/channels-dvr.pid)
pidof channels-dvr
ps aux | grep channels-dvr

SquelchOil · April 4, 2017, 6:00pm

Here is the output:
Linux 4.9.19-unRAID.
root@Tower:~# ls -alh /var/run/channels-dvr.pid
-rw-rw-rw- 1 root root 5 Apr 4 12:42 /var/run/channels-dvr.pid
root@Tower:~# cat /var/run/channels-dvr.pid
2785
root@Tower:~# ps $(cat /var/run/channels-dvr.pid)
PID TTY STAT TIME COMMAND
root@Tower:~# pidof channels-dvr
root@Tower:~# ps aux | grep channels-dvr
root 11821 0.0 0.0 9640 1896 pts/0 S+ 12:58 0:00 grep channels-dvr

tmm1 · April 4, 2017, 6:02pm

Great, a couple more:

ps $(cat /var/run/channels-dvr.pid); echo $?
pidof channels-dvr; echo $?
pidof init; echo $?
/etc/rc.d/rc.channels-dvr start; echo $?

SquelchOil · April 4, 2017, 6:04pm

root@Tower:~# ps $(cat /var/run/channels-dvr.pid); echo $?
PID TTY STAT TIME COMMAND
1
root@Tower:~# pidof channels-dvr; echo $?
1
root@Tower:~# pidof init; echo $?
1
0
root@Tower:~# /etc/rc.d/rc.channels-dvr start; echo $?
0
root@Tower:~#

tmm1 · April 4, 2017, 6:07pm

Okay. Can you run this to remove the pid file:

rm /var/run/channels-dvr.pid

Then reboot and see if the DVR starts up automatically.

SquelchOil · April 4, 2017, 6:17pm

I issued the command and rebooted. It’s been back up for about 7m and Channels is unresponsive via GUI. It shows as running under settings. I’ve done nothing else yet in case there is something else you’d like me to try.

tmm1 · April 4, 2017, 6:19pm

Okay, it should have come up right away.

Try this again:

ls -alh /var/run/channels-dvr.pid
cat /var/run/channels-dvr.pid

SquelchOil · April 4, 2017, 6:22pm

root@Tower:~# ls -alh /var/run/channels-dvr.pid
-rw-rw-rw- 1 root root 5 Apr 4 13:10 /var/run/channels-dvr.pid
root@Tower:~# cat /var/run/channels-dvr.pid
2786

tmm1 · April 4, 2017, 6:28pm

Hrm ok. How about:

find /mnt/user/channels-dvr -ls | grep -v USA

SquelchOil · April 4, 2017, 6:32pm

root@Tower:~# find 10 0 drwxrwxrwx 1 nobody users 15 0 drwxr-xr-x 1 root root 82 968 -rw-rw-rw- 1 root root 83 32 -rw------- 1 root root 84 520 -rw------- 1 root root 85 0 drwxrwxrwx 1 root root 86 4 -rw-r–r-- 1 root 87 19772 -rwxr-xr-x 1 root 88 13048 -rwxr-xr-x 1 root 89 13988 -rwxr-xr-x 1 root 90 13924 -rwxr-xr-x 1 root 91 4 -rw-r–r-- 1 root 92 0 drwxrwxrwx 1 root root 93 4 -rw-r–r-- 1 root 94 13048 -rwxr-xr-x 1 root 95 13992 -rwxr-xr-x 1 root 96 13928 -rwxr-xr-x 1 root 97 19772 -rwxr-xr-x 1 root 98 4 -rw-r–r-- 1 root 99 0 drwxrwxrwx 1 root root 100 4 -rw-r–r-- 1 root 101 19788 -rwxr-xr-x 1 root 102 13048 -rwxr-xr-x 1 root 103 13992 -rwxr-xr-x 1 root 104 13928 -rwxr-xr-x 1 root 105 4 -rw-r–r-- 1 root 106 0 lrwxrwxrwx 1 root root@Tower:~# /mnt/user/channels-dvr -ls | grep -v USA
110 Jan 28 03:42 /mnt/user/channels-dvr
78 Feb 8 14:19 /mnt/user/channels-dvr/data
988735 Apr 4 12:41 /mnt/user/channels-dvr/data/channels-dvr.log
65536 Apr 4 12:31 /mnt/user/channels-dvr/data/settings.db
532480 Apr 4 12:31 /mnt/user/channels-dvr/data/recorder.db
116 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308
root 300 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/linux-x86_64.sha256
root 20243661 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/channels-dvr
root 13358096 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/comskip
root 14321264 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/ffmpeg
root 14255408 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/ffprobe
root 15 Mar 30 23:06 /mnt/user/channels-dvr/2017.03.31.0308/version
116 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134
root 300 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/linux-x86_64.sha256
root 13358096 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/comskip
root 14325360 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/ffmpeg
root 14259504 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/ffprobe
root 20243661 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/channels-dvr
root 15 Mar 31 21:07 /mnt/user/channels-dvr/2017.03.31.2134/version
116 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009
root 300 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/linux-x86_64.sha256
root 20260853 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/channels-dvr
root 13358096 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/comskip
root 14325360 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/ffmpeg
root 14259504 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/ffprobe
root 15 Apr 3 21:07 /mnt/user/channels-dvr/2017.04.04.0009/version
root 15 Apr 3 21:07 /mnt/user/channels-dvr/latest -> 2017.04.04.0009

tmm1 · April 4, 2017, 6:34pm

Thanks, getting pretty close to finding the fix.

Can you go ahead and stop/start via the web UI, so the DVR actually comes up.

Then run these:

cat /var/run/channels-dvr.pid
pidof channels-dvr
test -f /var/run/channels-dvr.pid -a “$(cat /var/run/channels-dvr.pid)” = “$(pidof channels-dvr)”; echo $?

SquelchOil · April 4, 2017, 6:38pm

Restarted Channels, Web GUI is up.
Commands issued:
root@Tower:~# cat /var/run/channels-dvr.pid
14258
root@Tower:~# pidof channels-dvr
14258
root@Tower:~# test -f /var/run/channels-dvr.pid -a “$(cat /var/run/channels-dvr.pid)” = “$(pidof channels-dvr)”; echo $?
0

I’ve got to run out for a bit. I’ll check back in on this later this afternoon.

tmm1 · April 4, 2017, 6:55pm

No problem. Try to install this new version of the plugin, it should fix the issue:

https://channels-dvr.s3.amazonaws.com/unraid/channels-dvr.plg

SquelchOil · April 4, 2017, 10:13pm

Well, that didn’t go so smoothly. I installed the plug in and cannot get Channels to run. It shows as running, but no GUI. When I select “Stopped” state and hit apply though it just comes back showing it’s still running. I tried a reboot but that didn’t help, but it did show there had been some sort of error during the install. At that point Channels was no longer showing under plug ins or settings, so I tried installing again. Here’s the unRaid log, not a lot of detail but clearly something went wrong:

Apr 4 17:04:00 Tower emhttp: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin install https://channels-dvr.s3.amazonaws.com/unraid/channels-dvr.plg
Apr 4 17:04:01 Tower root: plugin: skipping: /boot/packages/channels-dvr-1.0.4-x86_64-1.tgz already exists
Apr 4 17:04:01 Tower root: plugin: running: /boot/packages/channels-dvr-1.0.4-x86_64-1.tgz
Apr 4 17:04:25 Tower root: plugin: running: anonymous
Apr 4 17:04:42 Tower shfs/user: err: shfs_rmdir: rmdir: /mnt/disk2/Recordings/Database/backup-20170328.150637 (39) Directory not empty
Apr 4 17:06:02 Tower emhttp: cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin_rm /boot/config/plugins-error/channels-dvr.plg

tmm1 · April 4, 2017, 10:34pm

Yuck.

Try this over SSH and see if the server comes up:

/etc/rc.d/rc.channels-dvr start; echo $?

SquelchOil · April 4, 2017, 10:37pm

Yes, it sure did.

tmm1 · April 4, 2017, 10:42pm

Okay cool. Can you also try rebooting now and see if it comes up or not?