Unfortunately, the auto-upgrade process wasn’t successful for my FreeNAS system.
2017/01/27 18:11:44 [SYS] Downloading new version v2017.01.27.2325
2017/01/27 18:12:51 [SYS] Update downloaded and verified successfully.
2017/01/27 18:12:51 [SYS] Shutting down for upgrade from v2017.01.25.0441 to v2017.01.27.2325
2017/01/27 18:12:51 [DVR] Recording engine stopped.
… kernel: pid 11568 (channels-dvr), uid 0: exited on signal 4
Damn!
I wish it would coredump or something so we could more information on what’s causing the Illegal Instruction.
There’s another building coming tonight which will have the same issue. I will try the other fix I had in mind on Monday.
My system updated and restarted successfully twice tonight.
2017/01/27 20:43:40 [SYS] Downloading new version v2017.01.27.2325
2017/01/27 20:44:18 [SYS] Update downloaded and verified successfully.
2017/01/27 20:44:18 [SYS] Shutting down for upgrade from v2017.01.25.0441 to v2017.01.27.2325
2017/01/27 20:44:18 [DVR] Recording engine stopped.
2017/01/27 20:44:19 [SYS] Starting Channels DVR v2017.01.27.2325 (freebsd-x86_64) in /usr/pbi/channels-dvr-amd64/channels-dvr/data
and then 2 hours later…
2017/01/27 22:44:24 [SYS] Downloading new version v2017.01.28.0312
2017/01/27 22:45:01 [SYS] Update downloaded and verified successfully.
2017/01/27 22:45:01 [SYS] Shutting down for upgrade from v2017.01.27.2325 to v2017.01.28.0312
2017/01/27 22:45:01 [DVR] Recording engine stopped.
2017/01/27 22:45:02 [SYS] Starting Channels DVR v2017.01.28.0312 (freebsd-x86_64) in /usr/pbi/channels-dvr-amd64/channels-dvr/dataAfter the most recent update the plugin has been fine for 3 days now!
@pAvem3nt I wonder why you still have the error but others aren’t seeing it anymore. Can you post your FreeNAS version and the output of uname -a so we can compare to others?
Might be its still happening but its less intermittent now.
As requested:
FreeBSD channels 10.3-STABLE FreeBSD 10.3-STABLE #0 r295946+1805185(9.10.2-STABLE): Wed Jan 11 17:12:42 UTC 2017
However, great news, just tested forcing an upgrade from v2017.01.27.2325 to v2017.01.28.0312 and it was successful:
2017/01/28 11:22:36 [SYS] Downloading new version v2017.01.28.0312
2017/01/28 11:22:40 [IDX] Finished pruning 743 airings.
2017/01/28 11:23:41 [SYS] Update downloaded and verified successfully.
2017/01/28 11:23:41 [SYS] Shutting down for upgrade from v2017.01.27.2325 to v2017.01.28.0312
2017/01/28 11:23:41 [DVR] Recording engine stopped.
2017/01/28 11:23:42 [SYS] Starting Channels DVR v2017.01.28.0312 (freebsd-x86_64) in /channels-dvr/data
2017/01/28 11:23:43 [HDR] Found 3 devices
Just following up again – an automatic upgrade to v2017.01.30.2232 was successful today.
Thanks!
2017/01/30 15:27:48 [SYS] Update downloaded and verified successfully.
2017/01/30 15:27:48 [SYS] Shutting down for upgrade from v2017.01.28.0312 to v2017.01.30.2232
2017/01/30 15:27:48 [DVR] Recording engine stopped.
2017/01/30 15:27:49 [SYS] Starting Channels DVR v2017.01.30.2232 (freebsd-x86_64) in /channels-dvr/data
Another followup for me - My system restarted successfully when it upgraded to 2017.01.30.2232 today. That makes 3 updates in a row with no problem. Thanks for addressing it so quickly!
Woohoo!
Hate raining on the “woohoo” but I had shutdowns again the last two days😟.
Thanks for reporting. Seems like it got better at least, which is a good sign. Hope we can fix it for good soon.
Yeah the first few were working, but over the last 3 days it’s been hit or miss for me too. In case it helps…
v2017.01.31.2207 didn’t restart
v2017.02.01.0203 did restart
v2017.02.01.2149 did restart
v2017.02.02.0208 didn’t restart
v2017.02.02.2018 didn’t restart
Maybe I can come up with a workaround in the meantime, to restart the DVR automatically when it crashes.
On an unrelated note, can anyone with an Intel CPU in their FreeNAS setup run ls -alh /dev/dri and paste the output here. I want to see if we can support hardware accelerated transcoding on FreeBSD.
ls -alh /dev/dri
ls: /dev/dri: No such file or directory
I don’t see a “dri” (?) in the /dev/. This is an Intel FreeNas box.
OSFreebsd
10.3-stable (kernel: 10.3-stable)
CPU
4 cores / Intel® Core™ i5-2400 CPU @ 3.10GHz
load averages: 1.46 0.98 0.64
RAM16.67 GB
68.2% free
Trying to circle back to this upgrade issue on FreeNAS… I assume it’s still happening?
Can someone run this command and post output:
sysctl -a | grep core
I finally managed to get my development VM for FreeNAS working again, and sure enough the first upgrade I tried this hit bug.
Also discovered that the coredump folder doesn’t exist by default inside the jail (after jexec customplugin_1):
$ sysctl kern.corefile
kern.corefile: /var/db/system/cores/%N.core
$ ls -alh /var/db/system/cores
ls: /var/db/system/cores: No such file or directory
$ mkdir -p /var/db/system/cores
After creating the dir, the crash now coredumps so I can analyze it:
$ dmesg | tail -1
pid 13406 (channels-dvr), uid 0: exited on signal 4 (core dumped)
$ ls -lh /var/db/system/cores
total 1100
-rw------- 1 root wheel 14M Mar 7 18:12 channels-dvr.core
I’m loading the coredump into a debugger now so hopefully I can squash this once and for all soon.
Thanks - I do still see it from time to time. Do you still need the output of that command after getting a coredump? I can try it when I get home tonight.
I am able to reproduce this issue locally (on both FreeNAS 9.10.2 and the latest FreeBSD 11.0) so I don’t need anyone to run any commands anymore.
The bug itself is particularly nasty, but hopefully it can be squashed soon.
I think I finally tracked this down and fixed it. The changes could use a little more testing before I push them out to everyone via automatic update. If you want to help test run the command in Volunteers to test new DVR build?