Sounds pretty bad. One thing we could try is a kernel upgrade to something more modern. I can put together a test build tomorrow.
There are some reports on other forums about this being related to power. Just to rule this out, I'm going to try a powered hub tonight (it's the recommended drive and didn't need it for the past four months-- this just started-- but can't hurt to look into it.)
1 Like
Try the new OS beta (click and hold the OS update button). It has an updated exfat driver with some performance improvements.
Installed. Can't tell yet if it's working but will report back.
1 Like
Ntp hang from looks of it, when I upgraded. Hard power cycle revived. Submitted 0b4b8a49-5603-4a5b-849a-8636c09cd573
Still a mess. Skipping forward by 30 seconds takes 10 seconds while recording two things.
I can't find the Y cable I had so ordered a powered USB hub. Let's see if it helps. Will report back tomorrow.
Not better. Either from the new build or from the powered hub. Recording 4 TVE streams. Playing something else— skip forward twice is fine, the third 30 seconds leads to a 15 second delay. Rewinding 7 seconds takes over 15 seconds. Skip a commercial, 12 seconds.
Not sure what to make of this, but almost surely it’s an IO problem related to the exfat driver. I guess next I will point it to backup from rsync and clear the drive out with a brand new build, then slowly copy things back.
Can you submit diagnostics and I'll see if there are any errors from the driver
Thanks.
Diagnostics have been submitted as 236d597f-ff9d-4ce9-8517-681dc32a6a4e .
I submitted logs from the AppleTV too, in case that helps.
Something happened last night with a hard crash. This morning, no connection from any devices, the web interface, and even SSH. So I forced a reboot by pulling the plug and now I'm getting
2021/04/17 09:06:14.342819 [DIAG] Could not package /providers/playon/library: 404 Not Found
2021/04/17 09:06:19.458465 [DIAG] Could not find file: channels-comskip-controller.log: open channels-comskip-controller.log: no such file or directory
2021/04/17 09:06:26.880144 New diagnostic log submitted under 1204d120-55f8-4642-8a98-354a74810db6
2021/04/17 09:06:45.337283 [ERR] Could not create backup: stat /media/DVR: no such file or directory
Resubmitted diagnostics:
Logs have been submitted as 1204d120-55f8-4642-8a98-354a74810db6 .
This afternoon I decided to start again, restore the database, and leave the Pi with a clean HD, pointing to the backup drive for additional storage for anything recorded to date. Later, if this works, I can rsync things back to the Pi drive. I also gave up on the powered hub just to take one thing out of the mix since it was fine without a powered hub before this all started.
A couple of things happened that were strange starting again from the image.
-
The first time I updated the OS it failed and refused to reboot, so I reimaged and did it again.
-
I kept getting this problem:
on the third reboot, randomly it stopped.
- Adding my share was fine the first time, but after the first reboot, and now continuously there is an error connecting to the share automatically at reboot, and when I try manually I am getting:
I had no problem with
mount -t cifs //Server/volume /shares/volume
in SSH so I did that.
- When I restored from the old database, the latest version it offered was the last backup internal to channels (even though I had copied the entire drive to the backup drive and so the active database was fine). As a result everything that was recorded today has to be added in as Imports. Note to self: next time, do an internal backup.
(Diagnostics submitted again one last time
Logs have been submitted as eb9cae5d-0e82-4c98-bfb2-e19269b87b8a .
in case that helps.)
For now everything is running and it's finding my stuff. I'll report back later about watching something new while recording, which should all be from the USB drive, to see if the problem was created somehow by the drive filling up. If it works, after a few days I'll try to rsync back some of the recorded shows and take it from there.
So, everything is working faster. rsync is 10-20 MB/s which is at least twice as fast and it has stopped dropping to 500k/s. Something happened as the drive filled up. Not sure what the problem is but very happy things are back to normal. If it persists OK then I'll copy some stuff back a bit at a time. Thank you!
same thing here - was working earlier this morning, but hard down all of a sudden.
98b7d1a3-4b6a-4c89-8ec5-6b3f6fe03d94 submitted
2021/04/18 05:59:50.516408 [DVR] Recording for job 1618739990-66 from 131B163D ch783 into "TV/Eyewitness News Sunday at 600am/Eyewitness News Sunday at 600am 2021-04-18-0559.mpg" for 1h1m9.981278944s
2021/04/18 05:59:50.578318 [IDX] Generating video index for job 1618739990-66
2021/04/18 06:26:25.864653 [SYS] Starting Channels DVR v2021.04.15.2306 (linux-arm64 pid:337) in /mnt/data/channels-dvr/data
2021/04/18 06:26:26.075919 [SYS] Started HTTP Server
2021/04/18 06:26:26.121927 [SYS] Waiting on dependencies time-sync.target
2021/04/18 09:06:56.216316 [SYS] Done waiting on dependencies
2021/04/18 09:06:56.803943 [HDR] Found 3 devices
2021/04/18 09:06:59.947093 [DVR] Recording engine started in /media/DVR
2021/04/18 09:06:59.949612 [SYS] Bonjour service running for dvr-server.local. [192.168.50.56]
2021/04/18 09:06:59.996878 [DVR] Marking expired job 1618739990-66 Eyewitness News Sunday at 6:00am
2021/04/18 09:07:00.065724 [NAT] Successfully mapped port 8089 using natpmp
2021/04/18 09:07:00.361139 [ERR] Failed to read directory /media/DVR: open /media/DVR: no such file or directory
2021/04/18 09:07:00.369516 [ERR] Failed to read directory /media/DVR: open /media/DVR: no such file or directory
2021/04/18 09:07:00.387120 [ERR] Failed to read directory /media/DVR: open /media/DVR: no such file or directory
2021/04/18 09:07:01.293439 [ERR] Could not create backup: stat /media/DVR: no such file or directory
2021/04/18 09:07:11.243764 [IDX] Pruned 4032 expired airings from USA-CT56421-X in 469.803352ms.
2021/04/18 09:07:11.568754 [IDX] Pruned 2291 expired airings from USA-OTA06468 in 324.843759ms.
2021/04/18 09:07:12.077708 [IDX] Pruned 4301 expired airings from X-TVE in 508.829222ms.
One addition - i rebooted via the gui, in hopes it would help. It did, but my log had an interesting error after the shutdown:
2021/04/18 09:21:25.110727 [DVR] Recording engine stopped.
2021/04/18 09:21:25 [Recovery] 2021/04/18 - 09:21:25 panic recovered:
runtime error: invalid memory address or nil pointer dereference
runtime/panic.go:212 (0x44a503)
runtime/signal_unix.go:742 (0x44a39c)
github.com/fancybits/channels-server/dvr/db_rules.go:63 (0x112b764)
github.com/fancybits/channels-server/dvr/db_jobs.go:136 (0x112a827)
github.com/fancybits/channels-server/http_dvr_job.go:28 (0x1224b9b)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/fancybits/channels-server/http.go:234 (0x11e7d37)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/fancybits/channels-server/http.go:214 (0x11e79cb)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/fancybits/channels-server/http.go:206 (0x11e6e2b)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/gin-gonic/[email protected]/recovery.go:106 (0x944f1f)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/gin-gonic/[email protected]/logger.go:241 (0x94435b)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/gin-contrib/[email protected]/sessions.go:65 (0x119b2ff)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/gin-contrib/[email protected]/gzip.go:47 (0x1194cff)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/fancybits/channels-server/http.go:275 (0x120205f)
github.com/gin-gonic/[email protected]/context.go:165 (0x933c7f)
github.com/gin-gonic/[email protected]/gin.go:420 (0x93c7c3)
github.com/gin-gonic/[email protected]/gin.go:376 (0x93c093)
net/http/server.go:2843 (0x6b5f7b)
net/http/server.go:1925 (0x6b2173)
runtime/asm_arm64.s:1136 (0x4693b3)
2021/04/18 09:21:26.111052 [SYS] Goodbye.
Now the time is wonky. Even though I've set the time zone
all of the log entries are in the wrong time, and some of my custom channels don't work (maybe because of the timecode?)
Time seems to have fixed itself, but custom channels still no good.
Things still not really behaving with multiple recordings. I think the main problem though is comskip. Anecdotally multiple recordings alone seem to not be problematic?
Nope. Bunch of stuff interrupted. So slow that it took 15 seconds to open the web interface.
I'll try again with another physical drive just in case there are bad sectors or something, but this is just really annoying. I'm about to give up and use an old Mac mini.
Yeah, something is seriously wrong now - can't put my finger on it, but between the collections build of the DVR, the latest OS pre-release, and the recent manual chrome upgrades, things are getting worse.
Last night - recording The Voice on my Spectrum cable tuner. It's "still recording" when i wake up this morning.
Simultaneously watching same show on OTA, and buffer filled up for both OTA and cable source.
Running diagnostics, and everything is red on the summary page. "disk permissions" outright hangs.
2 hard reboots, and back to normal, but what a mess. submitted bbb762b3-ff59-4fc1-980e-f8cf50167545
even worse, the messages about buffering and whatnot didn't survive the reboots. sigh. I hope they came across in diagnostics @tmm1 ?
I'm on Raspberry Pi4.
I am also having this issue.
Turns out, I also had the same issue when using NextPVR.
How I fixed it was putting the operating system on an SD card and booting from that, and just using the HDD for recording storage.
I went 3 years after that without a single corrupted recording.
Is that possible to do with ChannelsDVR?
1 Like