Crash on last beta server, lost everything

[CScott]

Went through a remove , reinstall. The normal routine for a crash. Been through this twice before.

The only backup available to me were from June, lost all my collections, virtual channels. As well as the last few months of recordings. Even having issues trying to re-establish DirectTV Stream which had approx 180 TVEs channels now down to 104.

I'm at a lost how to proceed

Is there a way to have the server re-scan the drives for the latest up to date recordings?

Got the DirectTV Stream back my uninstalling the TV Everywhere for it then reinstalling it

[tmm1]

Huh, what crashed? Your computer? Your drive? I don't understand why the recent backups would be lost.

[CScott]

the server.
a show i was watching stopped
i tried watching the show again nd the beta server could not be found

[tmm1]

Okay. Did you try to restart the server at that point, via start menu? Did you save any of the logs before you removed/resinstalled?

What is in the Database folder next to your recordings? Any folders in there created after june?

[CScott]

The last errors I got screen capture fragment

Then I attempted to stop the server and restart it
Got this

Then uninstalled rebooted the computer reinstalled server
Things went down hill from there

[CScott]

No other folders on any either of the drives were newer than 6/11/2021

Now this ive got it semi-up and running there is a current one for today's date. But that is all. There should have been 3 for today alone

---

The system has been odd all week (random memory errors, I made a post see below-but I doubt that had anything to do with this) but I'd attributed it to the last Tuesday windows update

Commercial skip suddenly less accurate

Perhaps the server is getting bogged down and causing this? I've noticed for me, commercials are not being recognized as well when they then occur while the server is feeding multiple tv's at the same time where upon tv's connected via WiFi begin 2+ second pausing of streams. Perhaps a pause button for commercial skipping or a defer until idle bottom? Looking at the log during those times I see this 2021/09/24 23:41:01.778907 [HLS] ffmpeg: file21593-ip10.0.0.209: [NULL @ 0000000000fdd300] i…

[CScott]

I can rebuild the collections and the 96 virtual channels

Thing is my wife really wanted to watch the Season Premiere of the Rookie that premiered last night.. and I've no clue how to get it to show in the library now

[tmm1]

You would need to move TV/Rookie/xxx.mpg to Imports/TV/Rookie/xxx.mpg

[CScott]

Ok I just located a second DATABASE folder on the H: drive
It contains backups for today and yesterday

What my best course of action? Reinstall?

[tmm1]

Open http://x.x.x.x:8089/restore and pick the latest backup

[CScott]

Perfect, thanks tmm1!

[CScott]

Awesome, it's all back!

Thanks

(I'm remembering now there was a database change awhile back, is that why there were 2 database folders and should I remove the June one?)

[tmm1]

The backups are stored on whatever your primary DVR storage location is. You must have swapped drives in June or something. Yea you can delete that old Database folder.

[CScott]

Your right, I believe I did add a new main drive about then

[CScott]

The exact same issue is occurring again.

The server is in an endless loop
Error:
goroutine 198 [running]:
github.com/fancybits/channels-server/transcoder.(*ExtraData).UnmarshalJSON(0xc0011205f8, {0xc001428b53, 0xc4, 0xc00152da00})
github.com/fancybits/channels-server/transcoder/ffprobe.go:275 +0x38f
encoding/json.(*decodeState).literalStore(0xc000eb4120, {0xc001428b53, 0xc4, 0x12ad}, {0x249e440, 0xc0011205f8, 0x26c08a0}, 0x0)
encoding/json/decode.go:871 +0x1f83
encoding/json.(*decodeState).value(0xc000eb4120, {0x249e440, 0xc0011205f8, 0x9})
encoding/json/decode.go:387 +0x126
encoding/json.(*decodeState).object(0xc000eb4120, {0x26c08a0, 0xc001120488, 0xb})
encoding/json/decode.go:774 +0xca5
encoding/json.(*decodeState).value(0xc000eb4120, {0x26c08a0, 0xc001120488, 0x2})
encoding/json/decode.go:373 +0x45
encoding/json.(*decodeState).array(0xc000eb4120, {0x2454980, 0xc0003362d0, 0xc00005f801})
encoding/json/decode.go:561 +0x626
encoding/json.(*decodeState).value(0xc000eb4120, {0x2454980, 0xc0003362d0, 0xc0014281d0})
encoding/json/decode.go:363 +0x7e
encoding/json.(*decodeState).object(0xc000eb4120, {0x255fb80, 0xc000336240, 0x0})
encoding/json/decode.go:774 +0xca5
encoding/json.(*decodeState).value(0xc000eb4120, {0x255fb80, 0xc000336240, 0x90})
encoding/json/decode.go:373 +0x45
encoding/json.(*decodeState).unmarshal(0xc000eb4120, {0x255fb80, 0xc000336240})
encoding/json/decode.go:180 +0x1de
encoding/json.Unmarshal({0xc001428000, 0x58, 0x0}, {0x255fb80, 0xc000336240})
encoding/json/decode.go:107 +0x125
github.com/fancybits/channels-server/transcoder.ProbeFile({0xc000c3e660, 0x1})
github.com/fancybits/channels-server/transcoder/ffprobe.go:152 +0x69
github.com/fancybits/channels-server/dvr.(*File).ProcessProbe(0xc000c19400)
github.com/fancybits/channels-server/dvr/file.go:1405 +0x94
github.com/fancybits/channels-server/dvr.(*File).Process(0x273de61)
github.com/fancybits/channels-server/dvr/file.go:1255 +0x1e
github.com/fancybits/channels-server/dvr.(*Recorder).RunProcessor.func3(0xc000c19400)
github.com/fancybits/channels-server/dvr/recorder.go:463 +0x152
github.com/fancybits/channels-server/dvr.(*Recorder).EachFilteredFile(0xc000351200, 0x270b206, 0xc0009b5dd8)
github.com/fancybits/channels-server/dvr/db_files.go:265 +0x15a
github.com/fancybits/channels-server/dvr.(*Recorder).EachFile(...)
github.com/fancybits/channels-server/dvr/db_files.go:230
github.com/fancybits/channels-server/dvr.(*Recorder).RunProcessor(0xc000351200)
github.com/fancybits/channels-server/dvr/recorder.go:451 +0x796
created by github.com/fancybits/channels-server/dvr.(*Recorder).Run
github.com/fancybits/channels-server/dvr/recorder.go:221 +0x129
2021/09/28 07:52:14.590135 [SYS] Starting Channels DVR v2021.09.27.0039 (windows-x86_64 pid:4772) in C:\ProgramData\ChannelsDVR\data
2021/09/28 07:52:14.590135 [SYS] Running as a Windows system service.
2021/09/28 07:52:15.671660 [SYS] Started HTTP Server
2021/09/28 07:52:16.230271 [HDR] Found 2 devices
2021/09/28 07:52:16.622913 [TVE] action=cbs_station err=unavailable in location
2021/09/28 07:52:16.669904 [M3U] Refreshed lineup for PlutoTV with 329 channels
2021/09/28 07:52:16.762843 [M3U] Refreshed lineup for WEBCAMExamples with 3 channels
2021/09/28 07:52:16.980277 [M3U] Refreshed lineup for PBSStations with 140 channels
2021/09/28 07:52:16.989653 [M3U] Refreshed lineup for StirrTV with 109 channels
2021/09/28 07:52:17.037254 [M3U] Refreshed lineup for SamsungTVPlus with 182 channels
2021/09/28 07:52:17.304632 [DVR] Recording engine started in H:\ChannelsDVR
2021/09/28 07:52:17.325558 [DVR] Starting job 1632828590-154 Painting and Travel With Roger & Sarah Bansemer on ch=[30.3]
2021/09/28 07:52:17.325558 [DVR] Waiting 7m42.6744415s until next job 1632830400-98 Murdoch Mysteries
2021/09/28 07:52:17.325558 [DBG] Scheduling wake timer for scheduled recording in 6m12.6744415s
2021/09/28 07:52:17.325558 [SYS] Bonjour service running for dvr-windows10-pc.local. [10.0.0.76 172.23.80.1]
2021/09/28 07:52:19.301963 [DVR] Processing file-22367: TV\The X-Files\The X-Files S03E23 1996-05-10 Wetwired 2021-09-27-2159.mpg
panic: runtime error: slice bounds out of range [:50] with length 12

goroutine 112 [running]:
github.com/fancybits/channels-server/transcoder.(*ExtraData).UnmarshalJSON(0xc0002f05f8, {0xc000320b53, 0xc4, 0xc00100ba00})
github.com/fancybits/channels-server/transcoder/ffprobe.go:275 +0x38f
encoding/json.(*decodeState).literalStore(0xc000bb4120, {0xc000320b53, 0xc4, 0x12ad}, {0x249e440, 0xc0002f05f8, 0x26c08a0}, 0x0)
encoding/json/decode.go:871 +0x1f83
encoding/json.(*decodeState).value(0xc000bb4120, {0x249e440, 0xc0002f05f8, 0x9})
encoding/json/decode.go:387 +0x126
encoding/json.(*decodeState).object(0xc000bb4120, {0x26c08a0, 0xc0002f0488, 0xb})
encoding/json/decode.go:774 +0xca5
encoding/json.(*decodeState).value(0xc000bb4120, {0x26c08a0, 0xc0002f0488, 0x2})
encoding/json/decode.go:373 +0x45
encoding/json.(*decodeState).array(0xc000bb4120, {0x2454980, 0xc000cb4150, 0xc000217c01})
encoding/json/decode.go:561 +0x626
encoding/json.(*decodeState).value(0xc000bb4120, {0x2454980, 0xc000cb4150, 0xc0003201d0})
encoding/json/decode.go:363 +0x7e
encoding/json.(*decodeState).object(0xc000bb4120, {0x255fb80, 0xc000cb40c0, 0x2030003})
encoding/json/decode.go:774 +0xca5
encoding/json.(*decodeState).value(0xc000bb4120, {0x255fb80, 0xc000cb40c0, 0x90})
encoding/json/decode.go:373 +0x45
encoding/json.(*decodeState).unmarshal(0xc000bb4120, {0x255fb80, 0xc000cb40c0})
encoding/json/decode.go:180 +0x1de
encoding/json.Unmarshal({0xc000320000, 0x58, 0x0}, {0x255fb80, 0xc000cb40c0})
encoding/json/decode.go:107 +0x125
github.com/fancybits/channels-server/transcoder.ProbeFile({0xc000fc89c0, 0x27})
github.com/fancybits/channels-server/transcoder/ffprobe.go:152 +0x69
github.com/fancybits/channels-server/dvr.(*File).ProcessProbe(0xc001291800)
github.com/fancybits/channels-server/dvr/file.go:1405 +0x94
github.com/fancybits/channels-server/dvr.(*File).Process(0x273de61)
github.com/fancybits/channels-server/dvr/file.go:1255 +0x1e
github.com/fancybits/channels-server/dvr.(*Recorder).RunProcessor.func3(0xc001291800)
github.com/fancybits/channels-server/dvr/recorder.go:463 +0x152
github.com/fancybits/channels-server/dvr.(*Recorder).EachFilteredFile(0xc0004bc600, 0x270b206, 0xc000c39dd8)
github.com/fancybits/channels-server/dvr/db_files.go:265 +0x15a
github.com/fancybits/channels-server/dvr.(*Recorder).EachFile(...)
github.com/fancybits/channels-server/dvr/db_files.go:230
github.com/fancybits/channels-server/dvr.(*Recorder).RunProcessor(0xc0004bc600)
github.com/fancybits/channels-server/dvr/recorder.go:451 +0x796
created by github.com/fancybits/channels-server/dvr.(*Recorder).Run
github.com/fancybits/channels-server/dvr/recorder.go:221 +0x129
2021/09/28 07:52:21.565615 [SYS] Starting Channels DVR v2021.09.27.0039 (windows-x86_64 pid:3488) in C:\ProgramData\ChannelsDVR\data
2021/09/28 07:52:21.565615 [SYS] Running as a Windows system service.
2021/09/28 07:52:22.650696 [SYS] Started HTTP Server
2021/09/28 07:52:23.210222 [HDR] Found 2 devices
2021/09/28 07:52:23.554782 [TVE] action=cbs_station err=unavailable in location
2021/09/28 07:52:23.573967 [M3U] Refreshed lineup for StirrTV with 109 channels
2021/09/28 07:52:23.657587 [M3U] Refreshed lineup for WEBCAMExamples with 3 channels
2021/09/28 07:52:23.789211 [M3U] Refreshed lineup for PBSStations with 140 channels
2021/09/28 07:52:24.099310 [M3U] Refreshed lineup for SamsungTVPlus with 182 channels
2021/09/28 07:52:24.149955 [M3U] Refreshed lineup for PlutoTV with 329 channels
2021/09/28 07:52:24.414952 [DVR] Recording engine started in H:\ChannelsDVR
2021/09/28 07:52:24.435421 [DVR] Starting job 1632828590-154 Painting and Travel With Roger & Sarah Bansemer on ch=[30.3]
2021/09/28 07:52:24.435421 [DVR] Waiting 7m35.5645789s until next job 1632830400-98 Murdoch Mysteries
2021/09/28 07:52:24.435593 [DBG] Scheduling wake timer for scheduled recording in 6m5.5645789s
2021/09/28 07:52:24.436666 [SYS] Bonjour service running for dvr-windows10-pc.local. [10.0.0.76 172.23.80.1]
2021/09/28 07:52:26.399039 [DVR] Processing file-22367: TV\The X-Files\The X-Files S03E23 1996-05-10 Wetwired 2021-09-27-2159.mpg
panic: runtime error: slice bounds out of range [:50] with length 12

goroutine 182 [running]:
github.com/fancybits/channels-server/transcoder.(*ExtraData).UnmarshalJSON(0xc00007a5f8, {0xc00167cb53, 0xc4, 0xc0009e0600})
github.com/fancybits/channels-server/transcoder/ffprobe.go:275 +0x38f
encoding/json.(*decodeState).literalStore(0xc000eae090, {0xc00167cb53, 0xc4, 0x12ad}, {0x249e440, 0xc00007a5f8, 0x26c08a0}, 0x0)
encoding/json/decode.go:871 +0x1f83
encoding/json.(*decodeState).value(0xc000eae090, {0x249e440, 0xc00007a5f8, 0x9})
encoding/json/decode.go:387 +0x126
encoding/json.(*decodeState).object(0xc000eae090, {0x26c08a0, 0xc00007a488, 0xb})
encoding/json/decode.go:774 +0xca5
encoding/json.(*decodeState).value(0xc000eae090, {0x26c08a0, 0xc00007a488, 0x2})
encoding/json/decode.go:373 +0x45

[CScott]

It continues to repeat that changing only gotoroutine number

It's in a continuous reboot
Server says : up to date then Booting...

Cannot even issue a x.x.x.z:8089/restore

Next I'll try a cpu reboot

[CScott]

Restarting the computer didn't work.

Uninstalled the server. To allow the restore to function

No recordings happened since at least 4am local yet: recording backup increments

Non of the backups appear uncorrupted. Rules are wrong, number of recordings are wrong.

A screen shot from yesterday..which could also be corrupted

[CScott]

Finally got it restored and running.. using a backup database rules/recordings close to that last screen shot from the day before yesterday

[CScott]

Created an auxiliary "backup" from this point so I've got a known backup to reference

[mjitkop]

Came here to confirm the same problem.

Since update yesterday afternoon (3:06 PM EST), server down, missed a good dozen of scheduled recordings.

As I type this email, trying to reinstall and restore from backup. I did the restore last night already but as soon as I opened the DVR Schedule page, it froze and no more running server after that.

@tmm1 if you need additional data from me, please let me know how I can help. Once I get it back and running again, I guess I can send diagnostics. Would that help in this case? Otherwise, I can share files from disk?