New Option on Recording - Regenerate Streaming Index

Channels DVR Server
#14

I believe they playback normally. Since they are pretty large where would you suggest I upload them?

Here is the output of the regenerate command:

Regenerating 5137
{"ID":"5137","JobID":"1563152400-ch8.1","RuleID":"","GroupID":"11114803","Path":"TV/Grantchester on Masterpiece/Grantchester on Masterpiece S04E00 2019-07-14 2019-07-14-2100.mpg","Checksum":"","CreatedAt":1563152400,"Watched":false,"Deleted":false,"PlaybackTime":0,"Duration":7200.832456,"Commercials":[0,5.81,38.91,146.61,6718.81,6749.61,6795.52,7199.83],"Processed":true,"Airing":{"Source":"tms","Channel":"8.1","OriginalDate":"2019-07-14","Time":1563152400,"Duration":7200,"Title":"Grantchester on Masterpiece","Summary":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge.","Image":"http://fanc.tmsimg.com/assets/p11114803_b_h6_ap.jpg","Categories":["Episode","Series"],"Genres":["Crime drama","Mystery","Entertainment"],"Tags":["Season Premiere","New","HD","DVS","CC"],"SeriesID":"11114803","ProgramID":"EP025199990022","TeamIDs":null,"SeasonNumber":4,"EpisodeNumber":0,"Raw":{"startTime":"2019-07-15T01:00Z","endTime":"2019-07-15T03:00Z","duration":120,"channels":["8.1"],"stationId":"59210","qualifiers":["CC","DVS","HD 1080i","HDTV","New","Season Premiere","Stereo"],"ratings":[{"body":"USA Parental Rating","code":"TV14"}],"program":{"tmsId":"EP025199990022","rootId":"16910724","seriesId":"11114803","subType":"Series","title":"Grantchester on Masterpiece","titleLang":"en","seasonNum":4,"releaseYear":2019,"releaseDate":"2019-01-11","origAirDate":"2019-01-11","entityType":"Episode","descriptionLang":"en","shortDescription":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge.","longDescription":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge, and a murder sees racial tensions spike; Sidney struggles to remember a murdered woman's final words.","topCast":["James Norton","Robson Green","Tessa Peake-Jones"],"directors":["Tim Fywell"],"genres":["Crime drama","Mystery","Entertainment"],"preferredImage":{"uri":"http://fanc.tmsimg.com/assets/p11114803_b_h6_ap.jpg","height":"540","width":"720","primary":"true","category":"Banner-L1","text":"yes","tier":"Series"}}}},"PlayedAt":0,"UpdatedAt":1563193168044,"HighestPTS":0,"CommercialsAligned":true,"CommercialsEdited":true,"CommercialsVerified":false}
#15

You can upload to Firefox Send, Google Drive, Microsoft OneDrive, Dropbox or whatever is most convenient.

#16

Here is the link to the file. file_link

Let me know after you have downloaded so I can remove. Thanks.

#17

Thanks. I am downloading now.

I poked at the beginning of the file that downloaded and it looks like an mp4 file. Are you processing your recordings somehow?

#18

Yes. I use ffmpeg to wrap the mpeg in an mp4 container to add commercial markers so plex and see the chapters and allow you to skip in plex.

#19

Using this script

#20

Gotcha, okay. This is normal then. The new remuxer streaming index will not work on mp4 and other re-processed files.

#21

Ok. I tried running it on a file before I did the conversion and I got one of the errors you wanted us to look out for.

2019/07/16 09:50:50 [M3U8] Regenerating m3u8 for /Volumes/ChannelsDVR/recordings/Movies/The Wedding Singer (1998) 2019-07-16-0000.mpg

2019/07/16 09:51:29 [M3U8] m3u8 generation for /Volumes/ChannelsDVR/recordings/Movies/The Wedding Singer (1998) 2019-07-16-0000.mpg failed with MPEG-TS discontinuity: PTS jump was too far: 3445.095967
#22

Running Channels DVR v2019.07.17.1713
I regenerated the Streaming Index for 15 recordings a couple hours ago.
Looking at the processes running on my NAS I see 15 ffprobe processes listed.
Thinking that was one for each run I did, I reran it for one of the recordings again and I now have 16 ffprobe processes.
Appears they didn't terminate after they're finished?

#23

Nice catch!

1 Like
#24

The processes didn't terminate when I toggled the DVR OFF/ON in the web UI.
I had to stop and start the Channels DVR package from the Synology Package Center.

#25

Just curious how this processes a recording so fast.
When I did the "Regenerate Streaming Index" for the recordings (one by one manually), it only seemed to take a few seconds for each and the only way I could tell if it was done was watching the web UI->Settings->Activity.
The log only shows that it started, not if it was successful or finished.
Is it really that fast?

#26

The zombie ffprobe issue is fixed in the latest build.

The indexer is pretty fast- it is limited only by your HDD speed. If you can read 100MB/s off your drive and a typical 1h TVE recording is 1GB, it should only takes 10s to process.

#27

Thanks,

Re: The zombie ffprobe issue is fixed in the latest build.
Assume you mean 2019.07.18.0212? If so, will test.
Seems to work, kicked off regen on all recordings one after the other from the web UI without waiting.

Re: The log only shows that it started, not if it was successful or finished.
Can you add logging that shows if it was successful or not and finished?
Logs only show [M3U8] Regenerating m3u8 for...
No further info if successful or not or finished.

Thanks

#28

Are you ready to run this automatically on our old recordings?

#29

Are the index files being created with certain permissions set?

#30

I'm not sure, are you seeing something weird? We're using a new way of creating for these files so there might be something different compared to other files created by the DVR.

#31

Nothing weird, I think the index files just have user only rw permissions now.

#32

I just checked the files generated from 11 July when I reindexed a bunch of files when the feature was introduced, and they all have permissions of 600, just like the current beta produces.

#33

Currently running 2019.07.20.0419 and I'm seeing file perms of 000 for mine.

# ls -lR /volume1/arkives/ChannelsDVR/Streaming/m3u8/*/stream.m3u8
---------- 1 root root 210981 Jul 17 19:57 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1265/stream.m3u8
---------- 1 root root 109240 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1418/stream.m3u8
---------- 1 root root 114815 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1420/stream.m3u8
---------- 1 root root 112231 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1421/stream.m3u8
---------- 1 root root 114458 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1422/stream.m3u8
---------- 1 root root 110688 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1425/stream.m3u8
---------- 1 root root 117205 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1426/stream.m3u8
---------- 1 root root 110692 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1429/stream.m3u8
---------- 1 root root 107677 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1431/stream.m3u8
---------- 1 root root 116962 Jul 17 20:03 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1432/stream.m3u8
---------- 1 root root 112417 Jul 17 20:02 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1433/stream.m3u8
---------- 1 root root 115078 Jul 17 20:02 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1434/stream.m3u8
---------- 1 root root 108180 Jul 17 20:02 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1444/stream.m3u8
---------- 1 root root 116853 Jul 17 20:02 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1445/stream.m3u8
---------- 1 root root 109877 Jul 17 20:02 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1447/stream.m3u8
---------- 1 root root 107177 Jul 18 23:00 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1527/stream.m3u8
---------- 1 root root   5608 Jul 20 20:13 /volume1/arkives/ChannelsDVR/Streaming/m3u8/1533/stream.m3u8
# stat /volume1/arkives/ChannelsDVR/Streaming/m3u8/1533/stream.m3u8
  File: ‘/volume1/arkives/ChannelsDVR/Streaming/m3u8/1533/stream.m3u8’
  Size: 5608            Blocks: 16         IO Block: 4096   regular file
Device: fc00h/64512d    Inode: 428605450   Links: 1
Access: (0000/----------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2019-07-20 20:13:22.423086514 -0700
Modify: 2019-07-20 20:13:22.423086514 -0700
Change: 2019-07-20 20:13:22.423086514 -0700
 Birth: -
1 Like