New Option on Recording - Regenerate Streaming Index

Channels DVR Server
#4

Is there an API endpoint we can use to initiate the generation of the index?

#5

Yes, PUT /dvr/files/XXX/m3u8

1 Like
#6

Awesome! Thanks!

For those interested in (re-)generating the M3U8s for all recordings, here's my oneliner:

for _f in /media/channels/Logs/comskip/*
do
  curl -sSXPUT http://127.0.0.1:8089/dvr/files/$(basename ${_f})/m3u8
done

(Of course, this will only do it for recordings that have had their commercials indexed; on my system that just so happens to be all of them. Also, don't forget to substitute the root of your recordings directory, as yours may not be /media/channels/ like mine is.)

#7

If you do decide to run over everything, keep an eye out in the Log for any errors like this:

2019/07/11 20:10:50 [M3U8] m3u8 generation for /dvr/TV/The Rifleman/The Rifleman S01E08 1958-11-18 The Safe Guard 2019-07-11-1500.mpg failed with MPEG-TS discontinuity: PTS jump was too far: 204963823039951.750000

and report them to us. Once the new indexer is considered stable, we plan to automatically run over everyone's recordings in the background so you don't have to run any scripts.

#8

I just tried this on an older build and it worked perfectly. Great addition for me since I only watch TV on my Windows 10 computer using the Web UI.

Thank you.

1 Like
#9

If you have jq installed you can use this trick:

for id in $(curl -qs http://localhost:8089/dvr/files | jq -r '.[].ID'); do
  if [ "true" != "$(curl -sq http://localhost:8089/dvr/files/$id/mediainfo.json | jq -r '.m3u8_up_to_date')" ]; then
    echo "Regenerating $id"
    curl -sSXPUT "http://localhost:8089/dvr/files/$id/m3u8"
    echo
  fi
done

It'll grab all of the files on your DVR and only regenerate the ones that need to be.

3 Likes
Is there any way to force re-index video using command line?
#10

Much better! (I'm not at home, so I didn't really have time to think something up better. And I indeed have jq.)

1 Like
#11

This new remuxer is great! it fixed the crappy stream i was getting from SNY. Hopefully this new transcoder that is due to come will fix my hardware transcoding issue. great work!

#12

I don't get those errors, but I am seeing a lot of these types of errors:

2019/07/15 11:18:04 [M3U8] m3u8 generation for /Volumes/ChannelsDVR/recordings/TV/Today/Today 2019-07-15 2019-07-15-0700.mpg failed with No Keyframes were found in the stream
2019/07/15 11:19:19 [M3U8] m3u8 generation for /Volumes/ChannelsDVR/recordings/TV/Today/Today 2019-07-15 2019-07-15-0700.mpg failed with No Keyframes were found in the stream
2019/07/15 11:19:19 [M3U8] Regenerating m3u8 for /Volumes/ChannelsDVR/recordings/TV/Jackie Robinson/Jackie Robinson S01E02 2016-04-12 Part Two 2019-07-14-2300.mpg
2019/07/15 11:20:37 [M3U8] m3u8 generation for /Volumes/ChannelsDVR/recordings/TV/Jackie Robinson/Jackie Robinson S01E02 2016-04-12 Part Two 2019-07-14-2300.mpg failed with No Keyframes were found in the stream

Server Version: 2019.07.15.0203

#13

Huh weird. Do those recording play back normally? Can you upload one of them.

#14

I believe they playback normally. Since they are pretty large where would you suggest I upload them?

Here is the output of the regenerate command:

Regenerating 5137
{"ID":"5137","JobID":"1563152400-ch8.1","RuleID":"","GroupID":"11114803","Path":"TV/Grantchester on Masterpiece/Grantchester on Masterpiece S04E00 2019-07-14 2019-07-14-2100.mpg","Checksum":"","CreatedAt":1563152400,"Watched":false,"Deleted":false,"PlaybackTime":0,"Duration":7200.832456,"Commercials":[0,5.81,38.91,146.61,6718.81,6749.61,6795.52,7199.83],"Processed":true,"Airing":{"Source":"tms","Channel":"8.1","OriginalDate":"2019-07-14","Time":1563152400,"Duration":7200,"Title":"Grantchester on Masterpiece","Summary":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge.","Image":"http://fanc.tmsimg.com/assets/p11114803_b_h6_ap.jpg","Categories":["Episode","Series"],"Genres":["Crime drama","Mystery","Entertainment"],"Tags":["Season Premiere","New","HD","DVS","CC"],"SeriesID":"11114803","ProgramID":"EP025199990022","TeamIDs":null,"SeasonNumber":4,"EpisodeNumber":0,"Raw":{"startTime":"2019-07-15T01:00Z","endTime":"2019-07-15T03:00Z","duration":120,"channels":["8.1"],"stationId":"59210","qualifiers":["CC","DVS","HD 1080i","HDTV","New","Season Premiere","Stereo"],"ratings":[{"body":"USA Parental Rating","code":"TV14"}],"program":{"tmsId":"EP025199990022","rootId":"16910724","seriesId":"11114803","subType":"Series","title":"Grantchester on Masterpiece","titleLang":"en","seasonNum":4,"releaseYear":2019,"releaseDate":"2019-01-11","origAirDate":"2019-01-11","entityType":"Episode","descriptionLang":"en","shortDescription":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge.","longDescription":"Sidney is swept up in the civil rights movement after a pastor and his daughter arrive in Cambridge, and a murder sees racial tensions spike; Sidney struggles to remember a murdered woman's final words.","topCast":["James Norton","Robson Green","Tessa Peake-Jones"],"directors":["Tim Fywell"],"genres":["Crime drama","Mystery","Entertainment"],"preferredImage":{"uri":"http://fanc.tmsimg.com/assets/p11114803_b_h6_ap.jpg","height":"540","width":"720","primary":"true","category":"Banner-L1","text":"yes","tier":"Series"}}}},"PlayedAt":0,"UpdatedAt":1563193168044,"HighestPTS":0,"CommercialsAligned":true,"CommercialsEdited":true,"CommercialsVerified":false}
#15

You can upload to Firefox Send, Google Drive, Microsoft OneDrive, Dropbox or whatever is most convenient.

#16

Here is the link to the file. file_link

Let me know after you have downloaded so I can remove. Thanks.

#17

Thanks. I am downloading now.

I poked at the beginning of the file that downloaded and it looks like an mp4 file. Are you processing your recordings somehow?

#18

Yes. I use ffmpeg to wrap the mpeg in an mp4 container to add commercial markers so plex and see the chapters and allow you to skip in plex.

#19

Using this script

#20

Gotcha, okay. This is normal then. The new remuxer streaming index will not work on mp4 and other re-processed files.

#21

Ok. I tried running it on a file before I did the conversion and I got one of the errors you wanted us to look out for.

2019/07/16 09:50:50 [M3U8] Regenerating m3u8 for /Volumes/ChannelsDVR/recordings/Movies/The Wedding Singer (1998) 2019-07-16-0000.mpg

2019/07/16 09:51:29 [M3U8] m3u8 generation for /Volumes/ChannelsDVR/recordings/Movies/The Wedding Singer (1998) 2019-07-16-0000.mpg failed with MPEG-TS discontinuity: PTS jump was too far: 3445.095967
#22

Running Channels DVR v2019.07.17.1713
I regenerated the Streaming Index for 15 recordings a couple hours ago.
Looking at the processes running on my NAS I see 15 ffprobe processes listed.
Thinking that was one for each run I did, I reran it for one of the recordings again and I now have 16 ffprobe processes.
Appears they didn't terminate after they're finished?

#23

Nice catch!

1 Like