Multiple TVE Streams

Can you paste some of the full [MTS] line entries from the dvr log?

My t330 is still running the memtest and as I said it is headless so I can not interrupt it until it is done.

However, I installed your dvr software on an i5 kaby lake zotac running ubuntu mate and it works perfectly. Unfortunately, I can not run this system 24/7. My guess is there is a interaction between your code and the latest build of windows 10

The skipped=N messages mean the video stream is getting corrupted by the time it reaches the DVR. In our experience, this is always either due to a bad stick of RAM or a failing disk.

All drives check out fine in seatools. System passed memtest. Problem occurs on multiple machines all running the latest build of windows 10. Software works on ubuntu mate I know what you are saying, but my money is on a interaction with windows 10 and your code. Please look into it.

I just went over the log, there are no mts entries

MTS entries were printed after each TVE recording finished. However, it looks like the current prerelease removed those log entries.

Looking through my logs v2020.04.19.0128 has the [MTS] entries, but v2020.04.20.0048 does not.

Edit: Looks like the [MTS] logs were removed from TVE streams (but not sure when), but they remain for Locast streams. Here's an example:

2020/04/20 19:31:00.845885 [MTS] Statistics for "TV/Jeopardy!/Jeopardy! S36E161 2020-04-20-1900.mpg": skipped=0 unhandled_packets=0 discontinuity_detected=0 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=1872.833333

Edit 2: MTS is an abbreviation to indicate the log line is about a completed MPEG-TS stream. It's the analogue to the SNR statistics given for HDHomeRun-delivered recordings.

@techpro2004 can you try recording from Locast?

I am recording from locast now. I will try to get those mts lines for you once it is done. I am also updating a spare system to the release preview ring. I will let you know how that goes as well.

there are no mts lines with locast. Keep in mind I have updated to 2020.04.20.0048. I have submitted logs again Thanks

I downgraded to stable but still no mts entries

Also 2004 makes no difference.

Which log are you checking?

I am checking all the logs in record_dir/Logs

That's not the correct log. You need to check the main DVR log via http://x.x.x.x:8089/admin/log

found it see below.

2020/04/21 12:05:00.686437 [MTS] Statistics for "TV\The Real\The Real S06E139 The Real From Home Blair Underwood Hotline Bling 2020-04-21-1125.mpg": skipped=340 unhandled_packets=0 discontinuity_detected=2 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3681.956911

2020/04/21 12:05:00.363301 [MTS] Statistics for "TV\The View\The View S23E150 2020-04-21-1125.mpg": skipped=39807 unhandled_packets=0 discontinuity_detected=99 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=1026.566667

2020/04/21 12:05:00.530853 [MTS] Statistics for "TV\The Price Is Right\The Price Is Right S48E137 Kids Week 2020-04-21-1125.mpg": skipped=1960 unhandled_packets=0 discontinuity_detected=0 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3088.199556

2020/04/21 12:05:00.686437 [MTS] Statistics for "TV\The Real\The Real S06E139 The Real From Home Blair Underwood Hotline Bling 2020-04-21-1125.mpg": skipped=340 unhandled_packets=0 discontinuity_detected=2 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3681.956911
2

2020/04/21 12:05:00.719349 [MTS] Statistics for "TV\Maury\Maury S22E81 2020-02-21 Your Son Is Not Biracial so There Is No Way Im the Dad 2020-04-21-1126.mpg": skipped=1904 unhandled_packets=0 discontinuity_detected=4 transport_errors=0 invalid_pts=0 invalid_dts=0 saw_pcr=true saw_pmt=true highest_pts=3440.926144

So what do these mts entries show?

Its showing that the stream is getting corrupted by the time it reaches the DVR. If your disk and ram are fine, then it might be a network issue. I would check the ethernet cables. Was the linux DVR plugged into the same switch?

I'm pretty sure this is not a software issue. If the DVR was broken on Windows 10 we would be hearing a lot more reports. I've only seen this skipped=N issue maybe 3-4 times since we launched TVE, and it has always been due to faulty hardware somewhere in the stack.

These are all stats about the MPEG transport stream:

  • skipped=1904 – Bytes that were skipped by the MPEG-TS parser (malformed or corrupted TS packets)
  • unhandled_packets=0 – Number of packets that could not be recovered when a discontinuity is detected and are passed through un-fixed. If this is above 0, there were issues hat could not seamlessly be recovered from
  • discontinuity_detected=4 – Number of times that MPEG-TS streams were spliced together and Channels was able to properly recover from it and rewrite the stream in a consistent way
  • transport_errors=0 – Errors in the stream itself
  • invalid_pts=0 – Presentation timestamps that don't fit, such as inserted commercials using a different clock base
  • invalid_dts=0 – Decode timestamps that don't fit, just as above; sometimes packets are encoded in a different order than they're meant to display, hence two different types of timestamps
  • saw_pcr=true – Whether the stream had a program clock rate embedded, the basis used for the timestamps (PTS/DTS)
  • saw_pmt=true – Whether the stream had an embedded program map table, indicating a stream's various audio/video tracks
  • highest_pts=3440.926144 – Largest timestamp seen in the recording

Of course, these definitions are all my own inferences from prior experiences and logical conclusions; corrections are welcomed.

Edit: Definitions were updated with corrections from @tmm1 and @eric.

Skipped is the number of bytes that could not be parsed. Mpeg-TS packets are generally 188 bytes and start with the byte 0x47. If the stream is corrupted, the 0x47 gets overwritten with something else and the packets cannot be parsed correctly.

The TVE streams come over the network, are saved directly to the disk, and then parsed to feed into the rest of the DVR recording engine. Somewhere in the pipeline the packets are being corrupted. In the past I've seen this due to bad RAM (i.e. corruption between network and disk), or bad DISK (i.e. corruption between when the data is written and when it is read). I think there was also one isolated case where the network itself was corrupting data before it made it to the DVR machine.

Bytes, or packets?

So skipped refers to corrupted bytes/packets, not dropped. Good to know.

IIRC skipped is reported in bytes. These bytes are skipped by the mpegts parser as it looks for the next 'G' that represents a valid packet.