Player buffering issues

[syadnom]

Hi all, this didn't quite fit into DVR or individual clients. I'm seeing it across iOS and android clients, and on Windows, Linux, and Mac M1 DVR servers.

Playing any sort of live content, from HDHR or TVE sources, the players don't buffer enough. I bring up stats and my buffer is <1 second and the videos stutter. hitting pause and unpause is enough to resolve this, getting a bit more in the buffer. or just seeking back 'once'. It's super annoying.

All 3 of my servers are easily powerful enough for this, I don't seem to have any performance issues on DVR, network, etc, it just seems that the players don't want to add 1-2 seconds of buffer in.

I actually posted this some issue like 2 years ago and haven't really been using channels since then because of a couple of issues. This one is a big one, the channel will stutter endlessly until you pause it or seek backwards once. I have older family members that just can't tollerate this and can't really 'think' tech enough to remember to hit back so I get endless complaints.

I see no 'extra buffer' option in the client configs to address this.

Any ideas?

[HTV]

I am NO expert... but If I were having a similar issue, I would consider checking this item in the "experimental" box at the bottom of the "Settings" tab of the configuration page. I realize this isn't for a tuner and may be way off the mark, but it may be a buffer issue?

Old Tuner Sharing Buffer "Use deprecated Tuner Sharing buffer. Intended to be used for testing if there are problems with Tuner Sharing (resets on DVR restart)."

Good Luck

[syadnom]

Just tried it. Turned off all other streams, enabled that. Opened up channels on ipad and picked HDHR channel and it's immediately at 0.4 seconds cached and stutters after a few seconds. Tried a TVE source, same thing.

[chDVRuser]

You might want to try enabling the client debug setting Always use HLS for streaming

HLS option

So, I have been testing the HLS option from the Debug settings. It seems like a good solution for streaming devices that don't have a lot of free memory for the pause buffer. The HLS option provides a fixed 60 minute pause buffer stored on the server for any streaming device. In my testing on a Shield Pro (ethernet connected), a Chromecast Google TV, a Sony Google TV, and a Firestick 4k TV Max, I found it takes about an extra 2 seconds to start a live stream and it had a couple of short pauses …

[syadnom]

For the life of me I can't find any debug settings on ipad or iphone clients. Where are these debug settings?

[syadnom]

ok, guess I needed beta player. Got it, enabled always use HLS.

I don't thinkg this has helped. I'm noticing that pre and post setting the channel I'm testing against is alraedy saying "HLS Remote" from the HDHR.

Really just need a setting that is 'Players pre-buffer seconds or MB'. That would solve things IMO. And if the buffer does go to zero, fill back up to the pre-buffer.

Also

[syadnom]

ok, just to confirm after a few minutes testing.

Test WITH and WITHOUT force HLS. No changes.

2 difference channels instances. Both have all data and caches etc on SSD.

ubuntu 20.04 i5-6500T
and
mac mini m1

testing iphone, ipad, and macos m1 (ipad app) clients from testflight

forcing transcoding via setting streaming max rates

tried with and without HEVC transcoding from experimental

See attached image, buffer runs to 0.0 sec and it stutters. this repents forever, until I seek back once or just pause for like 1-2 seconds and let the buffer fill up.

[eric]

Could you take a little video of the issue and upload it here? Also, if you could submit diagnostics from the app you capture after you've done this, that will help as well.

Dropbox - Submit files

The issue you're describing doesn't make a lot of sense and we'd like to figure out what's going on.

[syadnom]

Eric, I just pushed diagnostics from the app (beta app) and uploaded a video showing the buffer emptying out and the video stuttering and pausing.

These are mac mini m1 as the channels server and the player to get any networking out of the mix. No processes are stressing the CPU at all during the test, and this is from an HDHR source.

[eric]

Do you have these problems if you use Streaming Quality set to Original? Can you try that and take a video and diagnostics?

Also, the audio didn't come through on your previous recording. It would be helpful to hear what you're hearing.

[syadnom]

the audio perfectly matches what the video does. the little stutter is reflected in the audio and the pause is as well. I just mac screen recorder so no audio on that. If you need it I can do a recording with audio, but it's exactly as described above.

I don't have this issue when the transcoder isn't in use. ie, 'original' plays without a problem. ANY transcoder use at all and it happens.

It 'feels' very much like the time it takes to download/load the HLS chunk is just long enough that I'm beating the next chunk from writing. nipping at the heals of the transcoder.

And again, if I just hit pause for the shortest amount of time it doesn't happen.

[eric]

What sort of drive are you using? I’m wondering if you’re running into IO issues.

We’ve had many good experiences with the M1 Mini and haven’t had any of the issues you’re describing, so it sounds like something unusual is happening.

[syadnom]

I'm using 2 different machines. The Mac Mini M1 is using it's internal drive. The Dell i5-6500T unit is using a samsung 840 ssd. wildly different platforms and getting identical behavior. I've also tried this on wired gigabit LAN as well as on WiFi6 routers (Eero 6E).

[eric]

Can you set the Streaming Quality back to Original and set the Debug setting Always use HLS Streaming and try it again and submit diagnostics if you run into the problem?

[syadnom]

I've already tested 'original'. that doesn't do transcoding and it doesn't do the studdering at all. With or without the 'Always use HLS'. that mode works perfectly.

I just need to get the streams down to ~4Mbps via transcoder. TVE are coming in around 8Mbps pretty often, and HHDR mpeg2ts are anywhere from 4 to 18Mbps.

I just feel like the behavior suggests that my player is riding the front edge of the encoding. It's wanting the newest chunk fromt he transcoder as it's being written to disk. Immediately that suggests IO issues, but my Mac M1 with an SSD that shows no contention really shouldn't have this issue on a single transcode, nor should the older box with the samsung ssd either. So here I am

[eric]

It's confusing that you wouldn't see these IO issues with Always use HLS but would with transcoding. They both would be using similar amounts of disk IO.

I did notice that your load average on the DVR was at 9 — a little high for just doing hardware transcoding.

To rule out GPU contention, can you run these tests with clients not on the same physical machine as the DVR?

I agree that it seems like your player is interacting strangely with the encoder. The thing that puzzles me is that I run an M1 Mac Mini as my main DVR and have not experienced the issues you're describing when playing transcoded video. So I'd like to understand what it is that is different about your setup compared to mine to see why you're experiencing this issue that I am not.

[Michael_Birk]

The only time I had a problem with remote channels doing excessive buffering, I had to lower the Streaming limit to 1mbit/s. My family only had a DSL internet connection that would barely exceed 1mbit/s. Channels still looked and worked quite well on the 40in TV. We still connect periodically at this location to watch Yellowstone. Not sure if you have run a Channels Android Client Speed Test using your remote connection, but might be worth knowing.

[syadnom]

haven't been able to hack on this due to some other commitments.

@Eric, I'm not seeing load of 9 on the machine when testing. 2-3 on the high side. I can't actually convince the mac to do a load of 9 for more that the short duration (in top) unless I spin up a couple VMs in parralells. Also note that this does happen exactly the same when connecting to the i5 based box. I also have an i5-2400 windows box with an nvidia GPU that also does the same thing.

@Michael_Birk I actually don't have this issue remotely, the 'extra time' it takes to buffer for a remote connection seems to be plenty to solve this. When I'm on the local network it's like my players are filling their buffer so quickly they get ahead of it.

[slampman]

@syadnom

I actually don't have this issue remotely, the 'extra time' it takes to buffer for a remote connection seems to be plenty to solve this. When I'm on the local network it's like my players are filling their buffer so quickly they get ahead of it.

I gotta ask.... If this problem only happens locally then why don't you just disable transcoding and set to original quality?? Your local network should have more than enough throughput to handle the stream. If it doesn't then you have bigger problems.

[racameron]

Is this only at the beginning of a stream? If so, this sounds more like a disk I/O issue. Specifically your disk sleeping and taking longer than expected for it to spin up. Have you checked your drives' power settings or what SMART is reporting?