Stuttering playback on Apple TV (Windows Server 2019)

Channels Apple TV Troubleshooting
#136

Now that we've fixed the issue that was causing your DVR to not send diagnostics properly, can you please send Diagnostics from the Apple TV the next time this happens?

Based on the logs last time, it appeared that the issue was due to the data not getting from the DVR to the client fast enough, which is generally an issue with the network or with the resources on the DVR server (CPU, Disk IO).

Once we have another diagnostic from the Apple TV it may give us more pointers (or it will just make us think it's a resource issue on the DVR).

#137

Diagnostics from Apple TV just uploaded.

As a reminder, the issue only presents on the Apple TV. I have played a recording on the ATV and the Shield simultaneously and the slow motion only happens on the ATV. Also all performance counters on the server indicate very light usage.

#138

@eric I uploaded 2 sets of logs from 2 different Apple TVs. Any more thoughts on identifying the root cause?

#139

Is playback stuttering or stalling completely?

The logs show a similar issue as others had reported, where video data basically runs out because no more is being received from the DVR server.

#140

It plays in slow motion and if I let it go long enough, eventually it will resume normal playback.

#141

Have you run a speed test between the Apple TV and DVR already?

#142

Yes.

Speedtest

Download
987.54 Mbit/s

Latency
3.03 ms

Jitter
0.44 ms

#143

@tmm1 Since we have multiple users reporting the issue, is there any testing in your lab that you can do using Windows Server 2019 and an Apple TV? Microsoft offers Windows Server 2019 available for testing purposes.

#144

On the client side we're not seeing much useful stuff in the logs beyond the cache exhaustion discussed earlier. We're trying to figure out what other diagnostics we can add which might help shed light into what's going on inside tvOS.

On the DVR side we have a lot of good diagnostics, but I need another one which is sent from the DVR while the Apple TV video player is still active and stalling during recording playback.

How often is this issue happening? In your last logs I saw you watched a full hour+ recording without problems, then starting the next recording it happened almost right away (buffered 95s and stopped receiving data, then 95s later the video player stalls because the buffer is empty).

Is it happening on recordings only or live tv too?

#145

Recordings only. The issue is very random.

I will submit DVR logs next time the issue happens.

#146

Logs from the DVR uploaded.

#147

Problem is happening as a I type this. Speed test results from DVR to the Apple TV client.

Speedtest

Download
968.41 Mbit/s

Latency
3.05 ms

Jitter
0.23 ms

1 Like
#149

Thanks. I was able to see the DVR is indeed streaming the file out, and shows its waiting on the OS to inform it when the network connection is ready to accept more data.

Have you also already tried this:

#150

I have made the autotune change and rebooted the computer. I will report back.

#151

Autotune change made no difference. Logs submitted.

#152

Would it help to see logs from my shield when the issue occurs on the Apple TV?

#153

I've still been experiencing the issues myself, but the frequency I'd been seeing it happen diminished significantly for a couple of months after my last post. Maybe once or twice a week.

However, within the past few weeks, it's come back with a vengeance, and it's only been happening when I start watching a show within the first couple of minutes.

Given that there's three of us running Server 2019 with the same issue, that got me thinking that maybe there's a TCP stack setting interfering with it.

I figure that most users of Channels on Windows are probably using Win 10, and they aren't reporting any issues, so I did a comparison of the TCP settings between Win 10 1909 and Server 2019 last night.

Here are the differences I noted:
Server 2019:
ECN Capability: enabled
Initial RTO: 3000
Max SYN Retransmissions: 2
Fast Open: disabled
MemoryPressureProtection: enabled
PacketCoalescingFilter: Disabled

Win 10:
ECN Capability: disabled
Initial RTO: 1000
Max SYN Retransmissions: 4
Fast Open: enabled
MemoryPressureProtection: disabled
PacketCoalescingFilter: Enabled

The one that stood out to me is the memory pressure protection. According to this article, MPP is a feature designed to protect against DDoS attacks. It can potentially misidentify an attack, and when it thinks it's under attack, it kills the existing TCP connections and drops incoming SYN requests.

I disabled it last night, and I've watched something on all three of my ATVs since then. No issues yet, so I'm hopeful this might be it.

@mhartman, give it a try by running: netsh int tcp set security mpp=disabled
If you haven't already, also set the autotuning level back to normal. That also made no difference for me.

2 Likes
#154

@embj Nice write up and thank you! I will make this change and update the thread.

1 Like
#155

Great find! We do have a lot of users on Windows 10 who are not experiencing issues. It's definitely plausible this setting would cause the behavior we're seeing!

#156

So far so good. Watched two shows with no issues. Fingers crossed.

1 Like