For giggles it might be worth doing a ping test from one of your hard wired devices to www.google.com just to make sure you’re not having intermittent packet loss or something. I typically do 1,000 pings to get a good sample size.
1 Like
On top of this, I would ping and traceroute to fanc.tmsimg.com.
It could be done upstream carrier bottleneck, as bandwidth usage has surged greatly these days.
2 Likes
Good idea. I’ll try this when I get home from work.
Thanks Ed, I’ll give this a shot once I’m home.
I just did a small test and let the guide load and did absolutely no scrolling for about 30 seconds. This gave the guide enough time to load any images associated with it. Well, lo and behold, all of the images that weren’t in view as I loaded the guide, only appeared AFTER I began scrolling. This isn’t a problem with my system or setup. Changes were made to how the images are handled. I’ve seen this technique before, and it was actually implemented on one of my websites before. It saves bandwidth and decreases page load times, but horrible experience for the user. Which is why I ended up telling the developers to remove that from my website. I forgot what the technique is called, gonna look into it. Now, whether Channels or Gracenote are the ones using this technique remains to be seen, but in almost certain that this isn’t an issue with my devices or network.
Edit: the technique is called “Lazy loading”. So basically when you visit a site, only the images that the user can actually see, and depending on the device on a responsive website, are loaded. The other images are only loaded AFTER the user scrolls down and they become viewable. I’m almost 100% sure that this is what’s going on here because when this technique was used on one of my websites it was the exact same behavior. Although it saves bandwidth and speeds certain pages, it’s an all around bad experience for the user and it didn’t sit well with my site’s visitors. Almost immediately I started receiving emails that my site was “buggy” and wasn’t loading properly. Some people have fast devices and scroll quickly, and that’s when this technique starts rearing it’s ugly head. Being that I’m one of these fast scrollers who usually scrolls half way down the guide for my favorite channels, it’s apparent to me that things have changed under the hood as far as how images are being handled.
Is there a way to force the guide to load the thumbnails instead of this “Lazy Loading” thing that’s going on with the guide now? I loaded the guide and let it sit for about 30 seconds and only the images in view loaded. The rest of the images loaded slowly as I scrolled down. As I initially suspected, the images are being lazy loaded, and this is pretty new, because I’ve never experience this problem before with the Channels’ Guide.
No changes have been made to how images are loaded. As I mentioned earlier, the loading behavior is entirely controlled by tvOS and always has been.
What version of tvOS is installed, and what version number is shown in the Channels app?
Our last release was 3.2.25 in January (https://getchannels.com/releases/). There was a caching setting we changed in that release to try to make images show up more quickly, but it's possible that is not working as intended. Have you been noticing this problem since January 15?
I honestly couldn’t tell you a date. I know this, I watch the A&E Channel religiously when I do watch TV. So, whenever I opened Channels, I would immediately scroll all the way to the bottom of the guide where the Channel listing is. NEVER, until recently, did I experience this issue where the thumbnails weren’t already loaded when I scrolled down. Much less have the guide stutter or stall as the images load. Could have been since January and maybe I didn’t notice. I don’t get a chance to watch TV every single day. As far as a the software, the Apple TV and Channels App are on the latest versions. Maybe you guys didn’t make any changes, but maybe the host of where the images are pulled from did. I just hope this isn’t something I have to learn how to deal with either way. What attracted me to Channels was how fast and responsive the guide and channel loading was compared to others. The loading of Channels is instantaneous, and I’m happy with that, it’s just the guide’s thumbnails that are becoming a nuisance. But, the fact that I let the guide sit for 30 seconds before scrolling and only the viewable images appeared made it clear to me that the lazy loading technique is being used. One of my coworkers, who’s a network guru has been monitoring my network for almost 4hrs now. Has found ZERO issues.
Hello Eric. Over 230ms on each and every ping. But this is the thing Eric...although I load the guide and let it sit, the images are only being loaded as I scroll. This wasn’t the default behavior of the guide before.
Wow. That's really slow. We expect it to be much less than 50ms (for example, for me I am seeing 3ms response times). Can you paste a traceroute here?
What response times do you get when you ping cdn.channelsdvr.net?
Very interesting. Please provide the results from traceroute fanc.tmsimg.com as well.
I updated my previous answer so you can see the steady 230 ms ping results. Running the trace route now from Terminal, From what I’m already inspecting, seems like all the 230+ responses are coming from Amazon.
Traceroute Results:
majesty@server ~ % traceroute fanc.tmsimg.com
traceroute: Warning: fanc.tmsimg.com has multiple addresses; using 99.86.181.32
traceroute to d1k0gkbv5kl6h3.cloudfront.net (99.86.181.32), 64 hops max, 52 byte packets
1 192.168.1.1 (192.168.1.1) 0.672 ms 0.264 ms 0.295 ms
2 * * *
3 b3335.nwrknj-lcr-22.verizon-gni.net (130.81.27.48) 6.875 ms
b3335.nwrknj-lcr-21.verizon-gni.net (130.81.27.46) 8.051 ms
b3335.nwrknj-lcr-22.verizon-gni.net (130.81.27.48) 9.943 ms
4 * * *
5 * * *
6 0.ae1.br2.nyc4.alter.net (140.222.229.91) 5.127 ms 5.679 ms
0.ae2.br2.nyc4.alter.net (140.222.229.93) 6.380 ms
7 verizon.com.customer.alter.net (152.179.72.42) 5.937 ms 6.414 ms 5.597 ms
8 * * *
9 ae-5.r22.sttlwa01.us.bb.gin.ntt.net (129.250.6.177) 65.817 ms * 66.189 ms
10 ae-0.r23.sttlwa01.us.bb.gin.ntt.net (129.250.6.30) 65.473 ms 65.700 ms 68.452 ms
11 ae-16.r24.osakjp02.jp.bb.gin.ntt.net (129.250.3.61) 162.423 ms 162.813 ms 162.444 ms
12 ae-0.r25.osakjp02.jp.bb.gin.ntt.net (129.250.2.151) 171.367 ms 165.327 ms 165.148 ms
13 ae-0.r20.sngpsi07.sg.bb.gin.ntt.net (129.250.2.66) 223.176 ms 223.998 ms 229.568 ms
14 ae-1.r00.sngpsi07.sg.bb.gin.ntt.net (129.250.3.82) 229.583 ms
ae-1.r01.sngpsi07.sg.bb.gin.ntt.net (129.250.3.100) 231.167 ms
ae-1.r00.sngpsi07.sg.bb.gin.ntt.net (129.250.3.82) 236.143 ms
15 ae-1.a00.sngpsi07.sg.bb.gin.ntt.net (129.250.2.92) 237.284 ms 237.796 ms 230.932 ms
16 ae-1.amazon.sngpsi07.sg.bb.gin.ntt.net (116.51.17.130) 230.669 ms 228.075 ms
ae-0.amazon.sngpsi07.sg.bb.gin.ntt.net (116.51.17.46) 232.215 ms
17 52.93.8.122 (52.93.8.122) 226.444 ms
52.93.8.56 (52.93.8.56) 234.885 ms
52.93.8.100 (52.93.8.100) 229.354 ms
18 52.93.8.157 (52.93.8.157) 233.733 ms
52.93.8.21 (52.93.8.21) 239.422 ms
52.93.8.107 (52.93.8.107) 229.164 ms
19 * * *
20 150.222.240.82 (150.222.240.82) 237.376 ms 231.124 ms 230.723 ms
21 52.93.63.130 (52.93.63.130) 231.859 ms
52.93.63.146 (52.93.63.146) 233.539 ms
52.93.63.130 (52.93.63.130) 231.751 ms
22 150.222.78.19 (150.222.78.19) 241.755 ms
150.222.78.23 (150.222.78.23) 240.470 ms
150.222.78.13 (150.222.78.13) 234.466 ms
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 server-99-86-181-32.kul50.r.cloudfront.net (99.86.181.32) 233.628 ms 238.776 ms 239.180 ms
Are you using a VPN or custom DNS server?
I’m currently running a PiHole Server that uses DoH with CloudFlared.
If you turn that off the images will be fast again. It is making it so that AWS CloudFront can’t detect the closest location to serve the images from.
Ok Eric. I’ll try that then report the results of the new trace route. But, I’ve had this custom DNS solution for many many months, never experienced any issues. I’ll disable blocking from that device to see if the results improve. I can’t see myself completely getting rid of that server as it’s Golden to me. Blocks ads, trackers, etc etc.
To restate what was going on: The custom DNS server being used confused AWS CloudFront (the Content Delivery Network for these images) and caused the images to be served from Singapore instead of a closer location.
The solution is to switch to DNS servers that allow AWS CloudFront to properly identify what geography your DVR is in.
2 Likes
What upstream DNS server does your pihole setup use?