Tru TV TVE Error

[skipt]

From time to time Tru TV won't stream for me via an custom channels TVE m3u file. I found the following message in the Channels log file: (obliterated id and hmac in case there is some security risk)

“[ERR] Could not start stream for ANY ch6037 TRU: TVE: Could not fetch playlist from turnerlive.akamaized.net: GET: https://turnerlive.akamaized.net/hls/live/572101/true/noslate/tv/master.m3u8?hdnts=exp=1709431876~acl=/hls/live/*~id=xxxxxxx-xxxxx~hmac=xxxxxxxxxxxxx: 403 Forbidden

Other channels from the same custom channels file seem to be OK. I checked my host outgoing port availability and there seems to be plenty of ports available. When I restart the Channels server host laptop Tru TV works fine again. Next time I plan to only restart the CDVR server but not the laptop, will that give me any more insight to where the problem is?

[Morris_Altman]

That sounds like the cached credentials are expiring and your server needs to log in again. You could try editing the settings and change the "XMLTV Guide Data" refresh rate to make it more frequent. If your server has the power, make it once an hour. Otherwise experiment to see what works.

Good luck,
Morris

[skipt]

Thanks Morris. My refresh was set to daily, I reset it to 3 hours. My server is a well-loaded and dedicated i7, so I will go to 1 hour if things don't improve.

[skipt]

The problem and troubleshooting continues. When you mention "credentials expiring" I assume you mean at the Tru TV end, because all of my other TVE channels are working. When you say "the server needs to log in again" are you referring to the Channels server? Is there a way for me to refresh my credentials with Tru TV alone? Or, to refresh my credentials do I need to restart the Channels Server, or maybe the the host laptop? The following may also be related: during troubleshooting and testing, I got to a point where none of my channels were working, and got these messages:

[ERR] Could not start stream for ANY ch6082 FXX: TVE: Post "https://sp.auth.adobe.com/adobe-services/shortAuthorize": dial tcp 18.210.71.255:443: bind: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

[ERR] Could not start stream for ANY ch6140 ESPN1: TVE: Get "https://broadband.espn.com/espn3/auth/watchespn/startSession?SWID=C9D4960E-62BA-432B-8EC8-4DAC1723D614&channel=espn1&partner=espn&platform=ios&plt=ios_tablet&token=PHNpZ25hdHVyZUluZm8%2BTmZBdFJ3LzFhMldURDVXMU9FcXdtUUFidXNCaTlxdmpqWmdBQ1c4RlNOVzVqRlBYU20xZkJYTGJhV2hrL3ZwSWttSytvaVVyL1BhYllWZTFObmgyUVI0Zm01dUJtVkFrRHhKTU1ZQTdWRGlUWDdzTWZ2SFZORnoxL2hQOEo4L2xQSXYwR21UR2RRc2JJeFZRc3psaTVHQjdIMHJJYnIzWFUweHZFL040a3g4PTxzaWduYXR1cmVJbmZvPjxhdXRoVG9rZW4%2BPHNlc3Npb25HVUlEPmY1YzliODUyZDNjNzI5NDcyZmUxY2ExNDg2ZGQ4NDI2PC9zZXNzaW9uR1VJRD48cmVxdWVzdG9ySUQ%2BRVNQTjwvcmVxdWVzdG9ySUQ%2BPHJlc291cmNlSUQ%2BZXNwbjE8L3Jlc291cmNlSUQ%2BPHR0bD40MjAwMDA8L3R0bD48aXNzdWVUaW1lPjIwMjQtMDMtMDQgMDY6MDc6MTUgLTA4MDA8L2lzc3VlVGltZT48bXZwZElkPmF1dGhfYXRsYW50aWNiYl9uZXQ8L212cGRJZD48cHJveHlNdnBkSWQ%2BUHJveHlNVlBEX1N5bmFjb3I8L3Byb3h5TXZwZElkPjwvYXV0aFRva2VuPg%3D%3D&tokenType=ADOBEPASS&v=2.0.0": dial tcp 34.195.130.230:443: bind: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.

I note the auth.adobe portion of the error messages, and I believe that is part of the credentials process, right? So with this many things going wrong I restarted my Win 10 i7 laptop, which of course restarted the Channels server, and everything, including Tru TV, was working again.

BTW, in parallel with all of this I recently installed Docker Desktop w WSL2 and have been trying to get Olive-Tin-For Channels working on the same laptop (no luck yet) and I'm wondering if this is part of the problem. I am considering uninstalling everything associated with Docker/Olive-Tin and see if that makes a difference.

[Morris_Altman]

I have to make assumptions based on experience with channels DVR and as a developer/administrator of Windows systems. Channels is sending the login request and not getting an answer. Channels times out the login and tries again. For some reason the session is staying open for the original login, it probably should time out. As it's not the windows connection queue is filling.

Installing Docker Desktop w WSL2 and have been trying to get Olive-Tin-For Channels is complicating the picture. Did you have the problem before the install? If yes, I'd try install the latest drivers for you NIC. What NIC as some have a better reputation than others?

Do you experience any other network issues?

What do you get doing a speed test to the server using:
http://192.168.2.94:8089/speedtest
Replace 192.168.2.94 with your channels server's IP

Good luck,

Morris

[skipt]

I can't remember exactly the timing of the TVE problem and the Docker install, and I'm not really having any other network problems. The laptop has been in use reliably for a long time so I'm going to postpone any NIC investigations for now. I'm going to remove all things Docker, as a fresh start is probably in order there anyway. Thanks for your advice and please stay tuned if you can bear it.

[skipt]

So I removed Docker early today and I noticed that it didn't fully remove--see pic 1 which is a screenshot from TCPviewer showing some ports related to Docker still open.

After some googling I found that Docker uninstall leaves some garbage in the ~/etc/hosts file which I also removed. Removing this remaining garbage from etc/hosts got rid of any signs of Docker in my connections. See pics 2 and 3.

I monitored the streams all day but around 8 PM TNT failed to play, same symptoms as previous tests. I left TRU TV playing from 8 PM to 9:30 PM and it did not fail. (During previous evenings it always failed around 9 PM.

I've looked at the CDVR logs and have lots of them but the failures always seem to leave the same messages (pic4) in the logfile.

Looking for more ideas as to why individual channels stop working from time to time. It always seems to be around 8-9 PM and restarting the CDVR server laptop always seems to restore the streams. I've completely removed Docker and I have expanded my dynamic IPV4/6 TCP/UDP ports to large numbers. Should I send a more complete log file along with a link to this thread to the CDVR support team?

[chDVRuser]

You say TNT failed and TRU played without failure, but your screen pic of the log shows TRU errored out at 8:09 PM?

@skipt

I monitored the streams all day but around 8 PM TNT failed to play, same symptoms as previous tests. I left TRU TV playing from 8 PM to 9:30 PM and it did not fail. (During previous evenings it always failed around 9 PM.

I've looked at the CDVR logs and have lots of them but the failures always seem to leave the same messages (pic4) in the logfile.

[chDVRuser]

I just tried and could stream all my WarnerMedia TVE CHANNELS except
for 6036 TNTP and 6037 TRU which get 403 Forbidden errors.
6030 CNN WORKS
6031 HLN WORKS
6032 CNNI WORKS
6033 TBS WORKS
6034 TBSP WORKS
6035 TNT WORKS
6036 TNTP FAILS 403 Forbidden
6037 TRU FAILS 403 Forbidden
6038 TRUP WORKS
6041 TOON WORKS

Wonder if this (from 25 days ago) is still true.

@tmm1

They are changing their TVE streams and these old ones are not working reliably anymore. The new ones are DRM.

[skipt]

Sorry about that. The problem comes and goes on both channels, but I didn't want to post a gigantic log. From recent posts it sounds like a new DRM problem creeping in. At least I'm not crazy or on my own here.

[tarianjed]

I just had this issue with TOON TVE channel that I get through youtube tv and I just needed to rescan that channel.

[chDVRuser]

Rescan doesn't work for me. All that does is authorize the channel (which is already authorized).
They're all working now except for 6038 TRUP (worked last night) which gets a 403 Forbidden error.
And now an hour later, 6038 TRUP works, but 6033 TBS gets a 403 Forbidden error.

[skipt]

Curious: if (for the problems I'm seeing) they are switching over to DRM, can someone explain why sometimes it works for me and sometimes it doesn't? My guess is that there are multiple servers involved and they all haven't transitioned yet. Can I assume it has nothing to do with my setup, including my Channels server?

Aside: since in my case it is TRU and TNT, I'm wondering if they are trying to quickly get DRM in place before March Madness.

[chDVRuser]

@skipt

can someone explain why sometimes it works for me and sometimes it doesn't?

Probably the same reason it sometimes works for me and sometimes doesn't.
Right now these channels give me a 403 error. 6033-TBS, 6034-TBSP, 6037-TRU and 6038-TRUP. The rest of the Warner channels work. Later tonight that will probably change again.

Don't know why. The same issue happened with both TOON channels and the west feed was removed. I had the same issues with local FOX working and not working. Could never rely on it.

I basically just quit relying on unreliable (not all) TVE channels since I have them on my HDHR Prime through Xfinity.

[chDVRuser]

@chDVRuser

Later tonight that will probably change again.

Like right now 6038-TRUP is working.

 6038-TRUP FAILS
2024/03/07 17:06:14.379252 [ERR] Failed to start stream for ch6038: TVE: Could not fetch playlist from turnerlive.akamaized.net: GET: https://turnerlive.akamaized.net/hls/live/572105/truw/noslate/tv/master.m3u8?hdnts=exp=1709860267~acl=/hls/live/*~id=ID~hmac=HMAC: 403 Forbidden
2024/03/07 17:06:14.380411 [HLS] Couldn't generate stream playlist for ch6038-dANY-ip192.168.1.8: TVE: Could not fetch playlist from turnerlive.akamaized.net: GET: https://turnerlive.akamaized.net/hls/live/572105/truw/noslate/tv/master.m3u8?hdnts=exp=1709860267~acl=/hls/live/*~id=ID~hmac=HMAC: 403 Forbidden
2024/03/07 17:06:14.380507 [HLS] Stopping transcoder session ch6038-dANY-ip192.168.1.8

 6038-TRUP WORKS 44 minutes later
2024/03/07 17:50:10.045045 [TVE] stream timestamps: trup: start_at=2024-03-07T17:39:56-08:00 end_at=2024-03-07T17:49:53-08:00 live_delay=10.801040953s
2024/03/07 17:50:10.045183 [TNR] Opened connection to TVE-Comcast_SSO for ch6038 TRUP
2024/03/07 17:50:10.046032 [HLS] Starting live stream for channel 6038 from 192.168.1.8 (bitrate=6460)
2024/03/07 17:50:10.903620 [HLS] Probed live stream in 857.276834ms: h264 1280x720 progressive 5675125bps
2024/03/07 17:51:06.591330 [HLS] Stopping transcoder session ch6038-dANY-ip192.168.1.8 (out=1m6.0692s finished=false first_seq=1 last_seq=32)
2024/03/07 17:51:06.592345 [TNR] Closed connection to TVE-Comcast_SSO for ch6038 TRUP
2024/03/07 17:51:06.595797 [SNR] Buffer statistics for ch6038 TRUP: buf=0% drop=0%
2024/03/07 17:51:06.595864 [SNR] Streaming statistics for ch6038 TRUP: timeouts=0 segment_timeouts=0 playlist_timeouts=0

 6038-TRUP Scheduled a recording for the next airing and it's working
2024/03/07 17:51:18.363101 [DVR] Waiting 6m41.636900246s until next job 1709863080-ch6038 Impractical Jokers
2024/03/07 17:58:00.002621 [DVR] Starting job 1709863080-ch6038 Impractical Jokers on ch=[6038]
2024/03/07 17:58:01.206857 [TVE] stream timestamps: trup: start_at=2024-03-07T17:47:41-08:00 end_at=2024-03-07T17:57:39-08:00 live_delay=16.196853714s
2024/03/07 17:58:01.207106 [TNR] Opened connection to TVE-Comcast_SSO for ch6038 TRUP
2024/03/07 17:58:01.207435 [DVR] Recording for job 1709863080-ch6038 from TVE-Comcast_SSO ch6038 into "TV/Impractical Jokers/Impractical Jokers S09E26 2022-08-04 Brooke Shields 2024-03-07-1758.mpg" for 33m59.997235086s