XMLTV EPG failing

Channels DVR Server
#1

Hi,

As far as I know this link is the correct format to obtain the guide from Channels.

http://:8089/devices/ANY/guide/xmltv?duration=604800

Unfortunately, using the current daily build on Channels, and testing with both Emby and Jellyfin it's producing an error in the log.

From Emby:

2021-01-31 16:04:19.845 Info HttpClient: GetTempFileResponse url: http://<IP ADDRESS>:8089/devices/ANY/guide/xmltv?duration=604800
2021-01-31 16:04:21.488 Info Server: http/1.1 POST http://<IP ADDRESS>:8096/emby/Sessions/Playing/Progress. UserAgent: Dalvik/2.1.0 (Linux; U; Android 9; SHIELD Android TV Build/PPR1.180610.011)
2021-01-31 16:04:21.489 Info Server: http/1.1 Response 204 to 10.5.1.195. Time: 2ms. http://<IP ADDRESS>:8096/emby/Sessions/Playing/Progress
2021-01-31 16:04:31.493 Info Server: http/1.1 POST http://<IP ADDRESS>:8096/emby/Sessions/Playing/Progress. UserAgent: Dalvik/2.1.0 (Linux; U; Android 9; SHIELD Android TV Build/PPR1.180610.011)
2021-01-31 16:04:31.495 Info Server: http/1.1 Response 204 to 10.5.1.195. Time: 2ms. http://<IP ADDRESS>:8096/emby/Sessions/Playing/Progress
2021-01-31 16:04:39.873 Error HttpClient: Connection to http://<IP ADDRESS>:8089/devices/ANY/guide/xmltv?duration=604800 timed out
2021-01-31 16:04:40.032 Error LiveTvManager: Error refreshing channels for service
	*** Error Report ***
	Version: 4.5.4.0
	Command line: /usr/lib/emby-server/EmbyServer.dll -programdata /config -ffdetect -ffmpeg /usr/bin/ffmpeg -ffprobe /usr/bin/ffprobe -restartexitcode 3
	Operating system: Linux version 4.19.107-Unraid (root@Develop) (gcc version 9.2.0 (GCC)) #1 SMP Thu Mar 5 13:55:57 PST 2020
	Framework: .NET Core 3.1.8
	OS/Process: x64/x64
	Runtime: usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.8/System.Private.CoreLib.dll
	Processor count: 6
	Data path: /config
	Application path: /usr/lib/emby-server
	MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: Connection to http://<IP ADDRESS>:8089/devices/ANY/guide/xmltv?duration=604800 timed out
	 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
	 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
	 ---> System.Net.Sockets.SocketException (125): Operation canceled
	   --- End of inner exception stack trace ---
	   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
	   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
	   at System.Net.Http.HttpConnection.FillAsync()
	   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
	   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
	   --- End of inner exception stack trace ---
	   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
	   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
	   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFileResponse(HttpRequestOptions options)
	   --- End of inner exception stack trace ---
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFileResponse(HttpRequestOptions options)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFile(HttpRequestOptions options)
	   at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetXml(ListingsProviderInfo info, String path, MediaProtocol protocol, CancellationToken cancellationToken)
	   at Emby.LiveTV.Listings.XmlTv.XmlTvProvider.GetChannels(ListingsProviderInfo info, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetEpgChannels(IListingsProvider provider, ListingsProviderInfo info, Boolean enableCache, CancellationToken cancellationToken)
	   at Emby.LiveTV.EmbyTV.GetChannelsAsync(Boolean enableCache, CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(ILiveTvService service, IProgress`1 progress, CancellationToken cancellationToken)
	   at Emby.LiveTV.LiveTvManager.RefreshChannelsInternal(IProgress`1 progress, CancellationToken cancellationToken)
	Source: Emby.Server.Implementations
	TargetSite: Void MoveNext()
	InnerException: System.Threading.Tasks.TaskCanceledException: The operation was canceled.
	Source: System.Net.Http
	TargetSite: Void MoveNext()
	   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
	   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
	   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.GetTempFileResponse(HttpRequestOptions options)
	InnerException: System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
	Source: System.Net.Sockets
	TargetSite: Void ThrowException(System.Net.Sockets.SocketError, System.Threading.CancellationToken)
	   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
	   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
	   at System.Net.Http.HttpConnection.FillAsync()
	   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
	   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
	InnerException: System.Net.Sockets.SocketException: Operation canceled
	Source: 
	TargetSite:

If you want the Jellyfin log too just ask.

Thanks in advance!

#2

Client side timeout. You can increase the timeout if the software lets you, or request less data.

1 Like
#3

Is there a reason the IP address/host is missing in all of your listings?

protocol://host:port/path/to/resource?parameter1=value1&parameter2=value2

#4

I did. I tried it at 9600. It still fails within seconds using either Emby or Jellyfin. I've also never been able to make it work using "Tivi-mate", but since that's just an Android app I've never bothered to go hunting for log data.

#5

It's just a silly habit I have from working with gov customers.

#6

HAH! Never mind.

I forgot about all the maintenance functions on the CDVR settings page. I deleted and recreated the guide DB. Problem solved. It just populated the guide on Jellyfin.

But FYI I did check the guide before on the CDVR page when the problem existed, and it was fully populated. Weird.

#7

If the problem is the timeout, then perhaps a better option is to setup a cronjob to pull the XMLTV data and save it locally, and then have Emby, et al., use the static/local data.

(As to the redacted IP: understood. But perhaps to avoid confusion and red herrings, overwrite the sensitive data with placeholder text, rather than removing it.)

1 Like
#8

Apparently the guide DB was just corrupt. Deleting and rebuilding that solved the problem. I initially forgot that was a function on the settings page.

#9

What/where was corrupt?? I thought you said it was loading fine on the dvr web UI?

#10

It was, but it was unreadable to external systems. A quick rebuild resolved the issue.

My guess is it's a flat file, right? Is there defrag happening behind the scenes?

#11

It's not a flat file. It uses a sophisticated log-structured merge tree. Unless the dvr log says there was corruption, then it's not likely corruption has anything to do with it. The few cases of guide database corruption we've had reported over the years have consistently been traced back to a bad stick of RAM or failing disk.

The database is memory mapped, so the only thing you did by delete/recreate was load everything back into the file system cache. That means it's more likely to be in RAM which means faster response times because it's not waiting on the disk. But that's a temporary effect because the OS will start paging as other parts of the system use the fs.

From the limited logs, the only thing I can tell is that Emby gave up while waiting for the guide data to load and needs a higher timeout. If you submit dvr diagnostics then I can check the actual guide data response times and corruption indicators.

#12

This part is pretty clear. The guide data download was cancelled and thus not allowed to complete.

1 Like
#13

Emby does in fact time out when trying to parse Guide Data from Channels DVR in my Case I use a batch job to create a Guide.XMl and use that in EMBY.

#14

ea703bb5-22f3-4aaa-a93b-9de1e206560c should be there.

As far as whether or not it was corruption I don't know. All I know is it failed consistently right before, and worked perfectly right after deleting and rebuilding the guide db. Based on my experience with other products that's a strong corruption indicator, but you're right, it could have been a myriad of other things.

Keep in mind, due to some CLI stuff I had to do I moved that Pi a few times to put it on a monitor. Each time I carefully shut it down by pressing the power button, lol. So the possibility of me cutting power during a guide update is definitely there, and that could easily have corrupted the file, no?

#15

I just did a test and in fact it does timeout ...

Data path: C:\Users\SAGETVUSER\AppData\Roaming\Emby-Server\programdata
Application path: C:\Users\SAGETVUSER\AppData\Roaming\Emby-Server\system
MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: Connection to http://10.0.0.153:8089/devices/ANY/guide/xmltv?duration=1209600 timed out
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.

#16

Yeah but now delete and rebuild the guide. That's all I did.

#17

It will eventually fail like tmm1 stated when you rebuilt it it was probably still in Ram.

In Case you are interested this is how I get guide data for EMBY

curl -sSL "http://10.0.0.153:8089/devices/ANY/guide/xmltv?duration=1209600" > guide.xml
3 Likes
#18

Thanks!. I'm not even gonna try asking Emby or Jellyfin how to increase the timeout, lol.

I hope Aman reads this. In spite of its flaws, this might be the best supported media app out there. But it's also young. I have no doubt that someday I'll be switching back over to Channels as a total solution someday, but that's gonna take time. Emby and Plex have been around forever, and they're not where they are because they're necessarily better. They've just been at it for over a decade each. Possibly two, I can't remember.

For now I'm still going to use it because it beats all the others at one thing for sure. It aggregates all my channel sources into a nice little and wicked efficient server that handles everything quite nicely. I say "corruption" and you guys think it's a bad word. Man, enterprise products with hundreds of millions of dollars in seats deal with that every day all day long. I know, I used to support one of them. I was more so HOPING it was corruption. Simply because that's an easy maintenance level fix. Chasing other companies that put every enhancement request on the back burner to die for a timeout change is the true nightmare, lol.

#19

I'm gonna see if this cron job works before making the switch. This should download it every night at 2am to a directory called "epg" I created in the emby config directory. I haven't set up a cron job in over 10 years, lol.

0 0 2 1/1 * ? * curl -sSL "http://10.5.1.211:8089/devices/ANY/guide/xmltv?duration=1209600" > /mnt/user/appdata/binhex-emby/config/epg/guide.xml