[RELEASE] Playlist Manager for Channels [Streaming Library Manager Extension]

Maybe @bnhf could share examples that he implemented in OliveTin with project One-Click. :slightly_smiling_face:

Internal Server Error...
Hi All, first time posting.
I just updated to the latest V2025.01.16.1518 but I also saw it in the previous version.
BTW, thanks for this latest update, as it helps a lot when I'm assigning the channels due to the new search function when selected the Parent Station field.

Anyway, here is my issue. I am assigning my channels, and I then save them. I do about 10 at a time.
After it saves, then I click the button to Update the m3u and xml.
It works (spins) for a bit and then it gives me the internal server error.
I'm not sure if it actually crashes, as I just reload the main web page and I'm back up and running.
None of the container logs shows there was an issue, only the screen and the log in SLM (via Controls, Logs).

Below is a part of the SLM log that's of interest.
Note that the only time I get the error is when I try and Update the m3u.
Also note that it does indeed gives me a new m3u as when I load it in VLC, I actually see the increase in the number of channels I had added/assigned.

**************Start of log portion **************
[INFO | 2025-01-16 15:25:00,371] - 10.89.2.16 - - [16/Jan/2025 15:25:00] "e[36mGET /static/assets/js/core/bootstrap.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-16 15:25:00,392] - 10.89.2.16 - - [16/Jan/2025 15:25:00] "e[36mGET /static/assets/js/plugins/smooth-scrollbar.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-16 15:25:00,422] - 10.89.2.16 - - [16/Jan/2025 15:25:00] "e[36mGET /static/assets/js/material-dashboard.min.js?v=3.0.0 HTTP/1.1e[0m" 304 -
[INFO | 2025-01-16 15:25:00,499] - 10.89.2.16 - - [16/Jan/2025 15:25:00] "e[36mGET /static/assets/img/slm_flavicon.png HTTP/1.1e[0m" 304 -
Deleted: plm_epg_hls_m3u_01.m3u
Created: plm_epg_hls_m3u_01.m3u
[DEBUG | 2025-01-16 15:25:09,869] - Starting new HTTPS connection (1): bit.ly:443
[DEBUG | 2025-01-16 15:25:10,153] - https://bit.ly:443 "GET /moj-epg-gz HTTP/1.1" 301 128
[DEBUG | 2025-01-16 15:25:10,154] - Starting new HTTPS connection (1): github.com:443
[DEBUG | 2025-01-16 15:25:10,667] - https://github.com:443 "GET /dtankdempse/moveonjoy-m3u/raw/refs/heads/main/epg.xml.gz HTTP/1.1" 302 0
[DEBUG | 2025-01-16 15:25:10,671] - Starting new HTTPS connection (1): raw.githubusercontent.com:443
[DEBUG | 2025-01-16 15:25:11,164] - https://raw.githubusercontent.com:443 "GET /dtankdempse/moveonjoy-m3u/refs/heads/main/epg.xml.gz HTTP/1.1" 200 1585998
[ERROR | 2025-01-16 15:25:11,432] - Exception on /playlists/plm_main [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 1511, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 919, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/slm.py", line 2662, in webpage_playlists
get_final_m3us_epgs()
File "/app/slm.py", line 3247, in get_final_m3us_epgs
get_epgs_for_m3us()
File "/app/slm.py", line 3390, in get_epgs_for_m3us
response_text = response.content.decode('utf-8')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
2025-01-16 15:25:11.435368: ERROR: Webpage responded... 500 INTERNAL SERVER ERROR
Client IP: 10.89.2.16
Method: POST
URL: http://10.0.0.3:5000/playlists/plm_main
Headers:
Host: 10.0.0.3:5000
Connection: keep-alive
Content-Length: 39301
Cache-Control: max-age=0

**************End of log portion **************

I just want to say how much I appreciate the detailed documentation you produce for your projects. First rate!

Welcome to the community, @Waffles!

Okay, I see the issue. Your link to the XML Guide Data is a shortened URL to a compressed (.gz) file and PLM is expecting a direct link, looking for the link to end in .gz. This will be fixed in the next release (already tested a solution). In the meantime, you could either:

  • Use the non-compressed/"XML Format" link available at your provider

  • If you are concerned about the size and download time of the guide data, you could use the direct link to the compressed version by getting the "raw" link

  • If you are using Gracenote matching with the Parents, though, you could just remove the link to the XML Guide Data entirely as it is unnecessary

Sounds like a working version of hlstube on steroids.

1 Like

@babsonnexus - Thanks for the reply and the tip. I was able to get it to work using the uncompressed format. It took 4 minutes to download the xml file though.

The reply will be in two posts...one for the update on pulling the raw xml file... the second post will be a follow-up on the raw .gz file...

Part 1: Here is a portion with the other .gz files for comparison:
[DEBUG | 2025-01-17 09:04:10,558] - https://raw.githubusercontent.com:443 "GET /matthuisman/i.mjh.nz/d69fdfa30e5e12561c0d7c961476f7680589e95e/SamsungTVPlus/us.xml.gz HTTP/1.1" 200 353079
[DEBUG | 2025-01-17 09:04:10,747] - Starting new HTTPS connection (1): i.mjh.nz:443
[DEBUG | 2025-01-17 09:04:11,049] - https://i.mjh.nz:443 "GET /Stirr/all.xml.gz HTTP/1.1" 302 None
[DEBUG | 2025-01-17 09:04:11,051] - Starting new HTTPS connection (1): raw.githubusercontent.com:443
[DEBUG | 2025-01-17 09:04:11,422] - https://raw.githubusercontent.com:443 "GET /matthuisman/i.mjh.nz/d69fdfa30e5e12561c0d7c961476f7680589e95e/Stirr/all.xml.gz HTTP/1.1" 200 152423
[DEBUG | 2025-01-17 09:04:11,546] - Starting new HTTP connection (1): 10.0.0.3:8179
[DEBUG | 2025-01-17 09:04:11,548] - http://10.0.0.3:8179 "GET /epg HTTP/1.1" 302 None
[DEBUG | 2025-01-17 09:04:11,549] - Starting new HTTPS connection (1): raw.githubusercontent.com:443
[DEBUG | 2025-01-17 09:04:11,904] - https://raw.githubusercontent.com:443 "GET /dtankdempse/thetvapp-m3u/refs/heads/main/guide/epg.xml HTTP/1.1" 200 1510043
[DEBUG | 2025-01-17 09:04:12,234] - Starting new HTTPS connection (1): bit.ly:443
[DEBUG | 2025-01-17 09:04:12,613] - https://bit.ly:443 "GET /tubi-epg HTTP/1.1" 301 139
[DEBUG | 2025-01-17 09:04:12,617] - Starting new HTTPS connection (1): raw.githubusercontent.com:443
[DEBUG | 2025-01-17 09:04:12,980] - https://raw.githubusercontent.com:443 "GET /dtankdempse/tubi-m3u/refs/heads/main/tubi_epg_us.xml HTTP/1.1" 200 470300
[INFO | 2025-01-17 09:08:17,050] - 10.89.2.20 - - [17/Jan/2025 09:08:17] "GET /logs HTTP/1.1" 200 -

The issue is while I don't get the error, the screen never changes from Processing - Please wait while the process completes...
I waited over 10 minutes and then finally just reloaded the page...

@babsonnexus
This is part 2... this is just a fyi in case you see something that either I missed or if something else needs a review.

So, first I tried to use the raw .gz link to the file...
Unfortunately, it still gave me the 500 error...
Here's that part of the log:

[DEBUG | 2025-01-17 09:01:00,292] - Starting new HTTPS connection (1): github.com:443
[DEBUG | 2025-01-17 09:01:00,915] - https://github.com:443 "GET /dtankdempse/moveonjoy-m3u/blob/b44582e3f4c3797107f695da7244a1f3b79d9abf/epg.xml.gz HTTP/1.1" 200 None
[ERROR | 2025-01-17 09:01:01,017] - Exception on /playlists/plm_main [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 1511, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 919, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/slm.py", line 2662, in webpage_playlists
get_final_m3us_epgs()
File "/app/slm.py", line 3247, in get_final_m3us_epgs
get_epgs_for_m3us()
File "/app/slm.py", line 3388, in get_epgs_for_m3us
response_text = gz.read().decode('utf-8')
^^^^^^^^^
File "/usr/local/lib/python3.12/gzip.py", line 324, in read
return self._buffer.read(size)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/_compression.py", line 118, in readall
while data := self.read(sys.maxsize):
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/gzip.py", line 527, in read
if not self._read_gzip_header():
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/gzip.py", line 496, in _read_gzip_header
last_mtime = _read_gzip_header(self._fp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/gzip.py", line 456, in _read_gzip_header
raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b'\n\n')
2025-01-17 09:01:01.066522: ERROR: Webpage responded... 500 INTERNAL SERVER ERROR
Client IP: 10.89.2.20

---- I guess his .gz file is messed up as PLM is able to download other .gz files

Follow up- I guess I didn't wait long enough... I redid the update xml link and left the computer to do something else.. when I returned, I had the console back and below is the log. I guess the previous entry wasn't 10 minutes.

[INFO | 2025-01-17 09:34:28,218] - 10.89.2.20 - - [17/Jan/2025 09:34:28] "e[35me[1mGET /playlists/files/plm_epg_hls_m3u_01.m3u HTTP/1.1e[0m" 206 -
Deleted: plm_epg_hls_m3u_01.xml
Created: plm_epg_hls_m3u_01.xml

2025-01-17 09:42:39.982077: Finished generation of final m3u(s) and XML EPG(s).
[INFO | 2025-01-17 09:42:40,005] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "POST /playlists/plm_main HTTP/1.1" 200 -
[INFO | 2025-01-17 09:42:40,042] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/img/slm_navicon.png HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,044] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/css/nucleo-icons.css HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,049] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/css/material-dashboard.css?v=3.0.0 HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,050] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/css/nucleo-svg.css HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,088] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/js/plugins/perfect-scrollbar.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,089] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/js/core/popper.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,089] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/js/core/bootstrap.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,093] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/js/plugins/smooth-scrollbar.min.js HTTP/1.1e[0m" 304 -
[INFO | 2025-01-17 09:42:40,109] - 10.89.2.20 - - [17/Jan/2025 09:42:40] "e[36mGET /static/assets/js/material-dashboard.min.js?v=3.0.0 HTTP/1.1e[0m" 304 -

The link in your log to the .gz file isn't the right raw link; it's the Github location link (i.e., how Github renders). You'd need to click on the file and then select "Raw" to get the raw link.

FYI, I also recommend you just turning on MTM Automation instead of continually running updates in the foreground.

@babsonnexus Got it on both tips...
I found the other Raw link and updated my playlist...
I had already set up the automation, set it to run every 6 hours...
I was just doing the manual pull to test the process and trying to make it work.

Thanks again for the assist.

PUT http://<channels DVR IP>:8089/providers/m3u/sources/<name of source without spaces>

Example with IP = 127.0.0.1 and source name = "Test":

PUT http://127.0.0.1:8089/providers/m3u/sources/Test

JSON payload:

{
  "name": "Test",
  "type": "HLS",
  "source": "URL",
  "url": "http://localhost:5000/playlists/uploads/plmss_hls_m3u_01.m3u",
  "text": "",
  "refresh": "24",
  "limit": "",
  "satip": "",
  "numbering": "",
  "logos": "",
  "xmltv_url": "",
  "xmltv_refresh": ""
}

This will create a custom source named Test on the Channels DVR server located at 127.0.0.1:8089 and it will look like your screenshot on your Github:

This is good info, thank! Thus, it will be done:

2 Likes

I've been busy travelling so hadn't checked this out yet. Just wanted to say, this is a great implementation and it's working superbly over here. Assigning new channels is soooo much quicker now. Thanks for this update. Thanks for PLM and SLM and your excellent documentation! You rock!

2 Likes

This is awesome too! Wow, nicely done. I'm migrating over my favorite livestreams, that I previously integrated via the Kister Method. This new SLM feature is working very well and has some more customizability I'm really appreciating! It's certainly a more scalable workflow. One request I'd like to submit would be the option for some streams, for us to be able to turn off the time blocks. These are 24/7 backgroud music, ambient or white noise we'd never record anyway.

So rather than this:

I'd like to be able to make some of these streams appear in the guide as channels like this:

The continuous block helps with a visual cue that this is a 24/7 stream, not intended for recording, differentiating it from other channels with different episodes every hour, half hour, or whatever. Others may feel differently so if we could configure it, per stream, that would be great!

You can actually do this already! Just set the Guide Placeholder value to:

Ah, I was just gonna guess or suggest entering "0" to remove the blocks.

But you're miles ahead of me. Thanks again, times a million!

Where is this note, by the way? I'm not seeing it in the documentation, wondering where it is and what else I may have missed.

It's Channel's documentation, not SLM:

But I'll add something similar to the pop-up note, too, in the next release.

What does your Source and URL look like on this input screen form, is it Youtube again??

I've only just started, this is what mine looks like:

I don't think my "Test" button is working like yours though. I'm on a Mac. When I paste in a YouTube URL and click "Test," it just opens the YouTube page, as a seperate tab. It doesn't turn it into HLS or stream in a thumbnail, corner or popup. (I did have to allow pop-ups for even that to work, though.) And I tried with Safari and Chrome.

I'm not sure what else to try but maybe this is a Mac-specific bug.