Let's make our own smart channel collection to include sources

[mjitkop]

Continuing the discussion from OliveTin for Channels: An Interface for Misc Channels DVR Scripts & Tricks - #888 by Fofer in order not to pollute the OliveTin thread by @bnhf.

The idea is to try and make our own feature, developed by the community. That is unless the developers decide to make it an official feature in Channels DVR.

Until then, let's brainstorm and try to make it happen.

This is a feature more recently requested by @mmsketch:

Automatic Channel Collections - INCLUDE sources

Hi, I like to make channel collections of different sources so that they show up in the sidebar on my Apple TV app. For instance, when I added Pluto to my DVR, I went in and added every Pluto channel to a collection called Pluto. The problem with this method is that it's only a snapshot of what channels were available on Pluto when I created the collection. So when I look at the Pluto collection now (within the server) it's riddled with things like "This channel (pluto-cnn) no longer exists." …

I think the first thing to do is agree on the requirements. Here is my understanding of what has been requested:

1. REQ1: OliveTin (OT) shall enable the user to create smart channel collections that include one or more whole sources

2. REQ2: OT shall be able to detect when channel lineups change in channel sources

3. REQ3: OT shall update the smart channel collections (defined in REQ1) when any of the sources used in the smart channel collections have changed as specified in REQ2 so that the channel collections always mirror the current lineups

4. REQ4: OT shall update the Client-side settings when the old Channel Collection is deleted and a new one is created (per @Fofer )

I think this is mostly it for now. Some possible future requirements:

5. REQ5: OT shall enable the user to choose existing channel collections to be qualified as smart channel collections so that they get automatic updates as specified in REQ2 and REQ3

6. REQ6: OT shall enable the user to create a special channel collection called “Recent Channel Additions” that automatically contains newly added channels in any source (as mentioned by @Fofer in this post)

Does this list of requirements look correct?

Good news: REQ2 is already implemented.

[mjitkop]

I'm volunteering to offer some help with scripting REQ1 and REQ3 to start.
Everything GUI-related will have to be done by @bnhf in OT, of course.

Questions for @bnhf: in OT, can you easily read the list of channel lineup changes reported by Python script to be notified of channel lineup changes, or what would be better for you so you can get the changes per source?
I'm willing to modify/extend the script so that it can provide the necessary data in the most useable format for OT.

I was also thinking that we could create some persistent boolean variables (probably written to a file?) called: source_1_changed, source_2_changed, etc.
So the main script in OT would be looking for these variables to be set to True, which would be done by my script mentioned above (that's what I meant by extending it).
Then, once processed by OT, they would be reset to False.

Just brainstorming. Since you know OT better than anybody else in here, you are most qualified to let us know what would be better for you.

Just trying to figure out the implementation of the triggers for the channel collection updates.

[bnhf]

@mjitkop

Questions for @bnhf: in OT, can you easily read the list of channel lineup changes reported by Python script to be notified of channel lineup changes, or what would be better for you so you can get the changes per source?

Probably won't know for sure until I start working on it, but my early thinking is that the current text file you're updating should be sufficient.

I'm already able to pull the channels in a collection in the custom M3U generation script I wrote -- so I'm thinking I just pull the latest version of the collection, and add or subtract channels based on the output of your script.

So, in OliveTin one would specify the collection(s) they want monitored, and the frequency to check for updates -- which probably should match the frequency of checks they specified when setting up channel lineup change notifications.

[mjitkop]

@bnhf

I'm already able to pull the channels in a collection

Cool. So you already have the right command to create and update a channel collection?

From my observations while using the web inspector, creating a channel collection or updating a channel collection is exactly the same command.
The critical piece is the JSON payload, which basically overwrites the whole content of the channel collection all at once.

That simplifies things, and also allows for positioning channels exactly the way we want in the collection.

And this also means that the collection doesn't need to be deleted and recreated so I think REQ4 is not a concern and can be ignored.

So once we can link REQ2 to REQ3, this is pretty much the hardest part of this project.

[bnhf]

@mjitkop

So you already have the right command to create and update a channel collection?

No. Can you help with this part?

[mjitkop]

Yes, I can. Let me put some notes together and I will get back to you.

[fashioncents]

If REQ2 isn’t possible, an alternative would be to have a cronjob run on a regular basis, maybe every 15 minutes? But there are some problems with that, like it could create some weird side effects with the guide view and if you’re watching something the OSD would be displayed every time it’s updated (idk if that happens on all platforms but it does on Apple TV)

[mjitkop]

REQ2 is about looking at the contents of the channel sources in the background (on the server side) and comparing the channel lineups with previous copies to see the differences. This has already been implemented and works nicely in OliveTin.

This is purely a background task that interacts only with the server and has no impact on the clients.

[mjitkop]

Some technical details...

---

Create a new, empty channel collection:

request: POST http://127.0.0.1:8089/dvr/collections/channels/new

JSON payload: {name: "Untitled Collection"}
(when creating a default collection on the web UI; of course, a name can be passed directly in a script to match the name of the source)

My guess is that it's probably possible to create a new channel collection that contains channels to start if the JSON payload has an items list that is not empty (see the JSON response). To be verified later.

JSON response

{
    "slug": "27",
    "name": "Untitled Collection",
    "items": null,
    "created_at": 1726103434802
}

Once created, a channel collection is identified in a request by its slug number.

---

Retrieve the list of channel collections with their slug numbers:

request: GET http://127.0.0.1:8089/dvr/collections/channels

response preview:

full JSON response

[
    {
        "slug": "11",
        "name": "Virtual Channels",
        "items": [
            "virtual-8",
            "virtual-29",
            "virtual-30",
            "virtual-17",
            "virtual-21",
            "virtual-24",
            "virtual-19",
            "virtual-22"
        ],
        "created_at": 1674597842134,
        "updated_at": 1725944017151
    },
    {
        "slug": "22",
        "name": "DirecTV",
        "items": [
            "cbs",
            "fox",
            "abc-wabc",
            "abc",
            "frndly-55",
            "WEDH",
            "qvc",
            "hsn",
            "hsn2",
            "frndly-44",
            "cnn",
            "hln",
            "espn1",
            "espnews",
            "espnu",
            "espn2",
            "fs1",
            "fs2",
            "cbssports",
            "hgtv",
            "diy",
            "food",
            "cooking-channel",
            "frndly-6",
            "reelz",
            "sundance",
            "paramount",
            "tnt",
            "tntp",
            "tru",
            "trup",
            "tbs",
            "tbsp",
            "fx",
            "fxp",
            "cc",
            "lifetime",
            "frndly-35",
            "amc",
            "tcm",
            "tcmp",
            "fxm",
            "fxx",
            "fxxp",
            "wetv",
            "bbca",
            "aetv",
            "fyi",
            "history",
            "frndly-36",
            "frndly-12",
            "ngc",
            "ngcp",
            "ngwild",
            "travel",
            "discovery",
            "discovery-life",
            "own",
            "tlc",
            "motortrend",
            "animal-planet",
            "science",
            "investigation-discovery",
            "destination-america",
            "ahc",
            "disneyjr",
            "disneyjrp",
            "disney",
            "disneyp",
            "disneyxd",
            "disneyxdp",
            "toon",
            "freeform",
            "freeformp",
            "hallmark",
            "hallmarkmm",
            "hallmarkdrama",
            "frndly-30",
            "bet",
            "mtv",
            "ifc",
            "vh1",
            "plex-5e20b730f2f8d5003d739db7-606605549dbfca002d2ae875",
            "plex-5e20b730f2f8d5003d739db7-5fe27da18cdd46002d85efed",
            "pluto-5d486acc34ceb37d3c458a64",
            "plex-5e20b730f2f8d5003d739db7-5fb81660b7ad2d002dcb7335",
            "plex-5e20b730f2f8d5003d739db7-61d73ab0a7b7c7f703418516",
            "fbn",
            "fnc",
            "accuweather",
            "frndly-10",
            "frndly-17",
            "pluto-5ca671f215a62078d2ec0abf",
            "frndly-85",
            "pluto-5f21ea08007a49000762d349",
            "frndly-39",
            "plex-5e20b730f2f8d5003d739db7-62ffbeae0d5645524634ec99",
            "btn",
            "sec",
            "acc",
            "samsung-USBD300021B8",
            "nhkworld",
            "tastemade",
            "samsung-USBC3200022W6",
            "samsung-USBA300032J7",
            "cnni",
            "mysteryalley",
            "deportes"
        ],
        "created_at": 1706212597963,
        "updated_at": 1706402707340
    },
    {
        "slug": "23",
        "name": "Frndly TV",
        "items": [
            "frndly-44",
            "frndly-82",
            "frndly-46",
            "frndly-47",
            "frndly-57",
            "frndly-17",
            "frndly-20",
            "frndly-48",
            "frndly-6",
            "frndly-55",
            "frndly-56",
            "frndly-1",
            "frndly-2",
            "frndly-3",
            "frndly-30",
            "frndly-29",
            "frndly-15",
            "frndly-34",
            "frndly-35",
            "frndly-31",
            "frndly-33",
            "frndly-40",
            "frndly-10",
            "frndly-76",
            "frndly-90",
            "frndly-91",
            "frndly-85",
            "frndly-18",
            "frndly-22",
            "frndly-4",
            "frndly-45",
            "frndly-42",
            "frndly-54",
            "frndly-32",
            "frndly-7",
            "frndly-83",
            "frndly-84",
            "frndly-39",
            "frndly-16",
            "frndly-36",
            "frndly-12",
            "frndly-19",
            "frndly-53",
            "frndly-38",
            "frndly-41",
            "frndly-50",
            "frndly-102",
            "frndly-11",
            "frndly-127",
            "frndly-21"
        ],
        "created_at": 1706222791853,
        "updated_at": 1725944347087
    },
    {
        "slug": "24",
        "name": "Pluto TV",
        "items": [
            "pluto-62f54c11b3af68000702c304",
            "pluto-62f54c6439183b000769fb8f",
            "pluto-5efbd29e4aa26700076c0d06",
            "pluto-62cdc6c5a698740007d299b1",
            "pluto-5dc9b8223687ff000936ed79",
            "pluto-5dc9b875e280c80009a8a44a",
            "pluto-62cdc75b1a1cbd0007ed45dc",
            "pluto-604928d54a4f730007ff76bc",
            "pluto-632b5950b515b000079e6b81",
            "pluto-632b606e12a67b0007c9e371",
            "pluto-5eb1afb21486df0007abc57c",
            "pluto-5eb1aeb2fd4b8a00076c2047",
            "pluto-5dc481cda1d430000948a1b4",
            "pluto-5dc48170e280c80009a861ab"
        ],
        "created_at": 1706285187075,
        "updated_at": 1706285361147
    },
    {
        "slug": "25",
        "name": "tubi Live TV",
        "items": [
            "tubi-0",
            "tubi-1",
            "tubi-2",
            "tubi-3",
            "tubi-4",
            "tubi-5",
            "tubi-6",
            "tubi-7",
            "tubi-8",
            "tubi-159",
            "tubi-139",
            "tubi-9",
            "tubi-10",
            "tubi-11",
            "tubi-12",
            "tubi-13",
            "tubi-14",
            "tubi-15",
            "tubi-16",
            "tubi-17",
            "tubi-18",
            "tubi-19",
            "tubi-20",
            "tubi-21",
            "tubi-22",
            "tubi-23",
            "tubi-24",
            "tubi-25",
            "tubi-26",
            "tubi-27",
            "tubi-28",
            "tubi-29",
            "tubi-30",
            "tubi-31",
            "tubi-32",
            "tubi-33",
            "tubi-34",
            "tubi-35",
            "tubi-36",
            "tubi-37",
            "tubi-38",
            "tubi-39",
            "tubi-40",
            "tubi-41",
            "tubi-42",
            "tubi-43",
            "tubi-44",
            "tubi-45",
            "tubi-46",
            "tubi-47",
            "tubi-48",
            "tubi-49",
            "tubi-50",
            "tubi-51",
            "tubi-52",
            "tubi-53",
            "tubi-54",
            "tubi-55",
            "tubi-56",
            "tubi-57",
            "tubi-58",
            "tubi-59",
            "tubi-60",
            "tubi-61",
            "tubi-62",
            "tubi-63",
            "tubi-64",
            "tubi-65",
            "tubi-66",
            "tubi-67",
            "tubi-68",
            "tubi-69",
            "tubi-70",
            "tubi-71",
            "tubi-72",
            "tubi-73",
            "tubi-74",
            "tubi-75",
            "tubi-76",
            "tubi-77",
            "tubi-78",
            "tubi-79",
            "tubi-80",
            "tubi-81",
            "tubi-82",
            "tubi-83",
            "tubi-84",
            "tubi-85",
            "tubi-86",
            "tubi-87",
            "tubi-88",
            "tubi-89",
            "tubi-90",
            "tubi-91",
            "tubi-92",
            "tubi-93",
            "tubi-94",
            "tubi-95",
            "tubi-96",
            "tubi-97",
            "tubi-98",
            "tubi-99",
            "tubi-100",
            "tubi-101",
            "tubi-102",
            "tubi-103",
            "tubi-104",
            "tubi-105",
            "tubi-106",
            "tubi-107",
            "tubi-108",
            "tubi-109",
            "tubi-110",
            "tubi-111",
            "tubi-112",
            "tubi-113",
            "tubi-114",
            "tubi-115",
            "tubi-116",
            "tubi-117",
            "tubi-118",
            "tubi-119",
            "tubi-120",
            "tubi-121",
            "tubi-122",
            "tubi-123",
            "tubi-124",
            "tubi-125",
            "tubi-126",
            "tubi-127",
            "tubi-128",
            "tubi-129",
            "tubi-130",
            "tubi-131",
            "tubi-132",
            "tubi-133",
            "tubi-134",
            "tubi-135",
            "tubi-136",
            "tubi-137",
            "tubi-138",
            "tubi-140",
            "tubi-141",
            "tubi-142",
            "tubi-143",
            "tubi-144",
            "tubi-145",
            "tubi-146",
            "tubi-147",
            "tubi-148",
            "tubi-149",
            "tubi-150",
            "tubi-151",
            "tubi-152",
            "tubi-153",
            "tubi-154",
            "tubi-155",
            "tubi-156",
            "tubi-157",
            "tubi-158",
            "tubi-160",
            "tubi-161",
            "tubi-162",
            "tubi-163",
            "tubi-164",
            "tubi-165",
            "tubi-166",
            "tubi-167",
            "tubi-168",
            "tubi-169",
            "tubi-170",
            "tubi-171",
            "tubi-172",
            "tubi-173",
            "tubi-174",
            "tubi-175",
            "tubi-176",
            "tubi-177",
            "tubi-178",
            "tubi-179",
            "tubi-180",
            "tubi-181",
            "tubi-182",
            "tubi-183",
            "tubi-184",
            "tubi-185",
            "tubi-186",
            "tubi-187",
            "tubi-188",
            "tubi-189",
            "tubi-190",
            "tubi-191",
            "tubi-192",
            "tubi-193",
            "tubi-194",
            "tubi-195",
            "tubi-196",
            "tubi-197",
            "tubi-198",
            "tubi-199",
            "tubi-200",
            "tubi-201",
            "tubi-202",
            "tubi-203",
            "tubi-204",
            "tubi-205",
            "tubi-206",
            "tubi-207",
            "tubi-208",
            "tubi-209",
            "tubi-210",
            "tubi-211",
            "tubi-212",
            "tubi-213",
            "tubi-214",
            "tubi-215",
            "tubi-216",
            "tubi-217",
            "tubi-218",
            "tubi-219",
            "tubi-220",
            "tubi-221",
            "tubi-222",
            "tubi-223",
            "tubi-224",
            "tubi-225",
            "tubi-226",
            "tubi-227",
            "tubi-228",
            "tubi-229",
            "tubi-230",
            "tubi-231",
            "tubi-232",
            "tubi-233",
            "tubi-234",
            "tubi-235",
            "tubi-236",
            "tubi-237",
            "tubi-238",
            "tubi-239",
            "tubi-240",
            "tubi-241",
            "tubi-242",
            "tubi-243",
            "tubi-244",
            "tubi-245",
            "tubi-246",
            "tubi-247",
            "tubi-248",
            "tubi-249",
            "tubi-250",
            "tubi-251",
            "tubi-252",
            "tubi-253",
            "tubi-254",
            "tubi-255",
            "tubi-256",
            "tubi-257",
            "tubi-258",
            "tubi-259",
            "tubi-260",
            "tubi-261",
            "tubi-262",
            "tubi-263",
            "tubi-264",
            "tubi-265",
            "tubi-266",
            "tubi-267",
            "tubi-268",
            "tubi-269",
            "tubi-270",
            "tubi-271",
            "tubi-272",
            "tubi-273"
        ],
        "created_at": 1706817697156,
        "updated_at": 1706918964281
    },
    {
        "slug": "27",
        "name": "Untitled Collection",
        "items": [],
        "created_at": 1726103434802
    }
]

So the list of channels is in the items field. And the channels are referenced by their IDs.

---

Get the list of channels with their sources and IDs:

request: GET http://127.0.0.1:8089/devices

response preview (sample):

JSON response (sample)

[
    {
        "Provider": "m3u",
        "DeviceID": "M3U-PBS",
        "FriendlyName": "PBS",
        "ModelNumber": "HDHRCOMPAT-1",
        "Lineup": "X-M3U",
        "Channels": [
            {
                "ID": "WEDH",
                "GuideNumber": "33818",
                "GuideName": "WEDH",
                "HD": 1,
                "Station": "53126"
            },
            {
                "ID": "WNET",
                "GuideNumber": "33819",
                "GuideName": "WNET",
                "HD": 1,
                "Station": "26182"
            },
            {
                "ID": "WLIW",
                "GuideNumber": "33820",
                "GuideName": "WLIW",
                "HD": 1,
                "Station": "34325"
            }
        ]
    },
    {
        "Provider": "m3u",
        "DeviceID": "M3U-FrndlyTV(withEPG)",
        "TunerCount": 4,
        "FriendlyName": "Frndly TV (with EPG)",
        "ModelNumber": "HDHRCOMPAT-1",
        "Lineup": "X-M3U",
        "Channels": [
            {
                "ID": "frndly-31",
                "GuideNumber": "4000",
                "GuideName": "A\u0026E",
                "HD": 1,
                "Station": "51529",
                "Logo": "https://d229kpbsb5jevy.cloudfront.net/frndlytv/400/400/content/common/logos/channel/logos/jxidzf.png"
            },
            {
                "ID": "frndly-44",
                "GuideNumber": "4001",
                "GuideName": "MeTV",
                "HD": 1,
                "Station": "122696",
                "Logo": "https://d229kpbsb5jevy.cloudfront.net/frndlytv/400/400/content/common/logos/channel/logos/pkuofk.png"
            },
            {
                "ID": "frndly-127",
                "GuideNumber": "4002",
                "GuideName": "MeTV Toons",
                "HD": 1,
                "Station": "159817",
                "Logo": "https://d229kpbsb5jevy.cloudfront.net/frndlytv/400/400/content/common/logos/channel/logos/wllies.png"
            },
            {

The source name is in the field FriendlyName.
The channel ID is in the field ID (that's an easy one )

To modify a channel collection (change the channel order, remove channels, add channels) simply overwrite the items list for the collection.

---

Modify a channel collection:

request: PUT http://127.0.0.1:8089/dvr/collections/channels/27
(the channel collection is referenced by its slug number 27)

Adding 3 channels:

JSON payload: { "slug": "27", "name": "Untitled Collection", "items": [ "abc-wabc", "fox", "cbs" ], "created_at": 1726103434802 }

This was the payload created by the web UI. Later, I will try with a manual command to see if it's sufficient to provide only the items list since the slug number of the collection is provided in the request itself.

JSON response

{
    "slug": "27",
    "name": "Untitled Collection",
    "items": [
        "abc-wabc",
        "fox",
        "cbs"
    ],
    "created_at": 1726103434802,
    "updated_at": 1726107997497
}

What it looks like:

Now, I will remove "fox" and add "cspan" in its place:

request: PUT http://127.0.0.1:8089/dvr/collections/channels/27

JSON payload (still from the web UI for now): { "slug": "27", "name": "Untitled Collection", "items": [ "abc-wabc", "cspan", "cbs" ], "created_at": 1726103434802, "updated_at": 1726108519051 }

JSON response

{
"slug": "27",
"name": "Untitled Collection",
"items": [
"abc-wabc",
"cspan",
"cbs"
],
"created_at": 1726103434802,
"updated_at": 1726109104073
}

What it looks like:

[bnhf]

@mjitkop

Some technical details...

Excellent! Thank you. This should do nicely. Now to carve out a bit of time before we leave for Portugal...