I loaded all the US TV and Movies links into my server and I end up with alot of slowness(DS1019+ NAS on 2/4 version). I have Pluto and a few other other streaming service guides confined also. I just pushing it too hard importing everything?
What is slow and where?
I saw very high CPU and ram usage which lead to a very slow/hanging web site. The apple tv clients would be slow to load and sometime wouldn't see a guide or any content when loaded. After a few reboots, I restored the database to before I added Strmlinks, and its seems better now.
Should I be able to load all of them? I love this feature!
Thanks
I'm in the middle of the same thing. Adding all the US movie strmlnks to my DVR wasn't too bad but the processing the TV links completely maxed out my RAM and caused my entire NAS to become unusable.
I'm about to do a reinstall to try and restore my database.
Was channels-dvr using the RAM? How much was it using? Can you screenshot the activity monitor on your NAS?
What do you mean “all of them”?
How many movies and episodes does it report are in your library?
I cloned the folder from your github repo and tried to import all of the links, I'm assuming @Iommi meant the same thing.
I believe it was. I already restored from a backup and got rid of the links from my library, but I have 4GB of RAM and it was at 90+% for a few hours. The settings page of the admin web interface was stuck on the loading spinner and the log wasn't logging anything.
Also, all of my devices only loaded OTA channels (not TVE or custom M3U channels) and even those were struggling to stream.
I cloned the folder also.
I checked task manager and it was channels that was using close to 4BG and all the cpu. At one point I was able to stop Channels in Synology Web interface and Synology thought it was was stopped but task manager still saw it running and using all the cpu. I had to reboot the NAS to get it back. I'm sorry i should have taken some screen grabs at the time. Willing to debug it if you have any guidance. A few time I did try to submit a diagnostic log but the web interface was frozen before I did the restore
When you import something, any client that’s open and running will be notified and then make a request on the server. With thousands and thousands of stream links, you’re probably ddosing your server lol.
If the client is just slow browsing around, it’s probably due to 20,000 movies and tv episodes in your library.
Ymmv on using it day to say with this size of a library.
That repo with stream links was made to give examples of stream links and easy ways to pluck specific things. Importing the whole thing isn’t really a great idea.
1 Like
I finally decided to start playing around with strmlinks. Maybe not such a good idea. Like others in this tread I just dumped the entire GitHub contents into the imports directly. Doing that crippled the performance on my Synology DS920+ NAS. First it took several days for the DVR to scan all of those imports. Then it made startup of the Channels app on my ATV to take over 3 minutes to complete startup and display the guide.
Not sure if this is related but now I sometimes lose all of my sources on my ATV Channels client yet they still show up in the DVR web interface. I restart the DVR and they reappear.
Resource monitor on the NAS shows long periods of 85% CPU at times and a steady 25% memory.
I tried moving out the imported Moves and TV directories out of Import. Then I did a settings-->Local Content-->Prune Sources but that seemed to do nothing.
Ideas?
1 Like
So remember how long it was taking to add each record? That is about the same amount of time as it takes to remove them. If you check the logs, you'll probably see a lot more than "nothing" going on. You could restart the server or the service and that should kill the process, but you are going to have to do the prune anyway. Or start over with a fresh install. You could also delete them all instead in the front end, but there really isn't a quick way to do that like pruning.
BTW, the Github repository was a point in time reference; it is not up to date with current content. You can see all the files are dated 9 months ago. Also, even then it was based on Reelgood repository, which meant missing and misplaced episodes and things that don't match Gracenote.
For comparison sake, I have almost 20,000 StreamLinks between movies and episodes. Server performance (on my underwhelming 12 year old Windows laptop) is perfect and app start-up time is about 10-15 seconds. Before a large number of updates were done that made it a million times more efficient, I had to remove large swaths of my collection and prune. These were many, many hours long affairs. Since those updates, this number and I suspect a heck of a lot more would be fine, especially with the power of the server you are talking about.
Just for fun, I looked at the math. If you the dumped the entire repository, there are about 225,000 StreamLinks. You said it is taking about 180 second to open your app, so 180 / 225,000 = 0.0008. Take that number * 20,000 and you get 16 seconds, pretty close to the 10-15 seconds I said. So we could say as a general rule of thumb each new record (whether a StreamLink, recording, local file, whatever) is going to add something like 0.0006 to 0.0008 seconds to the start-up time of the app. Let's go high and say we want the app to start up at a minute or less. 60 seconds / 0.0008 seconds per record = 75,000 records.
Thus, there is a relationship we can use to estimate acceptable start-up time versus how many StreamLinks. Acceptable being the relative term to the individual. Let's say 20 seconds is the max one can stand, yielding 25,000 StreamLinks.
All said, I'm afraid your best solution is to just let the purge happen all weekend and come back to it again, but just slowly add a few things for testing purposes first before dumping a whole bunch in there.
Thank you @babsonnexus for the really informative response. I'll know next time to not try such a big import at one. Good to know that it is usable with lots of streamlinks. I should have looked at how old the GitHub updates was before importing.
Not knowing all the specifics you provided, I knew it was not going to be worth the effort trying to recover the system back to it's relatively stable state. So I uninstalled the DVR package in DSM then did a fresh install and restored from one of the Channels backups.
Side note to the Devs on the install wizard: This is only the second time I've reinstalled channels (first was to install the DSM7 beta image) each time I missed the tiny text on the sign-in page that need to click if you are doing a restore and ended up down the new user install path. It would be a better user experience if the restore from backup link more prominent.
Everything is back to working. I'll now be more judicious about what I import! Thanks for the help.
OMFG. I keep learning new stuff for Channels.
I always ignored strmlnks because I didn't want to find links to each thing. It seemed tedious.
Im totally going to crash my server this weekend trying to import the entire catalog. I'm EXCITED.
In case anyone else didn't know there was a repository
1 Like
I don't know if it is too late, but please don't. If you want to do a mass import, at least go through the catalogue first and only keep the shows/movies you are interested in with the services you pay for. Then confirm that they are still on the service. Per my note above, that is a 9-month-out-of-date incomplete catalogue that only exists for example/informational purposes.
OMGosh is it ever! But it's a one-time several week project followed by just regular maintenance as things get added/removed on each service. I'll continue to beg for an easier way like integrating JustWatch (they have an API, hint hint, wink wink), but it's not the worst thing in the world, especially after it is done.
2 Likes
Well, I took my own hint and begging and used JustWatch as the basis to create Stream Link Manager for Channels!
2 Likes