Core Count for Comskip and Buffering

[mackid1993]

This is probably an easy answerable question: Is maxing out the core count for ComSkip going to introduce buffering in live streams? I noticed I was having intermittent buffering on a gigabit wired network and I had ComSkip maxed out to all 19 available threads on my server. I backed it off to four threads and this stuttering seemingly went away but I'm cautiously optimistic.

I'm just curious if this is probably what was causing that issue given that I have everything just completely wired and a straight up gigabit connection back to my server with very low latency and jitter With a i7-12700K 128GB of RAM what is an optimal number of threads to give ComSkip? Is 4 more than enough? Or should I give it like six or eight?

Thanks so much!

[chDVRuser]

I would recommend making an H.264 recording of an hour or more.
Then test doing comskip on that same recording using different thread counts.
You can view performance on your server while they run.
The time to complete a comskip run is logged in Channels DVR log.

Increasing the thread count gives diminishing returns.
In other words, using 4 threads is not twice as fast as using two.

[Shaggylive]

are you doing live comskip? I believe I only experienced this when live was on. I'm only on a n100 so comskip is limited to 3, but a few more threads would go a long way to keeping up with my adhd show flipping.

[mackid1993]

That's really good advice and ever since I lowered the thread count I so far haven't seen that buffering come back.

I was also half-ring buffering on Apple TV and my Nvidia Shield so I think it was that. I was just curious if anyone has experienced this before.
This is just the automated processing that Channels does in the background that I'm referring to.

[Rice]

@chDVRuser

Increasing the thread count gives diminishing returns.

This is my experience as well.

Thread Allocation for Commerical Detection

Here is the same 1 hour h264 TVE tv show 1 thread: finished in 16m38.476242275s 2 threads: finished in 9m35.732427208s 5 threads: finished in 4m39.551914119s 7 threads: finished in 4m33.719244678s 10 threads: finished in 3m32.847742912s All tests found the same amount of markers. CPU: i5-10400 16GB ram Based on these numbers it looks like the sweet spot is a cpu with a passmark of around 6000. I don’t currently have any OTA MPEG’s 2 recordings to test it on. Maybe later I’…

[hal9000]

Comskip should run at lower priority than the main DVR tasks related to streaming/recording. We have an unnecessary fight for CPU resources.

[mackid1993]

Thank you everyone! I haven't had any buffering since I lowered the thread count so I think I was just starving whatever Go Routine was running that handles the live TV streaming. However, I just changed the thread count to five because that seems like where returns start to diminish, and I have 19 threads, so I'll see how that works. If not, I can always back it off.

I really do appreciate all the input though.

[eric]

@mackid1993

That's really good advice and ever since I lowered the thread count I so far haven't seen that buffering come back.

Without knowing your exact setup it’s impossible to know for sure, but in general it’s actually disk contention/bandwidth that is the limiting factor that causes the buffering and not actually the CPU. What happens when you increase the CPU threads is it allows for comskip to process faster which uses more disk IO.

But all of this advise is good: you’re gonna be better off trying not to make comskip go as fast as possible. It’s better to let it happen more slowly to reduce the chances of using too many resources.

[mackid1993]

I mean reducing it seemed to solve the issue.
I mean, I have a gigabit connection back to the server with less than a millisecond of latency and jitter so it's not that.

It's an Unraid server with a i7-12700K and 128GB of RAM. The Channels database is running on a Samsung 980 Pro. So nothing's really starved for resources I would assume that you're probably right and it just had to do with possibly the number of CPU threads in disk IO.

Regardless, it hasn't happened since I dropped a number of threads so it must have been maxing out at the maximum amount of threads my server has that was causing the buffering. I appreciate the response Eric and all that you guys do!