DVR: Performance improvements (v2021.05.27.2341 prerelease)

[tmm1]

We are working on under-the-hood optimizations to improve performance with large libraries and many guide sources.

These changes are first being tested on the guide database starting in v2021.05.13.0008

In that version, guide data is now loaded twice as fast across the board.

Before:

[HTTP] | 200 | 458.176416ms | 10.0.1.10 | GET "/devices/TVE-Cox/guide?duration=86400"

After:

[HTTP] | 200 | 215.477542ms | 10.0.1.10 | GET "/devices/TVE-Cox/guide?duration=86400"

Similar improvements will be made in other areas in the coming days. The goal is to fix slowness and timeout experienced by users with a lot of imported content and stream links.

[tmm1]

There was a bug in v2021.05.14.0010 that was causing new recordings not to be scheduled along with this error in the logs:

Error creating job: gob: type not registered for interface: dvr.lazyAiring

This issue has been fixed in v2021.05.14.1917

After updating, reload your Schedule by going to Guide Database > Maintenance > Fetch Guide Updates

[Edwin_Perez]

I saw that error but everything in Prime Time appears to have recorded.

[tmm1]

With v2021.05.14.2020, you can now enable the new optimized storage backend for your recordings library.

This is currently done manually via curl API. Note that once you perform this conversion, your recorder.db and new backups will no longer work with older versions of the software.

The conversion process is quick and will finish in less than a few seconds. A backup will automatically be made before the conversion begins.

The API is:

curl -XPUT http://x.x.x.x:8089/dvr/database/optimize

For an example of the improvement, this is on my test DVR with 1844 Recordings:

Before:

[HTTP] | 200 | 543.508789ms | 10.0.1.10 | GET "/dvr/files"

After:

[HTTP] | 200 | 93.986996ms | 10.0.1.10 | GET "/dvr/files"

And the conversion api is very quick:

[HTTP] | 200 | 509.868909ms | 10.0.1.10 | PUT "/dvr/database/optimize"

[AeroR1]

great, already feels quicker. i might have to add my local files again if this speeds it up substantially.

[tmm1]

On my main DVR with 564 Shows, 4513 Recordings:

Before:

[HTTP] | 200 | 1.019867917s | 10.0.1.10 | GET "/dvr/files"
[HTTP] | 200 | 521.291417ms | 10.0.1.10 | GET "/dvr/recordings/upnext"

After:

[HTTP] | 200 | 310.716417ms | 10.0.1.10 | GET "/dvr/files"
[HTTP] | 200 | 105.362042ms | 10.0.1.10 | GET "/dvr/recordings/upnext"

More improvements are expected next week.

[djcastaldo]

This is fantastic! Everything UI is so much smoother. This was a much needed update.

[babsonnexus]

@tmm1, will this also allow adding larger libraries (i.e., >15,000 individual files) and avoid the timeout issue on the apps? I'm currently keeping my numbers at <9,000 to avoid it, but don't want to add the others back if not.

[tmm1]

Yes that is the goal of this work.

[Victor_Bodenmiller]

Can I run this API on an Nvidia Shield server?

[tmm1]

Yes you can run from any computer and put the IP of your shield server

[Victor_Bodenmiller]

That worked, thank you!

[Iommi]

I swore off the beta train to keep the wife happy till I heard about this. Works great, everything seems so much faster, she will be happy!

[scooter_scott]

Upgraded to the beta to try this out. Everything seems to be working good so far

[tmm1]

@tmm1

With v2021.05.14.2020, you can now enable the new optimized storage backend for your recordings library.

This is currently done manually via curl API. Note that once you perform this conversion, your recorder.db and new backups will no longer work with older versions of the software.

We released v2021.05.26.1807 stable yesterday, which is compatible with both new and old database formats.

Starting with pre-release v2021.05.27.2341, the library optimizations above will be applied automatically on upgrade so everyone can take advantage of the faster speeds.

Thanks to everyone who tried this over the last two weeks to make sure it works as expected.

[kylecopeland]

Great work. Congrats on getting this out there wider!

[babsonnexus]

@tmm1

Yes that is the goal of this work.

I just added back today everything I took out. So far, it appears to be working with no hiccups!

625 Shows
17,013 Recordings

Now on to Collections for Content!