The "Logs" folder is using 60% of my recording drive. Safe to remove the content?

[mjitkop]

I don't think this is really relevant but my server is running on Windows. Just FYI.

Currently, D:\DVR\Logs\ is using 410 GB, which is 60% of the whole D:\ drive.

I would think it should be safe to remove the content of this folder since it doesn't affect the database.

Can anybody confirm?

[Edwin_Perez]

I delete mine now every now and then.

[mjitkop]

Thanks for your quick reply. I thought it should be safe to clear it.

I wish there was an option to automatically delete a log file if the recording is successful (only keep log files in case of corrupted recordings).

[chDVRuser]

@mjitkop

Currently, D:\DVR\Logs\ is using 410 GB, which is 60% of the whole D:\ drive.

If you check you'll see that Windows is reporting the size of what the video.mpg symlink in the Logs/comskip folders point to (the actual recording file). There are actually no video.mpg files there, Windows is just following the symlink and reporting the size of the recording it points to that exists in other folders.

[mjitkop]

Thank you for the explanation.
Actually, the Logs/comskip folder is not much of a concern at this time because it is not taking much room compared to Logs/recording:

It's Logs/recording that is taking all the space with 6,116 folders that each contain a file called recording.log:

So I guess it's time for me to clean up.

[chDVRuser]

Ahh, the debug recording logs.

@eric

The good news is that you could run rm Logs/recording/*/recording.log at any point and it wouldn't cause any issues with the running of the system.

We haven't run into cases where people were recording large numbers of recordings and were space-conscious, so it hasn't been a priority for us to do more to handle this, but you bringing it up does put it back on our radar.

Before someone decides those debug recording logs aren't needed, they should read the rest of eric's post.

@eric

There are two kinds of data that we are logging in those logs.

One part is the HLS logs that you are referring to (that would have been enough for tracking down the issue with the looping).

The other part are the packet-by-packet logs that we have when rewriting the MPEG-TS streams to have consistent timestamps. Those are used less frequently, but are very important in being able to detect strange situations in TVE streams that could result in a corrupted stream.

We've decided to leave them on because we still occasionally run into situations where it is important for us to identify if there is an issue with our handling of the streams vs an actual corrupted stream coming from the source. Without this detail, we would just have to guess as to what the problem is, and would likely have to resort to assuming the issue is coming from the original source even though there's a small chance it is coming from us interpreting the stream badly.

And know when Channels DVR rewrites timestamps

@eric

It is used for all HLS-based sources across M3U, TVE, Pluto for both live and recordings.

[mjitkop]

Thanks for all the details.
For now, I will just clear the log files from last year to start and go from there.

[babsonnexus]

I just created a little script that runs once a week that deletes everything older than 7 days. The logs are only needed if there is an issue, and if you haven't had one in the past week, then they are good to go.

[vg425]

I'm running Channels DVR on an old MacBook with an external 4TB drive. My Logs folder is taking up about 40GB of space. Can I delete the two subfolders inside Logs called "comskip" and "recording"? Also there is a folder named "Streaming" taking up another 8GB. Any advice would be appreciated.

[Fofer]

@babsonnexus

I just created a little script that runs once a week that deletes everything older than 7 days. The logs are only needed if there is an issue, and if you haven't had one in the past week, then they are good to go

Sounds like it could be useful to lots of us, would you mind sharing the script?

Perhaps it’s something @bnhf can consider including in OliveTin.

[babsonnexus]

I don't know if this is a good fit for OliveTin, but here's the little PowerShell command I run:

ForFiles /p "C:\{YOUR CHANNELS DVR PATH HERE}\Logs\recording" /s /d -7 /c "cmd /c del /q @file"

I just put that in Task Scheduler and set it up to run everyday at 3am. It's not super universal for people using Macs or Dockers or whatnot.

[chDVRuser]

@vg425

My Logs folder is taking up about 40GB of space. Can I delete the two subfolders inside Logs called "comskip" and "recording"?

I'm assuming you already read this post above yours The "Logs" folder is using 60% of my recording drive. Safe to remove the content? - #6 by chDVRuser

@vg425

Also there is a folder named "Streaming" taking up another 8GB. Any advice would be appreciated.

Rather than repeat a topic here, see Channels Streaming Folders

[vg425]

Thanks for the reply. I deleted all files and folders inside the "recording" subfolder, but I have no idea if I can delete the contents of the "comskip" subfolder inside of Logs.

[chDVRuser]

The only reason you would want to keep the comskip folder contents are if you plan on viewing the comskip log (video.log) for your recordings or taking advantage of any of the comskip generated output files in there. If you delete them and need one of the files later, just re-comskip the recording and they'll be generated again.

I consider the size relatively small. Here's an example from one of my four servers.

[tmm1]

Comskip logs are also used for the commercial editor

[chDVRuser]

Thanks, didn't realize that!

[Edwin_Perez]

I would not touch the Comskip folder ... these should be automatically removed when deleting the file ? or am I wrong in thinking they are ?

[chDVRuser]

Yes, once the recording is deleted, its comskip folder will be deleted along with its Streaming/m3u8 folder.

[Edwin_Perez]

@chDVRuser

Yes, once the recording is deleted, its comskip folder will be deleted along with its Streaming/m3u8 folder.

On Window I use Robocopy ....

robocopy "Y:\ChannelsDVR\Logs" "C:\TempDir" /E /xd "Y:\ChannelsDVR\Logs\comskip" /move /minage:7
rmdir /s /q "C:\TempDir"

[bnhf]

@Fofer

Perhaps it’s something @bnhf can consider including in OliveTin.

@babsonnexus

I just put that in Task Scheduler and set it up to run everyday at 3am. It's not super universal for people using Macs or Dockers or whatnot.

I think the answer to if this is a good fit for OliveTin lies in whether or not most people have their "dvr" folder setup as a network share. Or maybe another way to look at it is that if someone wants to use an OliveTin Action for this, they'd need to setup that folder as a share.

Portainer makes it pretty easy to create a volume that maps to a CIFS (SMB) or NFS share, and then that volume can be bound to a directory in OliveTin. With that in place, various maintenance scripts including the deletion of logs would pretty easy, and platform independent.

Does that appeal?