The "Logs" folder is using 60% of my recording drive. Safe to remove the content?

[mjitkop]

Thank you for the explanation.
Actually, the Logs/comskip folder is not much of a concern at this time because it is not taking much room compared to Logs/recording:

It's Logs/recording that is taking all the space with 6,116 folders that each contain a file called recording.log:

So I guess it's time for me to clean up.

[chDVRuser]

Ahh, the debug recording logs.

@eric

The good news is that you could run rm Logs/recording/*/recording.log at any point and it wouldn't cause any issues with the running of the system.

We haven't run into cases where people were recording large numbers of recordings and were space-conscious, so it hasn't been a priority for us to do more to handle this, but you bringing it up does put it back on our radar.

Before someone decides those debug recording logs aren't needed, they should read the rest of eric's post.

@eric

There are two kinds of data that we are logging in those logs.

One part is the HLS logs that you are referring to (that would have been enough for tracking down the issue with the looping).

The other part are the packet-by-packet logs that we have when rewriting the MPEG-TS streams to have consistent timestamps. Those are used less frequently, but are very important in being able to detect strange situations in TVE streams that could result in a corrupted stream.

We've decided to leave them on because we still occasionally run into situations where it is important for us to identify if there is an issue with our handling of the streams vs an actual corrupted stream coming from the source. Without this detail, we would just have to guess as to what the problem is, and would likely have to resort to assuming the issue is coming from the original source even though there's a small chance it is coming from us interpreting the stream badly.

And know when Channels DVR rewrites timestamps

@eric

It is used for all HLS-based sources across M3U, TVE, Pluto for both live and recordings.

[mjitkop]

Thanks for all the details.
For now, I will just clear the log files from last year to start and go from there.

[babsonnexus]

I just created a little script that runs once a week that deletes everything older than 7 days. The logs are only needed if there is an issue, and if you haven't had one in the past week, then they are good to go.

[vg425]

I'm running Channels DVR on an old MacBook with an external 4TB drive. My Logs folder is taking up about 40GB of space. Can I delete the two subfolders inside Logs called "comskip" and "recording"? Also there is a folder named "Streaming" taking up another 8GB. Any advice would be appreciated.

[Fofer]

@babsonnexus

I just created a little script that runs once a week that deletes everything older than 7 days. The logs are only needed if there is an issue, and if you haven't had one in the past week, then they are good to go

Sounds like it could be useful to lots of us, would you mind sharing the script?

Perhaps it’s something @bnhf can consider including in OliveTin.

[babsonnexus]

I don't know if this is a good fit for OliveTin, but here's the little PowerShell command I run:

ForFiles /p "C:\{YOUR CHANNELS DVR PATH HERE}\Logs\recording" /s /d -7 /c "cmd /c del /q @file"

I just put that in Task Scheduler and set it up to run everyday at 3am. It's not super universal for people using Macs or Dockers or whatnot.

[chDVRuser]

@vg425

My Logs folder is taking up about 40GB of space. Can I delete the two subfolders inside Logs called "comskip" and "recording"?

I'm assuming you already read this post above yours The "Logs" folder is using 60% of my recording drive. Safe to remove the content? - #6 by chDVRuser

@vg425

Also there is a folder named "Streaming" taking up another 8GB. Any advice would be appreciated.

Rather than repeat a topic here, see Channels Streaming Folders

[vg425]

Thanks for the reply. I deleted all files and folders inside the "recording" subfolder, but I have no idea if I can delete the contents of the "comskip" subfolder inside of Logs.

[chDVRuser]

The only reason you would want to keep the comskip folder contents are if you plan on viewing the comskip log (video.log) for your recordings or taking advantage of any of the comskip generated output files in there. If you delete them and need one of the files later, just re-comskip the recording and they'll be generated again.

I consider the size relatively small. Here's an example from one of my four servers.

[tmm1]

Comskip logs are also used for the commercial editor

[chDVRuser]

Thanks, didn't realize that!

[Edwin_Perez]

I would not touch the Comskip folder ... these should be automatically removed when deleting the file ? or am I wrong in thinking they are ?

[chDVRuser]

Yes, once the recording is deleted, its comskip folder will be deleted along with its Streaming/m3u8 folder.

[Edwin_Perez]

@chDVRuser

Yes, once the recording is deleted, its comskip folder will be deleted along with its Streaming/m3u8 folder.

On Window I use Robocopy ....

robocopy "Y:\ChannelsDVR\Logs" "C:\TempDir" /E /xd "Y:\ChannelsDVR\Logs\comskip" /move /minage:7
rmdir /s /q "C:\TempDir"

[bnhf]

@Fofer

Perhaps it’s something @bnhf can consider including in OliveTin.

@babsonnexus

I just put that in Task Scheduler and set it up to run everyday at 3am. It's not super universal for people using Macs or Dockers or whatnot.

I think the answer to if this is a good fit for OliveTin lies in whether or not most people have their "dvr" folder setup as a network share. Or maybe another way to look at it is that if someone wants to use an OliveTin Action for this, they'd need to setup that folder as a share.

Portainer makes it pretty easy to create a volume that maps to a CIFS (SMB) or NFS share, and then that volume can be bound to a directory in OliveTin. With that in place, various maintenance scripts including the deletion of logs would pretty easy, and platform independent.

Does that appeal?

[cyoungers]

@bnhf

Does that appeal?

Yes. I would find this useful.

[mjitkop]

+1 (since I started this thread )

[bnhf]

I'm doing some preliminary work to set this up with OliveTin, and I'd like to confirm with someone what to delete.

On my Channels DVR server, I have a folder called dvr/Logs/recording, and I'm assuming I want to delete all of the numbered directories underneath "recording" that are older than x days. It looks like each numbered folder only contains one file.

So in Bash, I'm thinking of a command like this:

find /mnt/dvr/Logs/recording -type d -mtime +7 -exec rm -r {} \;

Which should find all directories under "recording", and if they're more than 7 days old delete them and everything underneath them (recursively).

[racameron]

Just a question to the OP:

Which files in particular in the Logs directory are taking up all of the space? In each subdirectory under the comskip directory is a video.mpg file which may show as taking space (but actually isn't), but is only just a link/reference to the actual recording.

The idea that a bunch of text files are consuming 60% of your drive is ludicrous, so I thought perhaps this situation might just be a misunderstanding. (That's not to say it isn't possible, just merely improbable. If your text files are indeed consuming that much disk space, then I'll just retreat.)