Feature request: Comskip ini access

[karlmitchell]

I am not customising per show. I am using the last version in this thread:

http://www.kaashoek.com/comskip/viewtopic.php?f=7&t=1741

with some modifications. I’m no expert, by any means.

Note that this is tuned as generally as possible to US broadcast TV. As is usual for comskip, what works for one circumstance will probably not work for all. It’s possible your simpler comskip.ini is better for non-US viewers.

---

detect_method=107 ; 1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
validate_silence=1 ; Default, set to 0 to force using this clues if selected above.
validate_uniform=1 ; Default, set to 0 to force using this clues (like pure white frames) if blackframe is selected above.
validate_scenechange=1 ; Default, set to 0 to force using this clues if selected above.
verbose=10 ; Show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=60 ; Frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=40 ; Frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=25 ; Maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
maxbright=1 ; New
brightness_jump=200 ; New
max_commercialbreak=500 ; Maximum length in seconds to consider a segment a commercial break
min_commercialbreak=21 ; Minimum length in seconds to consider a segment a commercial break, some sports require lower numbers than normal tv shows
max_commercial_size=197 ; Maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=4 ; Mimimum time in seconds for a single commercial
min_show_segment_length=250 ; Any segment longer than this will be scored towards show (sports and 30 minute programming), increase this to 250 for 1 hour long programming
non_uniformity=500 ; Set to 0 to disable cutpoints based on uniform frames
max_volume=500 ; Any frame with sound volume larger than this will not be regarded as black frame
max_silence=100 ;
min_silence=12 ; Any deep silence longer than this amount of frames is a possible cutpoint
noise_level=5 ; New
ticker_tape=0 ; Amount of pixels from bottom to ignore in all processing
ignore_side=0 ; New, default 0
subtitles=0 ; New, default 0
logo_at_bottom=0 ; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
use_existing_logo_file=1 ; Use logo file already created at beginning of show, default 0
two_pass_logo=1 ; New
punish=1 ; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3 ; Multiply when amount is above average punish_threshold
punish_modifier=4 ; When above average * threshold multiply score by this value
intelligent_brightness=0 ; Set to 1 to use a USA specific algorithm to tune some of the settings, not advised outside the USA
logo_percentile=0.92 ; If more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.7250 ; Digital signals can be higher
global_threshold=1.79 ; New
min_black_frames_for_break=1 ; Default 1
punish_no_logo=0 ; Default, set to 0 to avoid show segments without logo to be scored towards commercial
aggressive_logo_rejection=0
connect_blocks_with_logo=1 ; Set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable
delay_logo_search=0 ;
logo_filter=0 ; Set the size of the filter to apply to bad logo detection, 4 seems to be a good value
edge_step=2 ; New, default 1
cut_on_ar_change=1 ; Set to 1 if you want to cut also on aspect ratio changes when logo is present, set to 2 to force cuts on aspect ratio changes. set to 0 to disable
delete_show_after_last_commercial=0 ; Set to 1 if you want to delete the last block if its a show and after a commercial
delete_show_before_first_commercial=0 ; New, default 0
delete_show_before_or_after_current=0 ; Set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
delete_block_after_commercial=0 ; Set to max size of block in seconds to be discarded, set to 0 to disable
remove_before=0 ; Amount of seconds of show to be removed before ALL commercials
remove_after=0 ; Amount of seconds of show to be removed after ALL commercials
shrink_logo=5 ; Reduce the duration of the logo with this amount of seconds
after_logo=0 ; Set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
before_logo=0 ; Set to number of seconds before logo appears comskip should start to search for silence to insert an additional cutpoint
padding=0.50
ms_audio_delay=5
volume_slip=40
skip_b_frames=0 ; Set to 1 to force Comskip to skip frames for higher processing speed.
thread_count=2 ; Set to utilize two cores
max_repair_size=200 ; Will repair maximum 200 missing MPEG frames in the timeline, set to 0 to disable repairing for players that don’t use PTS
disable_heuristics=4 ; Bit pattern for disabling heuristics, adding 1 disables heristics 1, adding 2 disables heristics 2, adding 4 disables heristics 3, 255 disables all heuristics
delete_logo_file=1 ; Set to 1 if you want comskip to tidy up after finishing
output_edl=1
output_vdr=1 ; Set to 1 if you want XBMC to skipping commercials
output_ffmeta=1
output_ffsplit=1

[tmm1]

95% of those options are just being set to default values. I forgot how confusing and complicated all the comskip.ini examples out there are.

I suspect you’re getting better results because your detect method includes scene changes.

[tmm1]

For now, how about I use comskip.ini from the data directory if it exists there?

[karlmitchell]

Sounds good! Are there plans for internal optimization, to take into account recording settings (e.g. padding), country, channel, etc.? I imagine there are quite a few things that could be tweaked, and only a finite time to do so.

[tmm1]

This is implemented in v2017.02.22.0443.

[d21mike]

Is this dynamic based on available cores?
thread_count=2 ; Set to utilize two cores

I am running on MAC MINI i7 quad core and it runs very fast. And I think you only run 1 COMSKIP at a time which is probably a good idea. Assuming 1 COMSKIP at a time would be good to have it use all 4 cores.

The reason I ask is sometimes you can get a processor with 2 cores and each core runs faster then a each processor with 4 cores. My 4 cores run very fast but am considering upgrading my NAS in the future and might consider running the DVR on it (maybe).

[d21mike]

Based on my prior post I went to find the current comskip.ini on my MAC and could not find it. Pretty sure it used to be with the COMSKIP program. Did you move it or is it hidden?

You said you did the following. Is the file for changes to your default comskip.ini? Of is it replacement for your comskip.ini file. I.E. If you only used 2 cores and I wanted to use 4 cores could I just add that 1 parameter to a comskip.ini in the data folder and you would use both comskip.ini? Or do I need to copy your comskip.ini and make the changes to it?
For now, how about I use comskip.ini from the data directory if it exists there

[tmm1]

The comskip.ini is copied into each Log folder where comskip runs.

If you add a custom comskip.ini it needs to be the full config, as they are not combined.

Here is the default comskip.ini the DVR uses:

use_existing_logo_file=0
thread_count=1
verbose=10
lowres=1
logo_fraction=0.39
delete_logo_file=1
output_vdr=1
output_edl=1
output_ffmeta=1
output_ffsplit=1

[chDVRuser]

Just wanted to add...
In order to override the default Channels DVR comskip.ini, you need to place your modifed comskip.ini in its data directory which can be found by examining the Channels DVR log and looking for the line

2019/12/17 15:39:38 [SYS] Starting Channels DVR v2019.12.17.0213 (linux-x86_64 pid:24461) in /volume1/@appstore/ChannelsDVR/channels-dvr/data

So in my case the data directory is at /volume1/@appstore/ChannelsDVR/channels-dvr/data

[chDVRuser]

And once you have your modified comskip.ini in that data directory, Channels DVR will use it and no longer copy a comskip.ini file into the Logs directory. Channels DVR (when updated) will not overwrite your override comskip.ini file.

You can verify your override comskip.ini settings are being used by looking at the comskip log file in the Logs directory named video.log.

[arwrite]

I am going to put the comskip.ini into my data directory as suggested to that I may increase the thread count to '3'. The current comskip file is either 17 or 18 lines. The one with the extra line also contains 'lowres=1' at the end. So for my override comskip.ini file, do I include this line or not? What does this line do, and will it be a problem if my override file (which never changes?) does or does not contain this, depending on the content? Thanks

[arwrite]

Sorry, this goes with my last post. Here is the current 18 line comskip.ini I see, including 'lowres=1' at the end. The first line is blank:

verbose=10
detect_method=123 ;; everything but scenechange
punish_no_logo=0
cc_correct_type_modifier=1
length_nonstrict_modifier=1.5 ;; required before below due to comskip.ini parsing bug
combined_length_nonstrict_modifier=1
min_show_segment_length=222
min_commercialbreak=45
use_existing_logo_file=0
logo_fraction=0.39
delete_logo_file=1
output_vdr=1
output_edl=1
output_ffmeta=1
output_ffsplit=1
thread_count=1
lowres=1

[chDVRuser]

Just change thread_count=1 to thread_count=3.
You may want to check your CPU usage as that would take 3 cores.
Each Channels DVR transcode session also takes a core.
The daily Channels DVR guide update (between 9-11am) although short-lived also takes a core.

As far as what each setting does you can check the comskip.txt file in the distribution.
More info at http://www.kaashoek.com/comskip/

lowres=0 (0-10)
;Enables resolution reduction, setting to 10 makes comskip choose the best value automatically
;;This only works for certain types of video encoding, may not work for h.264 encoded video and may deliver bad results when used with hardware_decode=1

Here's what I'm using, with added output for VideoRedo Project file for commercial cutting.

verbose=10
detect_method=123 ;; everything but scenechange
punish_no_logo=0
cc_correct_type_modifier=1
length_nonstrict_modifier=1.5 ;; required before below due to comskip.ini parsing bug
combined_length_nonstrict_modifier=1
min_show_segment_length=222
min_commercialbreak=25
use_existing_logo_file=0
logo_fraction=0.39
delete_logo_file=1
output_vdr=1
output_edl=1
output_ffmeta=1
output_ffsplit=1
thread_count=2
lowres=1
output_videoredo3=1
videoredo_offset=0

[arwrite]

So, either 'lowres=1' or "null for this line" was what my system created for each recent comskip job within each individual comskip.ini. Since I use both TVE and silicondust quatro, I'm guessing the system inserted 'lowres=1' for content that was SD content (over silicondust tuner). Since the comskip.ini placed in 'data' will be "permanent" and used for ALL jobs, will having 'lowres=1' be bad for HD content recorded from TVE?

[chDVRuser]

No.
Not sure how you got different comskip.ini contents.
The default includes the line lowres=1

It basically tells comskip to look at a low res version of your recording to find the channel logo, etc. which speeds it up vs. having to scan the high res version.
You can play with all the comskip settings, but my life is too short for that stuff
i.e. return in time investment to do that isn't worth the gain

[arwrite]

Thanks for your input. So I looked at a few different 'Log' folders when looking for the system generated comskip.ini (to copy it). Upon examining a few recently dated log folders is where I noticed the discrepancy in the contents of the ini file. I guess that was my original question, why does the system create different ini files with or without the lowres setting? And, by placing a new 'default' one that is not system generated, I am no longer allowing the system to generate that specific file for the job and will only get the new default one I will put into the data folder.

Does anyone else know that answer to this? Maybe choosing to include the 'lowres' setting or not is not a big deal? I guess I just like to understand how things work and what the impact of my changes will be on the system before I make them, rather than 'playing with different settings'. I did look through the comskip forum in the link provided but nothing jumped out at me. Hoping the developers may know?

[tmm1]

lowres is only used for HD recordings. I don't recall offhand why that is.

[chDVRuser]

If you want to experience a true time sink, go there.
My suggestion is start with the defaults provided and experiment from there changing one thing at a time.
Many settings are interrelated.
The devs can tell you it's not a quick fix, but an iterative experience into your own patience and understanding.

[imx]

Old thread, I know, but is this still the default @tmm1?

I'd like to enable an extra thread or two, but I'd like to start out with the exact same settings as Channels uses normally (which works pretty well for me)... before potentially fiddling further.

EDIT: Ah, found the examples in the recorded directory. For anyone else, example:

/media/tv-recorded/Logs/comskip/1439

[RJ_DVR]

Just discovered if you feed Channels a tuned .ini file with all the settings it bombs.. Would love to know why though, I understand if the other settings are NOT used but to cause a failure?

detect_method=43 ;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
validate_silence=1 ; Default, set to 0 to force using this clues if selected above.
validate_uniform=1 ; Default, set to 0 to force using this clues (like pure white frames) if blackframe is selected above.
validate_scenechange=1 ; Default, set to 0 to force using this clues if selected above.
verbose=0 ;show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=60 ;frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=40 ;frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=25 ;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
max_commercialbreak=600 ;maximum length in seconds to consider a segment a commercial break
min_commercialbreak=25 ;minimum length in seconds to consider a segment a commercial break
max_commercial_size=125 ;maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=4 ;mimimum time in seconds for a single commercial
min_show_segment_length=250 ; any segment longer than this will be scored towards show.
non_uniformity=500 ; Set to 0 to disable cutpoints based on uniform frames
max_volume=500 ; any frame with sound volume larger than this will not be regarded as black frame
min_silence=12 ; Any deep silence longer than this amount of frames is a possible cutpoint
ticker_tape=0 ; Amount of pixels from bottom to ignore in all processing
logo_at_bottom=0 ; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
punish=0 ; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3 ; Multiply when amount is above average * punish_threshold
punish_modifier=2 ; When above average * threshold multiply score by this value
intelligent_brightness=0 ; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA
logo_percentile=0.92 ; if more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.75
punish_no_logo=1 ; Default, set to 0 to avoid show segments without logo to be scored towards commercial
aggressive_logo_rejection=0
connect_blocks_with_logo=1 ; set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable
logo_filter=0 ; set the size of the filter to apply to bad logo detection, 4 seems to be a good value.
cut_on_ar_change=1 ; set to 1 if you want to cut also on aspect ratio changes when logo is present, set to 2 to force cuts on aspect ratio changes. set to 0 to disable
delete_show_after_last_commercial=0 ; set to 1 if you want to delete the last block if its a show and after a commercial
delete_show_before_or_after_current=0 ; set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
delete_block_after_commercial=0 ;set to max size of block in seconds to be discarded, set to 0 to disable
remove_before=0 ; amount of seconds of show to be removed before ALL commercials
remove_after=0 ; amount of seconds of show to be removed after ALL commercials
shrink_logo=5 ; Reduce the duration of the logo with this amount of seconds
after_logo=0 ; set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
padding=0
ms_audio_delay=5
volume_slip=40
skip_b_frames=0 ; Set to 1 to force Comskip to skip frames for higher processing speed.
hardware_decode=0 ; Set to 1 to enable hardware accelerated video decoding, only available in donator version
max_repair_size=200 ; Will repair maximum 200 missing MPEG frames in the timeline, set to 0 to disable repairing for players that don't use PTS.
disable_heuristics=4 bit pattern for disabling heuristics, adding 1 disables heristics 1, adding 2 disables heristics 2, adding 4 disables heristics 3, 255 disables all heuristics
delete_logo_file=1 ; set to 1 if you want comskip to tidy up after finishing
output_framearray=0 ; create a big excel file for detailed analysis, set to 0 to disable
output_data=0 ; create a dump of the user data channel, used for CC and XDS (such as V-Chip info). Can be use together with output_framearray to remote debug CC decoding
output_videoredo=0 ; The old videoredo format
output_videoredo3=0 ; The new videoredo v3 format.
output_womble=0
output_mls=0 ; set to 1 if you want MPeg Video Wizard bookmark file output
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_dvrcut=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=1
output_dvrmstb=0 ; Set to 1 if you're running DVRMS-Toolbox
output_edlx=0
output_vcf=0
output_bsplayer=0
output_btv=0 ; set to 1 if you want Beyond TV chapter cutlist output
output_projectx=0 ; set to 1 if you want ProjectX cutlist output (Xcl)
output_avisynth=0
output_vdr=0 ; set to 1 if you want XBMC to skipping commercials
output_demux=0 ; set to 1 if you want comskip to demux the mpeg file while scanning
sage_framenumber_bug=0
sage_minute_bug=0
live_tv=0 ; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4 ; change to 16 when using live_tv in BTV, used for mpeg PS and TS
dvrms_live_tv_retries=300 ; only used for dvr_ms
standoff=0 ; change to 8000000 when using live_tv in BTV
cuttermaran_options="cut="true" unattended="true" muxResult="false" snapToCutPoints="true" closeApp="true""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25 /Z %2 %1"
avisynth_options="LoadPlugin("MPEG2Dec3.dll") \nMPEG2Source("%s")\n"
dvrcut_options="dvrcut "%s.dvr-ms" "%s_clean.dvr-ms" "
windowtitle="Comskip - %s"
use_existing_logo_file=0
two_pass_logo=1
maxbright=1
noise_level=5
brightness_jump=200
fps=25
global_threshold=1.05
lowres=0
play_nice_start=-1
play_nice_end=-1
play_nice_sleep=10
thread_count=8
ar_delta=0.08
added_recording=14
delete_show_before_first_commercial=0
min_commercial_break_at_start_or_end=39
always_keep_first_seconds=0
always_keep_last_seconds=0
black_percentile=0.0076
uniform_percentile=0.003
score_percentile=0.71
length_strict_modifier=3
length_nonstrict_modifier=1.5
combined_length_strict_modifier=2
combined_length_nonstrict_modifier=1.25
ar_wrong_modifier=2
ac_wrong_modifier=1
excessive_length_modifier=0.01
dark_block_modifier=0.3
min_schange_modifier=0.5
max_schange_modifier=2
logo_present_modifier=0.01
reward=0
reward_modifier=0.5
border=10
give_up_logo_search=2000
delay_logo_search=0
logo_max_percentage_of_screen=0.1
ignore_side=0
ignore_left_side=0
ignore_right_side=0
logo_percentage_threshold=0.25
edge_level_threshold=5
edge_radius=2
edge_weight=10
edge_step=1
num_logo_buffers=50
logo_fraction=0.4
before_logo=0
where_logo=0
min_black_frames_for_break=0
ccCheck=0
cc_commercial_type_modifier=4
cc_wrong_type_modifier=2
cc_correct_type_modifier=0.75
div5_tolerance=-1
output_default=1
output_plist_cutlist=0
output_scf=0
videoredo_offset=2
edl_skip_field=0
output_edlp=0
output_ipodchap=0
output_debugwindow=0
output_tuning=0
output_training=0
output_false=0
output_aspect=0
output_srt=0
output_smi=0
output_timing=0