Comskip: For my next trick, I'm going to make these commercials dissapear

For a couple months now I've been recording live tv as in comes in through my Over the air antenna with the help of my TabloTV. Check out this blog post to see how I copy the recorded shows from the tablo to my plex server.

But the idea came to me to take it a step further. Although it's nice that I can record these shows and watch them at my leisure (and even fast forward through any parts i want to) it's still pretty annoying to have to fast forward through the commercials.

As I searched around for a tool to fix this, I found several programs such as Avidemux that allow you to remove commercials (or any part) with relative ease and speed. I'm still not sure if Avidemux is a play on Avid (the sophisticated editing software) or A vide'o mux'er. Either way, there was one problem with these tools -- they were GUI based, which meant that automation would be difficult if not impossible.

Finally I found a tool that could also be used in the terminal -- Comskip. Comskip uses cues in the video such as black frames, audio silence/changes, and more to detect when commercials start. Then it outputs an ".edl" file which stands for edit decision list. The linux version isn't as well distributed as the windows version so I had to compile it from source and download some dependencies.
This page is where i started.

It requires ffmpeg 0.9 or greater and argtable2.

cd /var/tmp
mkdir ComSkipWork
wget http://ffmpeg.org/releases/ffmpeg-2.2.tar.bz2
tar xf ffmpeg-2.2.tar.bz2
cd ffmpeg-2.2
./configure --prefix=/var/tmp/ComSkipWork/install
make -j8
make install
cd ..
wget http://www.xilka.com/xilka/source/comskip-0.93i.tar.xz
tar xf comskip-0.93i.tar.xz
cd comskip-0.93i
PKG_CONFIG_PATH=/var/tmp/ComSkipWork/install/lib/pkgconfig ./configure --prefix=/usr
make -j8
sudo make install


Following the above steps line for line worked perfectly except for one spot

Running make -j8 shows an error about yasm not being installed. Simply run

sudo apt-get install yasm

**If you have other problems see the bottom of this post. I actually didn't start with the above steps and therefore ended up installing a lot things on my own which may or may not be necessary.**

Now that I had comskip installed, it was time to create/setup an .ini file.
A quick google search also revealed that some have customized their .ini files for different regions.

I downloaded USA_General by Wilky13.

For my purposes I just needed to make sure that it would produce an .edl file. After confirming it that, I could now run:

comskip "filename"

and get an .edl file which tells where the commercials are.

So using the .edl file I wrote a python script to scan for any raw videos (which in my case are always .mp4 files) and compress them into .mkv files. From there it analyzes the mkv file and creates multiple temp files of the content between commercial breaks -- or the actual show. The script then concatenates these temp files into a new mkv and moves the compressed version to a backup folder in case there was a mistake. Then it cleans up all the temp files and scans for the next raw mp4. Running from a cron job every night, the OTA shows goes all the way from being recorded to the tablo, copied to my hard drive, compressed and spliced, then loaded into plex automatically.

The code is a little sloppy, but one needs to simply modify the constants including path to a static ffmpeg, and the root directory of where all their tv shows are kept.
For example, on my server

I simply point to /../TabloTv/tv and all folders and sub folders in tv will be scanned.

Here's the python script.

usage:
./removeCommercials.py
(scans all folders and subfolders found in constants.TV_FOLDER)

or

./removeCommercials.py /path/to/filename
(executes script against passed in video only)

removeCommercials.zip


#cordCutter



TROUBLESHOOTING

My first attempt followed these similar steps (i already had a static build of ffmpeg so i thought i wouldn't need it here--wrong!!), but ended with much different results.

cd /var/tmp
mkdir ComSkipWork
wget http://www.xilka.com/xilka/source/comskip-0.93i.tar.xz
tar xf comskip-0.93i.tar.xz
cd comskip-0.93i
./configure --prefix=/usr
make -j8
sudo make install

Right about here

./configure --prefix=/usr

you might hit a error like this:

configure: error: *** A compiler with support for C++11 language features is required.

Ubuntu comes with the g++ compiler, but doesn't seem to cut it. Go ahead and run

sudo apt-get install g++

Now you might hit this issue:

 No package 'libavcodec' found

Another simple fix

 sudo apt-get install libavcodec-dev

And while you're at it

 sudo apt-get install libavformat-dev

 sudo apt-get install libargtable2-dev