Comskip: For my next trick, I'm going to make these commercials dissapear

For a couple months now I've been recording live tv as in comes in through my Over the air antenna with the help of my TabloTV. Check out this blog post to see how I copy the recorded shows from the tablo to my plex server.

But the idea came to me to take it a step further. Although it's nice that I can record these shows and watch them at my leisure (and even fast forward through any parts i want to) it's still pretty annoying to have to fast forward through the commercials.

As I searched around for a tool to fix this, I found several programs such as Avidemux that allow you to remove commercials (or any part) with relative ease and speed. I'm still not sure if Avidemux is a play on Avid (the sophisticated editing software) or A vide'o mux'er. Either way, there was one problem with these tools -- they were GUI based, which meant that automation would be difficult if not impossible.

Finally I found a tool that could also be used in the terminal -- Comskip. Comskip uses cues in the video such as black frames, audio silence/changes, and more to detect when commercials start. Then it outputs an ".edl" file which stands for edit decision list. The linux version isn't as well distributed as the windows version so I had to compile it from source and download some dependencies.
This page is where i started.

It requires ffmpeg 0.9 or greater and argtable2.

cd /var/tmp
mkdir ComSkipWork
wget http://ffmpeg.org/releases/ffmpeg-2.2.tar.bz2
tar xf ffmpeg-2.2.tar.bz2
cd ffmpeg-2.2
./configure --prefix=/var/tmp/ComSkipWork/install
make -j8
make install
cd ..
wget http://www.xilka.com/xilka/source/comskip-0.93i.tar.xz
tar xf comskip-0.93i.tar.xz
cd comskip-0.93i
PKG_CONFIG_PATH=/var/tmp/ComSkipWork/install/lib/pkgconfig ./configure --prefix=/usr
make -j8
sudo make install


Following the above steps line for line worked perfectly except for one spot

Running make -j8 shows an error about yasm not being installed. Simply run

sudo apt-get install yasm

**If you have other problems see the bottom of this post. I actually didn't start with the above steps and therefore ended up installing a lot things on my own which may or may not be necessary.**

Now that I had comskip installed, it was time to create/setup an .ini file.
A quick google search also revealed that some have customized their .ini files for different regions.

I downloaded USA_General by Wilky13.

For my purposes I just needed to make sure that it would produce an .edl file. After confirming it that, I could now run:

comskip "filename"

and get an .edl file which tells where the commercials are.

So using the .edl file I wrote a python script to scan for any raw videos (which in my case are always .mp4 files) and compress them into .mkv files. From there it analyzes the mkv file and creates multiple temp files of the content between commercial breaks -- or the actual show. The script then concatenates these temp files into a new mkv and moves the compressed version to a backup folder in case there was a mistake. Then it cleans up all the temp files and scans for the next raw mp4. Running from a cron job every night, the OTA shows goes all the way from being recorded to the tablo, copied to my hard drive, compressed and spliced, then loaded into plex automatically.

The code is a little sloppy, but one needs to simply modify the constants including path to a static ffmpeg, and the root directory of where all their tv shows are kept.
For example, on my server

I simply point to /../TabloTv/tv and all folders and sub folders in tv will be scanned.

Here's the python script.

usage:
./removeCommercials.py
(scans all folders and subfolders found in constants.TV_FOLDER)

or

./removeCommercials.py /path/to/filename
(executes script against passed in video only)

removeCommercials.zip


#cordCutter



TROUBLESHOOTING

My first attempt followed these similar steps (i already had a static build of ffmpeg so i thought i wouldn't need it here--wrong!!), but ended with much different results.

cd /var/tmp
mkdir ComSkipWork
wget http://www.xilka.com/xilka/source/comskip-0.93i.tar.xz
tar xf comskip-0.93i.tar.xz
cd comskip-0.93i
./configure --prefix=/usr
make -j8
sudo make install

Right about here

./configure --prefix=/usr

you might hit a error like this:

configure: error: *** A compiler with support for C++11 language features is required.

Ubuntu comes with the g++ compiler, but doesn't seem to cut it. Go ahead and run

sudo apt-get install g++

Now you might hit this issue:

 No package 'libavcodec' found

Another simple fix

 sudo apt-get install libavcodec-dev

And while you're at it

 sudo apt-get install libavformat-dev

 sudo apt-get install libargtable2-dev 

Open Sprinkler

As summer approached this year I realized my sprinkler controller was broken and my grass started to die. As a new homeowner, I was terrified and needed to find a new controller. Not wanting to buy a dinosaur i looked into smart controllers and found Open Sprinkler. There are other smart sprinkler systems but this one is by far the cheapest and from what i could tell, there are no real disadvantages.

As the name indicates it is an open source project and thus there is probably more assembly required than with a branded commercial product. But when it came down to it, installation was very easy. I simply switched the wires from the old controller

to the different zones on the new controller.

I was also able to reuse the power cord from my old controller and do all this setup in less than 20 minutes.

The real problem then came when i had to figure out a way to get my device connected to the internet. The controller has an ethernet port and wifi adapters can be purchased separately, but i had decided ahead of time i would try to convert my old raspberry pi into an adapter. After about two days following every tutorial i could and with no success i finally found an alternative solution. I used an old router (flashed with DD-WRT) to create a bridge. Now the router was working in reverse to receive internet connection over Wi-Fi and send it out through the ethernet into the Open Sprinkler controller. Within 30 minutes I had a connected open sprinkler system and was able to test all the zones.

Since that time i have not needed to use or even look at the controller sitting in the garage as all configuration can be done through either a free app or on a website which is served by the controller on your local network. Oh and did i mention that you can do remote access as well via port 80. Since I was already running a web server on port 80, I had to setup a subdomain and proxy to forward requests to the controller. It looks like this.

<VirtualHost *:80>
 ServerName sprinkler.tengentllc.com
 ProxyRequests Off
 <Proxy *>
  Order deny,allow
  Deny from all
  Allow from 192.168.1.4
 </Proxy>
 ProxyPass / http://192.168.1.4:80/
 ProxyPassReverse / http://192.168.1.4:80/
 <Location />
  Order allow,deny
  Allow from all
 </Location>
</VirtualHost>

Other great features include Weather and logs. By creating a developer account with weatherundergound.com your system can automatically access weather information and prevent recurring cycles if for example, it just rained that day. The logs are great as well and allow me to see total water usage and runtimes.

So glad I decided not to replace my broken controller with something like this!

Tablo Followup

I mentioned before that I would follow up on my post about TabloTv after I'd had some more experience with it.

The tablo tv is easy enough to setup and setting up recording is pretty straightforward too. I'm glad that I've had the 30-day trial for the TV guide feature and think i will be subscribing once it expires. Watching recorded shows via the Tablo interface or app was never my intention and i'm glad that this was the case as the startup can be a little slow at times. Upon entering the web client there is a "syncing" screen that appears and prevents you from doing anything for 30 seconds to a minute. Then when you decide on an episode to watch it can take 20 - 30 seconds before it actually starts playing. Playback is pretty smooth from that point on and I do like the live tile preview when using the 30 second and 20 second FF and RW buttons, but it's a little slow when compared to Plex which is more or less instantaneous depending on your setup. But the biggest downside is that I cannot easily share my recorded content as I have grown accustomed to in sharing my Plex library.

My intention with purchasing the Tablo was to record the shows and then allow Plex to be the player. Although there is a free Plex channel for Tablo which makes it very easy to view and play my Tablo library, I cannot share Plex channels the same way I share Plex libraries.

So I scoured the web and found this little gem. Actually it's a python script (insert ruby joke).

Essentially I've setup the Tablo to record the shows I want. This script (which runs on a cron job every night)

 pulls any newly recorded shows from the Tablo and places them into a library I've created on Plex called TabloTV. I've enabled Plex to automatically update my library twice a day so that every morning I wake up, all my new shows are there. Now I can share this TabloTV library with anyone else and goodbye HULU/Network sites, hello on-demand streamable, shareable, DVR with the freshest content available on all 4 networks.

The one downside is that I now have two copies of every episode. One of my Tablo and one on Plex. But no matter, the Tablo has a nice Auto Delete feature which I'm sure will free up space JIT style. I didn't think I would need the 4-tuner so I opted for the 2-tuner. What I didn't realize was that basically all the good shows run at the same time, so if I could do it again I would get the 4-tuner. But maybe it's a good thing. I probably watch too much TV as it is.


#Tablo
#Plex
#cordCutter

Tablo

"Somewhere in Las Vegas, a man just threw his Television outside his hotel room window."
            -Simon Cowell

Why do I remember this line from American Idol season 6 (which aired over 8 years ago)? Because it was hilarious. Would I ever go back and watch this season now that I know who won??? Doubtful. It's no wonder reality TV competition shows aren't on Netflix. But then what are the cord cutters who like The Bachelor and The Voice to do?!

Enter Tablo.

Basically it records OTA TV. With a plex plugin, it should be a snap to watch current shows both at home and remotely.

Since i don't actually have the device yet, (it's on the way) i'll be doing a follow up piece next week.

Why another timesheet app?

There are a lot of apps and websites that help you keep track of time.

For most of these the use cases vary slightly. Many are for an independent contractor wanting to keep track of their time so that they can properly charge their clients when the job has finished. Some are more of a employee manager relationship which is where mine fits in.

One of the apps I've seen is called FotoPunch. FotoPunch looks very professional and well developed. It also costs a lot. Timesheet by TengenT LLC is catered to the average small business owner. The point is to save money and find a happy medium between manual labor to enter the information and an automated system that doesn't cost a lot.

Timesheet focuses on the most important features.

1) Manage time accurately.

• Employees use the simple UI and can the system takes care of recording precision.

2) Data in usable format

• The app makes it easy to export a timesheet between any date range. It auto calculates and summarizes the most important information.

3) Built in safeguards

• The system automatically checks for inconsistencies and flags these behaviors. For example, if the employee should clock in and out in the same place, the shift will be flagged if this does not happen.
• 
  

4) Cost effective

• As mentioned above, the focus is to make this a more cost effective option. So it will be priced below other alternatives.

Who would benefit from this app? What features would you want to see?

How to backup

Every once in a while I have a friend or family member that asks me if I know how and would be willing to retrieve files off their hard drive that just crashed. While it is possible, it's very time consuming and almost never recovers the files that they actually want back. While a good external hard drive is probably the easiest solution (especially if you have a tool such as Time Machine for Mac or something comparable for Windows) there are other more elegant and free solutions. The main deciding factor is the media type.

Documents 

I want all my files to be saved locally so that I can access them while offline. Using Google Drive and Dropbox is the ideal solution. Instead of putting school assignments into the Documents folder on my Mac, I save them to a Documents folder inside my Google Drive folder. Since Google drive syncs all content to the cloud it's always backed up.  I prefer google drive to dropbox simply because it has more space (15GB instead of 2GB). With 15 GB of free storage, as long as it's just documents, I always have room and collaboration/sharing is already integrated.

Pictures

For those with Android, Google+ is a great app that has auto backup and unlimited storage. Any pictures uploaded through the app will not count against your 15GB so long as they are 2048px. Pictures can be set to private by default which is great considering Google+ is a social platform. You can view and download them any time via the app or your Google Plus page. As an extra your photos are automatically enhanced using Google's photo algorithms that they revealed last year at Google IO and duplicates are merged into fun gifs known as Auto Awesome.




For those with tons of pics that just want a place to dump them all at full res, check out Flickr. Signing up gives you 1TB of free photo storage. Make sure that any uploads are marked as private as that may not be default.

Music 

Google comes to the rescue again with Google Play music. You can now as of Feb 26, 2015 upload 50,000 songs to your own personal music library. Here are some of the extra perks for uploading to the cloud:

1) Stream using the Google play mobile app, chrome extension, browser or other 3rd party apps such as "Gear for Google Play and YouTube" 

2) Automatic organizing that happens upon upload. Your music is analyzed much like the way Shazam works and metadata is automatically added so that your content is organized and can be searched by album, artist, and anything else. 

3) Google play often gives away free top name albums as promotional items that can added to your library with just a click.

Videos

For home movies, Flickr may be a good choice. For movies, you're probably best getting a hard drive and setting up a plex server.

So before you think about getting a hard drive consider what you're really trying to do. It may be easier to use the cloud. What kinds of data do you need to backup? Is there any great service you know of that I should try out?

Rasperry pi.

In honor of the recent Pi day, I'll write about my neglected Raspberry Pi.

The Raspberry Pi is a $35 mini computer. If you kept it running all year it would use about $2 of electricity. Because it uses an ARM processor it's operating system capabilities are mostly limited to Linux. While I have no problem with Linux, this product falls under the cool, but not so useful category. I have attempted on many occasions to try and find a use for it, but each time i come back to the fact that either 1) I already have a fully fledged Ubuntu server running out of my house which can do this better or 2) it would cost a lot more money to buy all the extra equipment need to make something like a Power cat feeder.

The benefits of the Raspberry Pi are that it's cheap and portable. To build anything else means I would be better off doing it with something else.

Until I think of an idea, here's to hoping my Pi won't go on (unused) forever.

I know there are a lot of ideas online, but what are some of yours? What would you build with a Raspberry Pi if you could?