Audio Fingerprinting Skips A Show’s Intro, Reliably

Lacking a DVD drive, [jg] was watching a TV series in the form of a bunch of .avi video files. Of course, when every episode contains a full intro, it is only a matter of time before that gets too annoying to sit through.

Chapter breaks reliably inserted around the intro, even when it doesn’t always occur in the same place.

The usual method of skipping the intro on a plain video file is a simple one:

  1. Manually drag the playback forward past the intro.
  2. Oops that’s too far, bring it back.
  3. Ugh reversed it too much, nudge it forward.
  4. Okay, that’s good.

[jg] was certain there was a better way, and the solution was using audio fingerprinting to insert chapter breaks. The plain video files now have a chapter breaks around the intro, allowing for easy skipping straight to content. The reason behind selecting this method is simple: the show intro is always 52 seconds long, but it isn’t always in the same place. The intro plays somewhere within the first two to five minutes of an episode, so just skipping to a specific timestamp won’t do the trick.

The first job is to extract the audio of an intro sequence, so that it can be used for fingerprinting. Exporting the first 15 minutes of audio with ffmpeg easily creates a wav file that can be trimmed down with an audio editor of choice. That clip gets fed into the open-source SoundFingerprinting library as a signature, then each video has its audio track exported and the signature gets identified within it. SoundFingerprinting therefore detects where (down to the second) the intro exists within each video file.

Marking out chapter breaks using that information is conceptually simple, but ends up being a bit roundabout because it seems .avi files don’t have a simple way to encode chapters. However, .mkv files are another matter. To get around this, [jg] first converts each .avi to .mkv using ffmpeg then splices in the chapter breaks with mkvmerge. One important element is that the reformatting between .avi and .mkv is done without completely re-encoding the video itself, so it’s a quick process.┬áThe result is a bunch of .mkv files with chapter breaks around the intro, wherever it may be!

The script is available here for anyone to play with, and the project page is a good learning reference because [jg] kindly provides all the command-line options used for each tool. Interested in using audio fingerprinting in your own projects? Remember to also check out Olaf, the Overly Lightweight Acoustic Fingerprinting method that can be implemented in embedded systems and web browsers.

Olaf Lets An ESP32 Listen To The Music

The joys of overengineering a simple gift. [Joren] wanted to create a dress for his daughter’s fourth birthday that would react with lights in sequence for a song from Frozen. The dress and an LED strip, along with a digital microphone and a battery were easy to procure. But how to make it all work? An ESP32 did the trick.

While the project’s name–Olaf–sounds like it was from Frozen, according to the GitHub page it actually means Overly Lightweight Acoustic Fingerprinting. Right. However, as the name implies, it can learn to identify any sound you want.

Continue reading “Olaf Lets An ESP32 Listen To The Music”

3D Printing, Cybersecurity, And Audio Fingerprinting

We all understand the risk of someone taking over our computers or phones for nefarious purposes. But remote access to printers and fax machines was something most people took a little less seriously. After all, you might get some obscene printouts or someone wasting some paper, but in general, those are not big deals. Some researchers however have lately been pondering what might happen should someone break into your 3D printer. Of course, you could bring a printer down to deny service, or cause things to malfunction — maybe even in ways that could be dangerous if the printer didn’t have sufficient safety features. But these researchers are more crafty. They are studying how you know what you’ve printed hasn’t been subtly sabotaged. They also think they have an answer.

If you are printing another Benchy at home this probably isn’t a real concern. However, according to the paper, 3D printing now accounts for over $6 billion of revenue with 33.8% of all parts having some function. This includes a recent FAA approval for a 3D-printed fuel nozzle for a jet engine. So indulge us in a little science fiction. You are about to fly your drone to take video of an important social function. You are worried about one of your props, so you 3D print a new one. Too bad your competitor has hacked your computer with a phishing e-mail and modified your STL files so that the new prop will have built-in weak spots internally. The prop will look fine and you’ll be able to install it. But it is going to fail right when you are taking those critical shots.

Continue reading “3D Printing, Cybersecurity, And Audio Fingerprinting”