Audio Fingerprinting Skips A Show’s Intro, Reliably

November 25, 2020 by Donald Papp 23 Comments

Lacking a DVD drive, [jg] was watching a TV series in the form of a bunch of .avi video files. Of course, when every episode contains a full intro, it is only a matter of time before that gets too annoying to sit through.

Chapter breaks reliably inserted around the intro, even when it doesn’t always occur in the same place.

The usual method of skipping the intro on a plain video file is a simple one:

Manually drag the playback forward past the intro.
Oops that’s too far, bring it back.
Ugh reversed it too much, nudge it forward.
Okay, that’s good.

[jg] was certain there was a better way, and the solution was using audio fingerprinting to insert chapter breaks. The plain video files now have a chapter breaks around the intro, allowing for easy skipping straight to content. The reason behind selecting this method is simple: the show intro is always 52 seconds long, but it isn’t always in the same place. The intro plays somewhere within the first two to five minutes of an episode, so just skipping to a specific timestamp won’t do the trick.

The first job is to extract the audio of an intro sequence, so that it can be used for fingerprinting. Exporting the first 15 minutes of audio with ffmpeg easily creates a wav file that can be trimmed down with an audio editor of choice. That clip gets fed into the open-source SoundFingerprinting library as a signature, then each video has its audio track exported and the signature gets identified within it. SoundFingerprinting therefore detects where (down to the second) the intro exists within each video file.

Marking out chapter breaks using that information is conceptually simple, but ends up being a bit roundabout because it seems .avi files don’t have a simple way to encode chapters. However, .mkv files are another matter. To get around this, [jg] first converts each .avi to .mkv using ffmpeg then splices in the chapter breaks with mkvmerge. One important element is that the reformatting between .avi and .mkv is done without completely re-encoding the video itself, so it’s a quick process. The result is a bunch of .mkv files with chapter breaks around the intro, wherever it may be!

The script is available here for anyone to play with, and the project page is a good learning reference because [jg] kindly provides all the command-line options used for each tool. Interested in using audio fingerprinting in your own projects? Remember to also check out Olaf, the Overly Lightweight Acoustic Fingerprinting method that can be implemented in embedded systems and web browsers.

Tracking Drone Flight Path Via Video, Using Cameras We Can Get

November 13, 2020 by Roger Cheng 4 Comments

Calculating three-dimensional position from two-dimensional projections are literal textbook examples in geometry, but those examples are the “assume a spherical cow” type of simplifications. Applicable only in an ideal world where the projections are made with mathematically perfect cameras at precisely known locations with infinite resolution. Making things work in the real world is a lot harder. But not only have [Jingtong Li, Jesse Murray et al.] worked through the math of tracking a drone’s 3D flight from 2D video, they’ve released their MultiViewUnsynch software on GitHub so we can all play with it.

Instead of laboratory grade optical instruments, the cameras used in these experiments are available at our local consumer electronics store. A table in their paper Reconstruction of 3D Flight Trajectories from Ad-Hoc Camera Networks (arXiv:2003.04784) listed several Huawei cell phone cameras, a few Sony digital cameras, and a GoPro 3. Video cameras don’t need to be placed in any particular arrangement, because positions are calculated from their video footage. Correlating overlapping footage from dissimilar cameras is a challenge all in itself, since these cameras record at varying framerates ranging from 25 to 59.94 frames per second. Furthermore, these cameras all have rolling shutters, which adds an extra variable as scanlines in a frame are taken at slightly different times. This is not an easy problem.

There is a lot of interest in tracking drone flights, especially those flying where they are not welcome. And not everyone have the budget for high-end equipment or the permission to emit electromagnetic signals. MultiViewUnsynch is not quite there yet, as it tracks a single target and video files were processed afterwards. The eventual goal is to evolve this capability to track multiple targets on live video, and hopefully help reduce frustrating public embarrassments.

[IROS 2020 Presentation video (duration 14:45) requires free registration, available until at least Nov. 25th 2020.]

Super-Simple VGA Adapter Sports Low-Res Output With Only Four TTL Chips

October 24, 2020 by Dan Maloney 20 Comments

Here at Hackaday we cast a wary eye at tips that come in with superlative claims. Generally, if we post something that claims to be the fastest or the smallest of all time, we immediately get slapped down in the comments by someone who has done it faster or smaller. So we present the simplest TTL video card ever knowing the same thing will happen, but eager to see how anyone might scale things down.

To be fair, [George Foot] does qualify his claim to the simplest usable VGA adapter, and he does note that it descends from [Ben Eater]’s “world’s worst video card”, which he uses for his 6502 breadboard computer. But where [Ben]’s VGA adapter uses about 20 TTL chips and an EEPROM, [George] has managed to decrease the BOM to just four TTL chips along with the memory and a crystal oscillator. This required a fair number of compromises, of course; the color depth is fairly low, as is the resolution. Each pixel appears as a thin horizontal bar rather than a small square, leading the images to be smeared out across the screen. They’re still surprisingly viewable, though, which probably says more about the quality of the pattern-recognition wetware between our ears than anything about the quality of the adapter. [George] gives a tour of the circuit in the brief video below.

It looks like [George] has posted a few improvements to the project since we first spotted it, so we’re looking forward to seeing how much the parts count went up. We’re also keen to see if anyone can outdo the simplicity of this effort — be sure to let us know if you give it a shot.

Continue reading “Super-Simple VGA Adapter Sports Low-Res Output With Only Four TTL Chips” →

How To Shoot A Great Project Video

October 9, 2020 by Lewin Day 29 Comments

Creating projects is fun, but the real value, as far as the imaginary Internet points are concerned, is how well you show them off for the clout. Taking a few snaps is fine, but if you want to produce a quality video of your project, it pays to put some thought and effort into the process.

Telling The Story

Before setting out to document your project on video, think about what you’re trying to communicate to the viewer. Are you attempting to create a step-by-step guide on how to recreate what you’ve done, or are you simply trying to show off the awesome finished product? These are two very different types of video, and will require different content and delivery entirely. It also guides how you shoot your video.

If you want to show off your build process, you’ll need to shoot as you go. This can be time consuming, but also a great way to show the reality of what goes into your work. I always like it when people convey the pitfalls and successes they faced along the way, and get people involved in the story. It also means that you’ll end up shooting a lot of footage and you’ll spend plenty of time editing it all together. Continue reading “How To Shoot A Great Project Video” →

Boost Your Animation To 60 FPS Using AI

September 20, 2020 by Bryan Cockfield 43 Comments

The uses of artificial intelligence and machine learning continue to expand, with one of the more recent implementations being video processing. A new method can “fill in” frames to smooth out the appearance of the video, which [LegoEddy] was able to use this in one of his animated LEGO movies with some astonishing results.

His original animation of LEGO figures and sets was created at 15 frames per second. As an animator, he notes that it’s orders of magnitude more difficult to get more frames than this with traditional methods, at least in his studio. This is where the artificial intelligence comes in. The program is able to interpolate between frames and create more frames to fill the spaces between the original. This allowed [LegoEddy] to increase his frame rate from 15 fps to 60 fps without having to actually create the additional frames.

While we’ve seen AI create art before, the improvement on traditionally produced video is a dramatic advancement. Especially since the AI is aware of depth and preserves information about the distance of objects from the camera. The software is also free, runs on any computer with an appropriate graphics card, and is available on GitHub.

Continue reading “Boost Your Animation To 60 FPS Using AI” →

Commodore 64 Mini Man Makes Matching Mini Monitor

September 17, 2020 by Bryan Cockfield 8 Comments

While putting together a retro computer is a great project and can teach a lot about the inner workings of electronics, hooking that 70s- or 80s-era machine up to a modern 144 Hz 1440p display tends to be a little bit anticlimactic. To really recreate the true 8-bit experience it’s important to get a CRT display of some sort, but those are in short supply now as most are in a landfill somewhere now. [Tony] decided to create a hybrid solution of sorts by 3D printing his own Commodore replica monitor for that true nostalgia feel.

This build is a matching mini scale replica of the Commodore 1702 monitor, a color monitor produced by Commodore specifically for their machines. At the time it was top-of-the-line and even included an early predecessor of the S-Video method of video signalling. This monitor was modeled in Fusion 360 and then sent to the 3D printer for assembly, then populated with a screen with a period-correct 4:3 aspect ratio, required electronics for handling the Commodore’s video signal, and even includes an upgrade over the original monitor: stereo speakers instead of the single-channel speaker that was featured in the 80s.

While this monitor doesn’t use a CRT, it’s an impressive replica nonetheless, right down to the Commodore serial number sticker on the back. If you need a Commodore 64 to go along with it, there are plenty of possibilities available to consider like this emulated C64 on a Raspberry Pi or these refurbished OEM Commodores.

Continue reading “Commodore 64 Mini Man Makes Matching Mini Monitor” →

ESPFLIX Brings Streaming Video To The World Of Microcontrollers

August 11, 2020 by Lewin Day 9 Comments

These days, if you’ve got a TV that’s a little too old to directly access streaming services, you’ve got plenty of options. Apple TV, Chromecast, and a cavalcade of Android boxes are available to help get content on your screen. However, if you’re really stuck in the past, ESPFLIX might just be for you.

Control of the system is achieved by an Apple TV remote.

Yes, that’s right – it’s an online streaming service running on an ESP32. [rossumur] has achieved this feat through a careful use of codecs, and some efficient coding strategies to make it all come together. Video is MPEG1, at just 352×192 resolution. Audio is via the SBC codec, originally intended for use with Bluetooth devices. It’s chosen here for its tiny sample buffers, making it easier to decode in the limited RAM of the ESP32. Output is via composite video, generated on the ESP32 itself.

The titles themselves consist of public domain content, running off an Amazon Web Services instance. With limited RAM on the ESP32, there’s not much buffering to be had, so [rossumur] is bankrolling an AWS Cloudfront instance which should make it possible to use ESPFLIX from most places around the world with a solid internet connection.

We’ve seen [rossumur]’s work before, with the ESP_8_BIT serving as a prelude to this project’s capabilities. Video after the break.

Continue reading “ESPFLIX Brings Streaming Video To The World Of Microcontrollers” →