Capture Device Firmware Hack Unlocks All The Pixels

April 14, 2020 by Tom Nardi 7 Comments

According to [Mike Walters], the Elgato Cam Link 4K is a great choice if you’re looking for a HDMI capture device that works under Linux. But the bad news is, it wouldn’t work with any of the video conferencing software he tried to use it with because they expect the video stream to be in a different pixel format. For most people, that would probably have been the end of the story. But you’re reading this on Hackaday, so obviously he didn’t give up without a fight.

Early on, [Mike] found there was a software workaround for this exact issue. The problem isn’t that the Elgato can’t generate the desired format, it’s that the video conferencing programs just don’t know how to ask it to switch modes. The software fix is to create a dummy Video4Linux device and use that to change the format in real-time using ffmpeg. It’s a clever trick if you’ve got a conference call coming up in a few minutes, but it does waste CPU resources and adds some unnecessary hoop jumping.

Putting the device into bootloader mode.

Inspired by the software fix, [Mike] wondered if there was a way he could simply force the Elgato to output video in the desire format by default. He found a firmware dump for the device online, and found where the pixel formats were referenced by searching for their names in ASCII with hexdump. Looking through the source for the Linux USB Video Class (UVC) driver, he was then able to determine what the full 16 byte sequence should be for each video mode was so he could zero out the unwanted ones. Then it was just a matter of flashing his modified firmware back to the hardware.

But there was a problem: with the modified firmware installed, the device stopped working. After investigating the obvious culprits, [Mike] broke out the oscilloscope and hooked it up to the Elgato’s flash chip. It turns out that due to a bug in the program he was using, the SPI erase commands weren’t getting sent during the flash. This lead to corrupted firmware which was keeping the Elgato from booting. After making a pull request with his fixes, the firmware flashed without incident and the capture device now does double-duty as a webcam when necessary.

We could certainly think of easier and quicker was to roll your own webcam, but we’re glad that [Mike] took the time to modify his Elgato Cam Link 4K and document it. It’s a fantastic example of practical firmware hacking, even if you’re not in the market for a new high-definition video conferencing rig.

Background Substitution, No Green Screen Required

April 10, 2020 by Dan Maloney 28 Comments

All this working from home that people have been doing has a natural but unintended consequence: revealing your dirty little domestic secrets on a video conference. Face time can come at a high price if the only room you have available for work is the bedroom, with piles of dirty laundry or perhaps the incriminating contents of one’s nightstand on full display for your coworkers.

There has to be a tech fix for this problem, and many of the commercial video conferencing platforms support virtual backgrounds. But [Florian Echtler] would rather air his dirty laundry than go near Zoom, so he built a machine-learning background substitution app that works with just about any video conferencing platform. Awkwardly dubbed DeepBackSub — he’s working on a better name — the system does the hard work of finding the person in the frame with Tensorflow Lite. After identifying everything in the frame that’s a person, OpenCV replaces everything that’s not with whatever you choose, and the modified scene is piped over a virtual video device to the videoconferencing software. He’s tested on Firefox, Skype, and guvcview so far, all running on Linux. The resolution and framerates are limited, but such is the cost of keeping your secrets and establishing a firm boundary between work life and home life.

[Florian] has taken the need for a green screen out of what’s formally known as chroma key compositing, which [Tom Scott] did a great primer on a few years back. A physical green screen is the traditional way to do this, but we honestly think this technique is great and can’t wait to try it out with our Hackaday colleagues at the weekly videoconference.

Adding RGB To A CRT

April 9, 2020 by Jenny List 30 Comments

There was a time when all TVs came with only an antenna socket on their backs, and bringing any form of video input to them meant dicing with live-chassis power supplies. Then sets with switch-mode supplies made delving into a CRT TV much safer, and we could bodge in composite video and even RGB sockets by tapping into their circuitry. For Europeans the arrival of the SCART socket gave us ready-made connectivity, but in the rest of the world there was still a need to break out the soldering iron for an RGB input. [Jacques Gagnon] is in Canada, and has treated us to a bit of old-school TV input hacking as he put an RGB socket on his JVC CRT set.

Earlier hacks had inventive incursions into discrete analogue circuitry, but on later sets such as this one the trick was to take advantage of the on-screen-display features. The signal processing chip would usually have an RGB input with a blanking input to turn of the picture during the OSD chip’s output. These could be readily hijacked to provide an RGB input, and this is the course taken here. We see a VGA socket on the rear panel going to a resistor network on a piece of protoboard stuck in a vacant space on the PCB, from which a set of lines then go to the signal processing chip. The result is a CRT gaming monitor for retro consoles, of the highest quality.

For those of us who cut our teeth on CRT TVs it’s always good to see a bit of TV hacking. It’s a mod we’ve seen before, too.

Velocity Based Training, With A Camera

March 8, 2020 by Jenny List 3 Comments

In the world of weight training, the buzzword of the moment is VBT, or Velocity Based Training. This involves sensors being used to measure the speed and position of a weight as it moves through each repetition, and thus provide instant feedback for the athlete and glean information from which they can work upon their training routine. Typically the sensors involved may be accelerometers, but [Kris] has taken a different tack using a webcam and machine vision to do the same job.

The barbell has a green disc attached to its end, and the software tracks it and measures the velocity. It issues a warning when the velocity of a repetition drops below a preset level, telling the athlete to stop their set before pushing themselves too far. Under the hood is a Python script and OpenCV, and the write-up in his GitHub repository takes us through its camera calibration to remove the effects of distortion, and set-up. All calibration of distances within the image is made through the known size of the green disc, allowing the software to accurately chart the distance through which it travels.

We’ve not seen a machine vision approach to weight training before, but we have seen one using accelerometers. Maybe this project will re-ignite interest in this field.

Recording Video In The Era Of CRTs: The Video Camera Tube

February 27, 2020 by Maya Posch 45 Comments

We have all watched videos of concerts and events dating back to the 1950s, but probably never really wondered how this was done. After all, recording moving images on film had been done since the late 19th century. Surely this is how it continued to be done until the invention of CCD image sensors in the 1980s? Nope.

Although film was still commonly used into the 1980s, with movies and even entire television series such as Star Trek: The Next Generation being recorded on film, the main weakness of film is the need to move the physical film around. Imagine the live video feed from the Moon in 1969 if only film-based video recorders had been a thing.

Let’s look at the video camera tube: the almost forgotten technology that enabled the broadcasting industry. Continue reading “Recording Video In The Era Of CRTs: The Video Camera Tube” →

Raspberry Pi Serves Up 24 Hour Simpsons Channel

February 7, 2020 by Tom Nardi 38 Comments

Commercial-free video on demand was every couch potato’s dream for decades, and now we’ve got it. But nostalgia has a funny way of making some folks miss the old days, even if we know it’s technically be a step backwards. Wanting to recreate the TV watching experience circa 1998, [probnot] has come up with a way to run his very own television channel.

With the Raspberry Pi and a digital modulator, he’s got the only house on the block that’s wired to show The Simpsons all day. He has absolutely no control over which episode plays next, he can’t pause it, and its in presented in standard definition (a nightmare for anyone who grew up in the Netflix era) but a familiar viewing experience for the rest of us.

The key to this project is the Channel Plus Model 3025 modulator. It takes the feed from the antenna and mixes in two composite video sources on user-defined channels. All [probnot] had to do was find a channel that wouldn’t interfere with any of the over-the-air stations. The modulator has been spliced into the house’s coax wiring, so any TV connected to the wall can get in on the action. There’s no special setup required: when he wants to watch The Simpsons he just tunes the nearest TV to the appropriate channel.

Providing the video for the modulator is a Raspberry Pi, specifically, the original model that featured composite video output. While the first generation Pi is a bit long in the tooth these days, playing standard definition video is certainly within its capabilities. With a USB flash drive filled with a few hundred episodes and a bit of scripting it’s able to deliver a never-ending stream direct from Springfield. There’s still that second channel available on the modulator as well, which we’re thinking could be perfect for Seinfeld or maybe The X-Files.

Interestingly, this isn’t the first time we’ve seen a Raspberry Pi used to provide a never-ending stream of The Simpsons. But compared to previous attempts which had to be directly connected to the TV, we like the idea of using the modulator and creating a more authentic experience.

Continue reading “Raspberry Pi Serves Up 24 Hour Simpsons Channel” →

Tricking The Brain Into Seeing Boosted Contrast In Stereo Imagery

February 1, 2020 by Donald Papp 20 Comments

Last year a team of researchers published a paper detailing a method of boosting visual contrast and image quality in stereoscopic displays. The method is called Dichoptic Contrast Enhancement (DiCE) and works by showing each eye a slightly different version of an image, tricking the brain into fusing the two views together in a way that boosts perceived image quality. This only works on stereoscopic displays like VR headsets, but it’s computationally simple and easily implemented. This trick could be used to offset some of the limitations of displays used in headsets, for example making them appear capable of deeper contrast levels than they can physically deliver. This is good, because higher contrasts are generally perceived as being more realistic and three-dimensional; important factors in VR headsets and other stereoscopic displays.

Stereoscopic vision works by having the brain fuse together what both eyes see, and this process is called binocular fusion. The small differences between what each eye sees mostly conveys a sense of depth to us, but DiCE uses some of the quirks of binocular fusion to trick the brain into perceiving enhanced contrast in the visuals. This perceived higher contrast in turn leads to a stronger sense of depth and overall image quality.

Example of DiCE-processed images, showing each eye a different dynamic contrast range. The result is greater perceived contrast and image quality when the brain fuses the two together.

To pull off this trick, DiCE displays a different contrast level to both eyes in a way designed to encourage the brain to fuse them together in a positive way. In short, using a separate and different dynamic contrast range for each eye yields an overall greater perceived contrast range in the fused image. That’s simple in theory, but in practice there were a number of problems to solve. Chief among them was the fact that if the difference between what each eyes sees is too great, the result is discomfort due to binocular rivalry. The hard scientific work behind DiCE came from experimentally determining sweet spots, and pre-computing filters independent of viewer and content so that it could be applied in real-time for a consistent result.

Things like this are reminders that we experience the world only through the filter of our senses, and our perception of reality has quirks that can be demonstrated by things like this project and other “sensory fusion” edge cases like the Thermal Grill Illusion, which we saw used as the basis for a replica of the Pain Box from Dune.

A short video overview of the method is embedded below, and a PDF of the publication can be downloaded for further reading. Want a more hands-on approach? The team even made a DiCE plugin (freely) available from the Unity asset store.

Continue reading “Tricking The Brain Into Seeing Boosted Contrast In Stereo Imagery” →