Smart Camera Based On Google Coral

June 30, 2021 by Bryan Cockfield 8 Comments

As machine learning and artificial intelligence becomes more widespread, so do the number of platforms available for anyone looking to experiment with the technology. Much like the single board computer revolution of the last ten years, we’re currently seeing a similar revolution with the number of platforms available for machine learning. One of those is Google Coral, a set of hardware specifically designed to take advantage of this new technology. It’s missing support to work with certain hardware though, so [Ricardo] set out to get one working with a Raspberry Pi Zero with this smart camera build based around Google Coral.

The project uses a Google Coral Edge TPU with a USB accelerator as the basis for the machine learning. A complete image for the Pi Zero is available which sets most of the system up right away including headless operation and includes a host of machine learning software such as OpenCV and pytesseract. By pairing a camera to the Edge TPU and the Raspberry Pi, [Ricardo] demonstrates many of its machine learning capabilities with several example projects such as an automatic license plate detector and even a mode which can recognize whether or not a face mask is being worn, and even how correctly it is being worn.

For those who want to get into machine learning and artificial intelligence, this is a great introductory project since the cost to entry is so low using these pieces of hardware. All of the project code and examples are available on [Ricardo]’s GitHub page too. We could even imagine his license plate recognition software being used to augment this license plate reader which uses a much more powerful camera.

Teaching A Machine To Be Worse At A Video Game Than You Are

May 31, 2021 by Dan Maloney 6 Comments

Is it really cheating if the aimbot you’ve built plays the game worse than you do?

We vote no, and while we take a dim view on cheating in general, there are still some interesting hacks in this AI-powered bot for Valorant. This is a first-person shooter, team-based game that has a lot of action and a Counter-Strike vibe. As [River] points out, most cheat-bots have direct access to the memory of the computer which is playing the game, which gives it an unfair advantage over human players, who have to visually process the game field and make their moves in meatspace. To make the Valorant-bot more of a challenge, he decided to feed video of the game from one computer to another over an HDMI-to-USB capture device.

The second machine has a YOLOv5 model which was trained against two hours of gameplay, enough to identify friend from foe — most of the time. Navigation around the map was done by analyzing the game’s on-screen minimap with OpenCV and doing some rudimentary path-finding. Actually controlling the player on the game machine was particularly hacky; rather than rely on an API to send keyboard sequences, [River] used a wireless mouse dongle on the game machine and a USB transmitter on the second machine.

The results are — iffy, to say the least. The system tends to get the player stuck in corners, and doesn’t recognize enemies that pop up at close range. The former is a function of the low-res minimap, while the latter has to do with the training data set — most human players engage enemies at distance, so there’s a dearth of “bad breath range” encounters to train to. Still, we’re impressed that it’s possible to train a machine to play a complex FPS game at all, let alone this well.

Neural Networks Emulate Any Guitar Pedal For $120

May 30, 2021 by Adam Zeloof 62 Comments

It’s a well-established fact that a guitarist’s acumen can be accurately gauged by the size of their pedal board- the more stompboxes, the better the player. Why have one box that can do everything when you can have many that do just a few things?

Jokes aside, the idea of replacing an entire pedal collection with a single box is nothing new. Your standard, old-school stompbox is an analog affair, using a combination of filters and amplifiers to achieve a certain sound. Some modern multi-effects processors use software models of older pedals to replicate their sound. These digital pedals have been around since the 90s, but none have been quite like the NeuralPi project. Just released by [GuitarML], the NeuralPi takes about $120 of hardware (including — you guessed it — a Raspberry Pi) and transforms it into the perfect pedal.

The key here, of course, is neural networks. The LSTM at the core of NeuralPi can be trained on any pedal you’ve got laying around to accurately reproduce its sound, and it can even do so with incredibly low latency thanks to Elk Audio OS (which even powers Matt Bellamy’s synth guitar, as used in Muse‘s Simulation Theory World Tour). The result of a trained model is a VST3 plugin, a popular format for describing audio effects.

This isn’t the first time we’ve seen some seriously cool stuff from [GuitarML], and it also hearkens back a bit to some sweet pedal simulation in LTSpice we saw last year. We can’t wait to see this project continue to develop — over time, it would be awesome to see a slick UI, or maybe somebody will design a cool enclosure with some knobs and an honest-to-god pedal for user input!

Thanks to [Mish] for the tip!

Continue reading “Neural Networks Emulate Any Guitar Pedal For $120” →

Deep Learning Enables Intuitive Prosthetic Control

May 27, 2021 by Lewin Day 7 Comments

Prosthetic limbs have been slow to evolve from simple motionless replicas of human body parts to moving, active devices. A major part of this is that controlling the many joints of a prosthetic is no easy task. However, researchers have worked to simplify this task, by capturing nerve signals and allowing deep learning routines to figure the rest out.

The prosthetic arm under test actually carries a NVIDIA Jetson Nano onboard to run the AI nerve signal decoder algorithm.

Reported in a pre-published paper, researchers used implanted electrodes to capture signals from the median and ulnar nerves in the forearm of Shawn Findley, who had lost a hand to a machine shop accident 17 years prior. An AI decoder was then trained to decipher signals from the electrodes using an NVIDIA Titan X GPU.

With this done, the decoder model could then be run on a significantly more lightweight system consisting of an NVIDIA Jetson Nano, which is small enough to mount on a prosthetic itself. This allowed Findley to control a prosthetic hand by thought, without needing to be attached to any external equipment. The system also allowed for intuitive control of Far Cry 5, which sounds like a fun time as well.

The research is exciting, and yet another step towards full-function prosthetics becoming a reality. The key to the technology is that models can be trained on powerful hardware, but run on much lower-end single-board computers, avoiding the need for prosthetic users to carry around bulky hardware to make the nerve interface work. If it can be combined with a non-invasive nerve interface, expect this technology to explode in use around the world.

[Thanks to Brian Caulfield for the tip!]

Making Minty Fresh Music With Markov Chains: The After Eight Step Sequencer

May 18, 2021 by Adam Zeloof 7 Comments

Step sequencers are fantastic instruments, but they can be a little, well, repetitive. At it’s core, the step sequencer is a pretty simple device: it loops through a series of notes or phrases that are, well, sequentially ordered into steps. The operator can change the steps while the sequencer is looping, but it generally has a repetitive feel, as the musician isn’t likely to erase all of the steps and enter in an entirely new set between phrases.

Enter our old friend machine learning. If we introduce a certain variability on each step of the loop, the instrument can help the musician out a bit here, making the final product a bit more interesting. Such an instrument is exactly what [Charis Cat] set out to make when she created the After Eight Step Sequencer.

The After Eight is an eight-step sequencer that allows the artist to set each note with a series of potentiometers (which are, of course, housed in an After Eight mint tin). The potentiometers are read by an Arduino, which passes MIDI information to a computer running the popular music-oriented visual programming language Max MSP. The software uses a series of Markov Chains to augment the musician’s inputted series of notes, effectively working with the artist to create music. The result is a fantastic piece of music that’s different every time it’s performed. Make sure to check out the video at the end for a fantastic overview of the project (and to hear the After Eight in action, of course)!

[Charis Cat]’s wonderful creation reminds us of some the work [Sara Adkins] has done, blending human performance with complex algorithms. It’s exactly the kind of thing we love to see at Hackaday- the fusion of a musician’s artistic intent with the stochastic unpredictability of a machine learning system to produce something unique.

Thanks to [Chris] for the tip!

Continue reading “Making Minty Fresh Music With Markov Chains: The After Eight Step Sequencer” →

Thought Control Via Handwriting

May 16, 2021 by Al Williams 12 Comments

Computers haven’t done much for the quality of our already poor handwriting. However, a man paralyzed by an accident can now feed input into a computer by simply thinking about handwriting, thanks to work by Stanford University researchers. Compared to more cumbersome systems based on eye motion or breath, the handwriting technique enables entry at up to 90 characters a minute.

Currently, the feat requires a lab’s worth of equipment, but it could be made practical for everyday use with some additional work and — hopefully — less invasive sensors. In particular, the sensor used two microelectrode arrays in the precentral gyrus portion of the brain. When the subject thinks about writing, recognizable patterns appear in the collected data. The rest is just math and classification using a neural network.

If you want to try your hand at processing this kind of data and don’t have a set of electrodes to implant, you can download nearly eleven hours of data already recorded. The code is out there, too. What we’d really like to see is some easier way to grab the data to start with. That could be a real game-changer.

More traditional input methods using your mouth have been around for a long time. We’ve also looked at work that involves moving your head.

Mind-Controlled Flamethrower

May 12, 2021 by Bryan Cockfield 16 Comments

Mind control might seem like something out of a sci-fi show, but like the tablet computer, universal translator, or virtual reality device, is actually a technology that has made it into the real world. While these devices often requires on advanced and expensive equipment to interpret brain waves properly, with the right machine learning system it’s possible to do things like this mind-controlled flame thrower on a much smaller budget. (Video, embedded below.)

[Nathaniel F] was already experimenting with using brain-computer interfaces and machine learning, and wanted to see if he could build something practical combining these two technologies. Instead of turning to an EEG machine to read brain patterns, he picked up a much less expensive Mindflex and paired it with a machine learning system running TensorFlow to make up for some of its shortcomings. The processing is done by a Raspberry Pi 4, which sends commands to an Arduino to fire the flamethrower when it detects the proper thought patterns. Don’t forget the flamethrower part of this build either: it was designed and built entirely by [Nathanial F] as well using gas and an arc lighter.

While the build took many hours of training to gather the proper amount of data to build the neural network and works as the proof of concept he was hoping for, [Nathaniel F] notes that it could be improved by replacing the outdated Mindflex with a better EEG. For now though, we appreciate seeing sci-fi in the real world in projects like this, or in other mind-controlled projects like this one which converts a prosthetic arm into a mind-controlled music synthesizer.

Continue reading “Mind-Controlled Flamethrower” →