opencv

179 Articles

Smart Camera Based On Google Coral

June 30, 2021 by 8 Comments

As machine learning and artificial intelligence becomes more widespread, so do the number of platforms available for anyone looking to experiment with the technology. Much like the single board computer revolution of the last ten years, we’re currently seeing a similar revolution with the number of platforms available for machine learning. One of those is Google Coral, a set of hardware specifically designed to take advantage of this new technology. It’s missing support to work with certain hardware though, so [Ricardo] set out to get one working with a Raspberry Pi Zero with this smart camera build based around Google Coral.

The project uses a Google Coral Edge TPU with a USB accelerator as the basis for the machine learning. A complete image for the Pi Zero is available which sets most of the system up right away including headless operation and includes a host of machine learning software such as OpenCV and pytesseract. By pairing a camera to the Edge TPU and the Raspberry Pi, [Ricardo] demonstrates many of its machine learning capabilities with several example projects such as an automatic license plate detector and even a mode which can recognize whether or not a face mask is being worn, and even how correctly it is being worn.

For those who want to get into machine learning and artificial intelligence, this is a great introductory project since the cost to entry is so low using these pieces of hardware. All of the project code and examples are available on [Ricardo]’s GitHub page too. We could even imagine his license plate recognition software being used to augment this license plate reader which uses a much more powerful camera.

Auto-Aiming Nerf Gun To Give You The Edge In Battle

June 3, 2021 by 13 Comments

Ever wished for some robotic enhancements for your next nerf war? Well, it’s time to dig through the parts bin and build yourself a nerf gun with aimbot built right in, courtesy of [3Dprintedlife]. (Video, embedded below.)

The gun started with a design borrowed from [Captain Slug]’s awesome catalog of open source nerf guns. [3Dprintedlife] modified the design to include a two-axis gimbal between the lower and the upper, driven by a pair of stepper motors via an Arduino. For auto-aim, a camera module attached to a Raspberry Pi running OpenCV was added. When the user half-pressed the trigger, OpenCV will start tracking whatever was at the center of the frame and actively adjust the gimbal to keep the gun aimed at the object until the user fires. The trigger mechanism consists of a pair of microswitches that activate a servo to release the sear. It is also capable of tracking a moving target or any face that comes into view.

We think this is a really fun project, with a lot of things that can be learned in the process. Mount it on a remote control tank and you’d be able to wage some intense battles in your backyard. All the files are available on GitHub.

You are never too old for a good old nerf battle. Whether you want to be a sniper, a machine gunner, or a heavy weapons specialist, there’s a weapon to build for every role.

Adding A Laser Blaster To Classic Atari 2600 Games With Machine Vision

June 3, 2021 by 5 Comments

Remember the pistol controller for the original Atari 2600? No? Perhaps that’s because it never existed. But now that we’re living in the future, adding a pistol to the classic games of the 2600 is actually possible.

Possible, but not exactly easy. [Nick Bild]’s approach to the problem is based on machine vision, using an NVIDIA Xavier NX to run an Atari 2600 emulator. The game is projected on a wall, while a camera watches the game field. A toy pistol with a laser pointer attached to it blasts away at targets, while OpenCV is used to find the spots that have been hit by the laser. A Python program matches up the coordinates of the laser blasts with coordinates within the game, and then fires off a sequence of keyboard commands to fire the blasters in the game. Basically, the game plays itself based on where it sees the laser shots. You can check out the system in the video below.

[Nick Bild] had a busy weekend of hacking. This was the third project write-up he sent us, after his big-screen Arduboy build and his C64 smartwatch.

Continue reading “Adding A Laser Blaster To Classic Atari 2600 Games With Machine Vision” →

Teaching A Machine To Be Worse At A Video Game Than You Are

May 31, 2021 by 6 Comments

Is it really cheating if the aimbot you’ve built plays the game worse than you do?

We vote no, and while we take a dim view on cheating in general, there are still some interesting hacks in this AI-powered bot for Valorant. This is a first-person shooter, team-based game that has a lot of action and a Counter-Strike vibe. As [River] points out, most cheat-bots have direct access to the memory of the computer which is playing the game, which gives it an unfair advantage over human players, who have to visually process the game field and make their moves in meatspace. To make the Valorant-bot more of a challenge, he decided to feed video of the game from one computer to another over an HDMI-to-USB capture device.

The second machine has a YOLOv5 model which was trained against two hours of gameplay, enough to identify friend from foe — most of the time. Navigation around the map was done by analyzing the game’s on-screen minimap with OpenCV and doing some rudimentary path-finding. Actually controlling the player on the game machine was particularly hacky; rather than rely on an API to send keyboard sequences, [River] used a wireless mouse dongle on the game machine and a USB transmitter on the second machine.

The results are — iffy, to say the least. The system tends to get the player stuck in corners, and doesn’t recognize enemies that pop up at close range. The former is a function of the low-res minimap, while the latter has to do with the training data set — most human players engage enemies at distance, so there’s a dearth of “bad breath range” encounters to train to. Still, we’re impressed that it’s possible to train a machine to play a complex FPS game at all, let alone this well.

Guitar Hero Robot Actually Shreds

May 3, 2021 by 12 Comments

Once a popular craze, most of the public has sold or stashed away their plastic video game instruments and forgotten the likes of Guitar Hero and Rockband. Having never been quite satisfied with his scores, [Nick O’Hara] set out to create a robot that could play a Guitar Hero controller. It would be easy enough to use transistors to actuate the buttons or even just a Teensy to emulate a controller and have it play the perfect game, but [Nick] wanted to replicate what it was really like to play. So after burning out a fair number of solenoids (driving them over spec) and learning on his feet, [Nick] slowly began to dial in his robot, Jon Bot Jovi.

The brains of the bot are a Raspberry Pi running some OpenCV-based code that identifies blobs of different colors. The video feed comes from a PS2 via an HDMI capture card. Solenoids are driven via an 8 channel driver board, controlled by the Pi. While it missed a few notes here and there, we loved seeing the strumming solenoid whammy rapidly on the strummer. All in all, it’s a great project, and we love the design of the robot. Whether played by a robot, turned into a synthesizer, or recreated from toy pianos and mechanical keyboards, Guitar Hero controllers offer many hacking opportunities.

Continue reading “Guitar Hero Robot Actually Shreds” →

Putting Perseverance Rover’s View Into Satellite View Context

March 30, 2021 by 3 Comments

It’s always fun to look over aerial and satellite maps of places we know, seeing a perspective different from our usual ground level view. We lose that context when it’s a place we don’t know by heart. Such as, say, Mars. So [Matthew Earl] sought to give Perseverance rover’s landing video some context by projecting onto orbital imagery from ESA’s Mars Express. The resulting video (embedded below the break) is a fun watch alongside the technical writeup Reprojecting the Perseverance landing footage onto satellite imagery.

Some telemetry of rover position and orientation were transmitted live during the landing process, with the rest recorded and downloaded later. Surprisingly, none of that information was used for this project, which was based entirely on video pixels. This makes the results even more impressive and the techniques more widely applicable to other projects. The foundational piece is SIFT (Scale Invariant Feature Transform), which is one of many tools in the OpenCV toolbox. SIFT found correlations between Perseverance’s video frames and Mars Express orbital image, feeding into a processing pipeline written in Python for results rendered in Blender.

While many elements of this project sound enticing for applications in robot vision, there are a few challenges touched upon in the “Final Touches” section of the writeup. The falling heatshield interfered with automated tracking, implying this process will need help to properly understand dynamically changing environments. Furthermore, it does not seem to run fast enough for a robot’s real-time needs. But at first glance, these problems are not fundamental. They merely await some motivated people to tackle in the future.

This process bears some superficial similarities to projection mapping, which is a category of projects we’ve featured on these pages. Except everything is reversed (camera instead of video projector, etc.) making the math an entirely different can of worms. But if projection mapping sounds more to your interest, here is a starting point.

[via Dr. Tanya Harrison @TanyaOfMars]

Continue reading “Putting Perseverance Rover’s View Into Satellite View Context” →