Mastering Stop Motion Through Machine Learning

September 20, 2021 by Tom Nardi 13 Comments

Stop motion animation is notoriously difficult to pull off well, in large part because it’s a mind-numbingly slow process. Each frame in the final video is a separate photograph, and for each one of those, the characters and props need to be moved the appropriate amount so that the final result looks smooth. You don’t even want to know how long Ben Wyatt spent working on Requiem for a Tuesday, though to be fair, it might still get done before the next Avatar.

But [Nick Bild] thinks his latest project might be able to improve on the classic technique with a dash of artificial intelligence provided by a Jetson Xavier NX. Basically, the Jetson watches the live feed from the camera, and using a hand pose detection model, waits until there’s no human hand in the frame. Once the coast is clear, it takes a shot and then goes back to waiting for the next hands-free opportunity. With the photographs being taken automatically, you’re free to focus on getting your characters moving around in a convincing way.

If it’s still not clicking for you, check out the video below. [Nick] first shows the raw unedited video, which primarily consists of him moving three LEGO figures around, and then the final product produced by his system. All the images of him fiddling with the scene have been automatically trimmed, leaving behind a short animated clip of the characters moving on their own.

Now don’t be fooled, it’s still going to take awhile. By our count, it took two solid minutes of moving around Minifigs to produce just a few seconds of animation. So while we can say its a quicker pace than with traditional stop motion production, it certainly isn’t fast.

Machine learning isn’t the only modern technology that can simplify stop motion production. We’ve seen a few examples of using 3D printed objects instead of manually-adjusted figures. It still takes a long time to print, and of course it eats up a ton of filament, but the mechanical precision of the printed scenes makes for a very clean final result.

Continue reading “Mastering Stop Motion Through Machine Learning” →

Flamethrower weedkiller mounted on a robot arm riding a tank tracked base

Don’t Sleep On The Lawn, There’s An AI-Powered, Flamethrower-Wielding Robot About

September 11, 2021 by Dave Rowntree 25 Comments

You know how it goes, you’re just hanging out in the yard, there aren’t enough hours in the day, and weeding the lawn is just such a drag. Then an idea just pops into your head. How about we attach a gas powered flamethrower to a robot arm, drive it around on a tank-tracked robotic base, and have it operate autonomously with an AI brain? Yes, that sounds like a good idea. Let’s do that. And so, [Dave Niewinski] did exactly that with his Ultimate Weed Killing Robot.

And you thought the robot overlords might take a more subtle approach and take over the world one coffee machine at a time? No, straight for the fully-autonomous flamethrower it is then.

This build uses a Kinova Robots Gen 3 six-axis arm, mounted to an Agile-X Robotics Bunker base. Control is via a Connect Tech Rudi-NX box which contains an Nvidia Jetson Xavier NX Edge AI computing engine. Wow that was a mouthful!

Connectivity from the controller to the base is via CAN bus, but, sadly no mention of how the robot arm controller is hooked up. At least this particular model sports an effector mount camera system, which can feed straight into the Jetson, simplifying the build somewhat.

To start the software side of things, [Dave] took a video using his mobile phone while walking his lawn. Next he used RoboFlow to highlight image stills containing weeds, which were in turn used to help train a vision AI system. The actual AI training was written in Python using Google Collaboratory, which is itself based on the awesome Jupyter Notebook (see also Jupyter Lab on the main site. If you haven’t tried that yet, and if you do any data science at all, you’ll kick yourself for not doing so!) Collaboratory would not be all that useful for this by itself, except that it gives you direct, free GPU access, via the cloud, so you can use it for AI workloads without needing fancy (and currently hard to get) GPU hardware on your desk.

Details of the hardware may be a little sparse, but at least the software required can be found on the WeedBot GitHub. It’s not like most of us will have this exact hardware lying around anyway. For a more complete description of this terrifying contraption, checkout the video after the break.

Continue reading “Don’t Sleep On The Lawn, There’s An AI-Powered, Flamethrower-Wielding Robot About” →

Cheat At Cornhole With A Bazillion-Dollar Robot

April 4, 2021 by Bryan Cockfield 19 Comments

While the days of outdoor cookouts may be a few months away for most of us, that certainly leaves plenty of time to prepare for that moment. While some may spend that time perfecting recipies or doing various home improvement projects during their remaining isolation time, others are practicing their skills at the various games played at these events. Specifically, this group from [Dave’s Armory] which have trained a robot that helps play the perfect game of cornhole. (Video, embedded below.)

While the robot in question is an industrial-grade KUKA KR-20 robot with a hefty price tag of $32,000 USD, the software and control system that the group built are fairly accessible for most people. The computer vision is handled by an Nvidia Jetson board, a single-board computer with extra parallel computing abilities, which runs OpenCV. With this setup and a custom hand for holding the corn bags, as well as a decent amount of training, the software is easily able to identify the cornhole board and instruct the robot to play a perfect game.

While we don’t all have expensive industrial robots sitting around in our junk drawer, the use of OpenCV and an accessible computer might make this project a useful introduction to anyone interested in computer vision, and the group made the code public on their GitHub page. OpenCV can be used for a lot of other things besides robotics as well, such as identifying weeds in a field or using a Raspberry Pi for facial recognition.

Continue reading “Cheat At Cornhole With A Bazillion-Dollar Robot” →

Really Useful Robot

November 12, 2020 by Bryan Cockfield 8 Comments

[James Bruton] is an impressive roboticist, building all kinds of robots from tracked, exploring robots to Boston Dynamics-esque legged robots. However, many of the robots are proof-of-concept builds that explore machine learning, computer vision, or unique movements and characteristics. This latest build make use of everything he’s learned from building those but strives to be useful on a day-to-day basis as well, and is part of the beginning of a series he is doing on building a Really Useful Robot. (Video, embedded below.)

While the robot isn’t quite finished yet, his first video in this series explores the idea behind the build and the construction of the base of the robot itself. He wants this robot to be able to navigate its environment but also carry out instructions such as retrieving a small object from a table. For that it needs a heavy base which is built from large 3D-printed panels with two brushless motors with encoders for driving the custom wheels, along with a suspension built from casters and a special hinge. Also included in the base is an Nvidia Jetson for running the robot, and also handling some heavy lifting tasks such as image recognition.

As of this writing, [James] has also released his second video in the series which goes into detail about the mapping and navigation functions of the robots, and we’re excited to see the finished product. Of course, if you want to see some of [James]’s other projects be sure to check out his tracked rover or his investigations into legged robots.

Continue reading “Really Useful Robot” →

Garbage Can Takes Itself Out

September 10, 2020 by Bryan Cockfield 12 Comments

Home automation is a fine goal but typically remains confined to lights, blinds, and other things that are relatively stationary and/or electrical in nature. There is a challenge there to be certain, but to really step up your home automation game you’ll need to think outside the box. This automated garbage can that can take itself out, for example, has all the home automation street cred you’d ever need.

The garbage can moves itself by means of a scooter wheel which has a hub motor inside and is powered by a lithium battery, but the real genius of this project is the electronics controlling everything. A Raspberry Pi Zero W is at the center of the build which controls the motor via a driver board and also receives instructions on when to wheel the garbage can out to the curb from an Nvidia Jetson board. That board is needed because the creator, [Ahad Cove], didn’t want to be bothered to tell his garbage can to take itself out or even schedule it. He instead used machine learning to detect when the garbage truck was headed down the street and instruct the garbage can to roll itself out then.

The only other thing to tie this build together was to get the garage door to open automatically for the garbage can. Luckily, [Ahad]’s garage door opener was already equipped with WiFi and had an available app, unbeknownst to him, which made this a surprisingly easy part of the build. If you have a more rudimentary garage door opener, though, there are plenty of options available to get it on the internets.

Continue reading “Garbage Can Takes Itself Out” →

Open Source Kitchen Helps You Watch What You Eat

November 20, 2019 by Gerrit Coetzee 6 Comments

Every appliance business wants to be the one that invents the patented, license-able, and profitable standard that all the other companies have to use. Open Source Kitchen wants to beat them to it.

Every beginning standard needs a test case, and OSK’s is a simple one. A bowl that tracks what you eat. While a simple concept, the way in which the data is shared, tracked, logged, and communicated is the real goal.

The current demo uses a Nvidia Jetson Nano as its processing center. This $100 US board packs a bit of a punch in its weight class. It processes the video from a camera held above the bowl of fruit, suspended by a scale in a squirrel shaped hangar, determining the calories in and calories out.

It’s an interesting idea. One wonders how the IoT boom might have played out if there had been a widespread standard ready to go before people started walling their gardens.

LCD Panel Lamp Shade Makes For Eye-Catching Lighting

November 11, 2019 by Jenny List 14 Comments

At first sight, [Kyle]’s Elroy lamp is simply an attractive piece of modern-styled interior furnishing; its clean lines, wood grain, and contemporary patterning being an asset to the room. But when he pulls out his phone, things change. Because this lamp hides a secret: at its heart may be a standard LED bulb, but the shade conceals four LCD screens driven by an Nvidia Jetson. These can be controlled through a web app to display a variety of textures, completing the effect.

This is not however simply a set of laptop screens bolted to a lampshade. The screens started life in laptops sure enough, but have since had their reflective backing removed to create a transparent LCD panel. Then an appropriate diffuser had to be found, which after much experimentation became a composite including more than one textured paper. Finally the whole was enclosed in an attractive wooden lamp frame and became part of the furniture. We like it, both as an aesthetically pleasing lamp and as a genuine departure from the norm.

This isn’t the first eye-catching lampshade we’ve brought you, but it’s certainly raised the bar. You can see it in action in the video below the break.

Continue reading “LCD Panel Lamp Shade Makes For Eye-Catching Lighting” →