A man’s hand is visible holding a large, potato-shaped object in the foreground. A short, white, cylindrical structure is on the top of the potato, with black wires bending back into the potato. A smaller rectangular structure is to one side of it, and a red alligator clip connects to a nail protruding from the potato.

Building A Potato-based GLaDOS As An Introduction To AI

Although not nearly as intimidating as her ceiling-mounted hanging arm body, GLaDOS spent a significant portion of the Portal 2 game in a stripped-down computer powered by a potato battery. [Dave] had already made a version of her original body, but it was built around a robotic arm that was too expensive for the project to be really accessible. For his latest project, therefore, he’s created a AI-powered version of GLaDOS’s potato-based incarnation, which also serves as a fun introduction to building AI systems.

[Dave] wanted the system to work offline, so he needed a computer powerful enough to run all of his software locally. He chose an Nvidia Jetson Orin Nano, which was powerful enough to run a workable software system, albeit slowly and with some memory limitations. A potato cell unfortunately doesn’t generate enough power to run a Jetson, and it would be difficult to find a potato large enough to fit the Jetson inside. Instead, [Dave] 3D-printed and painted a potato-shaped enclosure for the Jetson, a microphone, a speaker, and some supplemental electronics.

A large language model handles interactions with the user, but most models were too large to fit on the Jetson. [Dave] eventually selected Llama 3.2, and used LlamaIndex to preprocess information from the Portal wiki for retrieval-augmented generation. The model’s prompt was a bit difficult, but after contacting a prompt engineer, [Dave] managed to get it to respond to the hapless user in an appropriately acerbic manner. For speech generation, [Dave] used Piper after training it on audio files from the Portal wiki, and for speech recognition used Vosk (a good programming exercise, Vosk being, in his words, “somewhat documented”). He’s made all of the final code available on GitHub under the fitting name of PotatOS.

The end result is a handheld device that sarcastically insults anyone seeking its guidance. At least Dave had the good sense not to give this pernicious potato control over his home.

Star Wars Pit Droid Has A Jetson Brain

In the Star Wars universe, pit droids are little foldable robots that perform automated repairs on spacecraft and the like. They were introduced in 1999’s The Phantom Menace, and beyond the podracing scenes, are probably the only good thing to come out of that particular film.

[Goran Vuksic] wanted a pit droid of his own, and reasoned that if he was going to go through the trouble of sanding and painting all the 3D printed components so they look like the real bot, he might as well add some smarts to it. While this droid won’t be fixing podracers anytime soon, its onboard Jetson Orin Nano Developer Kit does pack a considerable amount of processing under that dome.

A webcam mounted in the bot’s eye socket is connected to the Jetson, which is running an image detection and identification routine based on the example code provided by NVIDIA. The single-board computer uses a relay to blink some LEDs on and off when a human is detected, and a pair of servos pan-and-tilt the bot’s head towards whoever has caught its gaze.

It’s no surprise that [Goran] picked the Jetson Orin over competing SBCs for this task — in our review of the Orin Nano Developer Kit a few months ago, we found it was able to hit nearly 200 frames per second while performing this sort of real-time image analysis. So there’s plenty of room to grow should he want to integrate more complex image recognition tasks.

For example, he could follow in the footsteps of [Kris Kersey], and put a functional data overlay on top of the video to give his bot Iron Man vision.

Continue reading “Star Wars Pit Droid Has A Jetson Brain”