Decently useful AI has been around for a little while now, and robotic arms have been around much longer. Yet somehow, we don’t have little robot helpers on our desks yet! Thankfully, [Yifei] is working towards that reality with Tabletop Handybot.
What [Yifei] has developed is a robotic arm that accepts voice commands. The robot relies on a Realsense D435 RGB-D camera, which provides color vision with depth information as well. Grounding DINO is used for object detection on the RGB images. Segment Anything and Open3D are used for further processing of the visual and depth data to help the robot understand what it’s looking at. Meanwhile, voice commands are interpreted via OpenAI Whisper, which can feed prompts to ChatGPT for further processing.
[Yifei] demonstrates his robot picking up markers on command, which is a pretty cool demo. With so many modern AI tools available, we’re getting closer to the ideal of robots that can understand and execute on general spoken instructions. This is a great example. We may not be all the way there yet, but perhaps soon. Video after the break.
call me when it can sort screws :-)
Dummy mk1?
Fire extinguisher is missing! Upgrade needed!
“Powered by AI” is rapidly becoming more meaningless as “powered by electricity” and far more repulsive. At least tell us what it can do in the title.
Pessimists: AI will take over the world and destroy mankind.
Optimists: AI will help us find novel cures to cancers and other fatal diseases.
Realists: AI turns depressive 1990s psycho rap into cheerful disco-polo song or Pope’s John Paul II speech.
https://www.youtube.com/watch?v=7mhaK-p8onM
https://www.youtube.com/watch?v=9UJmIdV5Dxk
(src: https://www.youtube.com/watch?v=u3HeJFr01T0 )
Was this post written by an AI? Because it doesn’t seem to have a point.
Come back when you start seeing industrial robots everywhere…
https://www.youtube.com/watch?v=7f2wg1pqQDs
Useful as “Intel Inside”.
Agreed. Buzzwords are meaningless.
What is my purpose?
You pass butter.
Oh. My. God.
Oh my. Bulding robotics with depencies to some Cloud LLM service is a big big NO-GO for me as it should be with anybody.
Beyond the simple network outage that leaves the robot brain-dead, to malicious data injection in the LLM that potentially could trigger unwanted effects / reactions of the robot, to simply non- / hardly- reproducible results for industrial applications, there is just too much that can go wrong
To your point, I’d love to see some efforts to create an “AI Firewall” that can filter malicious behavior and prevent damage. I don’t know what that looks like, but a cool idea nonetheless . I do however think this is an instance where “we” decide that the reward is worth the risk until someone proves that the negative hypothetical is reality or until an actual local alternative is viable. That being said, the actual resource cost to run an LLM might be the best firewall at the moment.
What is the SBC which is used for the project?