Re-imagining Telepresence With Humanoid Robots And VR Headsets

Don’t let the name of the Open-TeleVision project fool you; it’s a framework for improving telepresence and making robotic teleoperation far more intuitive than it otherwise would be. It accomplishes this in part by taking advantage of the remarkable technology packed into modern VR headsets like the Apple Vision Pro and Meta Quest. There are loads of videos on the project page, many of which demonstrate successful teleoperation across vast distances.

Teleoperation of robotic effectors typically takes some getting used to. The camera views are unusual, the limbs don’t move the same way arms do, and intuitive human things like looking around to get a sense of where everything is don’t translate well.

A stereo camera with gimbal streaming to a VR headset complete with head tracking seems like a very hackable design.

To address this, researches provided a user with a robot-mounted, real-time stereo video stream (through which the user can turn their head and look around normally) as well as mapping arm and hand movements to humanoid robotic counterparts. This provides the feedback to manipulate objects and perform tasks in a much more intuitive way. In short, when our eyes, bodies, and hands look and work more or less the way we expect, it turns out it’s far easier to perform tasks.

The research paper goes into detail about the different systems, but in essence, a stereo depth and RGB camera is perched with a 3D printed gimbal atop a humanoid robot frame like the Unitree H1 equipped with high dexterity hands. A VR headset takes care of displaying a real-time stereoscopic video stream and letting the user look around. Hand tracking for the user is mapped to the dexterous hands and fingers. This lets a person look at, manipulate, and handle things without in-depth training. Perhaps slower and more clumsily than they would like, but in an intuitive way all the same.

Interested in taking a closer look? The GitHub repository has the necessary code, and while most of us will never be mashing ADD TO CART on something like the Unitree H1, the reference design for a stereo camera streaming to a VR headset and mirroring head tracking with a two-motor gimbal looks like the sort of thing that would be useful for a telepresence project or two.

Continue reading “Re-imagining Telepresence With Humanoid Robots And VR Headsets”

Hackaday Links Column Banner

Hackaday Links: June 25, 2023

Is it really a dystopian future if the robots are radio-controlled? That’s what came to mind reading this article on a police robot out of Singapore, complete with a breathless headline invoking Black Mirror, which is now apparently the standard by which all dystopias are to be judged. Granted, the episode with the robo-dogs was pretty terrifying, but it seems like the Singapore Police Force has a way to go before getting to that level. The bot, which has been fielded at Changi Airport after extensive testing and seems to be completely remote-controlled, is little more than a beefy telepresence robot. At 5.5 feet (1.7 meters) tall, the bot isn’t terribly imposing, although it apparently has a mast that can be jacked up another couple of feet, plus there are lights, sirens, and speakers that can get the message across. Plus cameras, of course; there are always cameras. The idea is to provide extra eyes to supplement foot patrols, plus the potential to cordon off an incident until meatspace officers arrive. The buzzword game here is weak, though; there’s no mention of AI or machine learning at all. We have a feeling that when the robots finally rise up, ones like this will be left serving the drinks.

Continue reading “Hackaday Links: June 25, 2023”

This Horrifying Robot Is Here To Teach You A Lesson

No, despite what it might look like, this isn’t some early Halloween project. The creepy creation before you is actually a tongue-in-cheek “robot” created by the prolific [Nick Bild], a topical statement about companies asking their remote workers to come back into the office now that COVID-19 restrictions are being lifted. Why commute every day when this ultra realistic avatar can sit in for you?

OK, so maybe it’s not the most impressive humanoid creation to ever grace the pages of Hackaday. But if you’re looking to spin up a simple telepresence system, you could do worse than browsing through the Python source code [Nick] has provided. Using a Raspberry Pi 4, a webcam, and a microphone, his client-server architecture combines everything the bot sees and hears into a simple page that can be remotely accessed with a web browser.

Naturally this work from home (WFH) bot wouldn’t be much good if it was just a one-way street, so [Nick] has also added a loudspeaker that replays whatever he says on the client side. To prevent a feedback loop, his software includes a function that toggles which direction the audio stream goes in by passing the appropriate commands to the bot over SSH; a neat trick to keep in mind for your own, less nightmarish, creations.

If you’re looking for something a bit more capable and have some cardboard laying around, this DIY telepresence mount for your phone might be a good place to start.

Continue reading “This Horrifying Robot Is Here To Teach You A Lesson”

OPARP Telepresence Robot

[Erik Knutsson] is stuck inside with a bunch of robot parts, and we know what lies down that path. His Open Personal Assistant Robotic Platform aims to help out around the house with things like filling pet food bowls, but for now, he is taking one step at a time and working out the bugs before adding new features. Wise.

The build started with a narrow base, an underpowered RasPi, and a quiet speaker, but those were upgraded in turn. Right now, it is a personal assistant on wheels. Alexa was the first contender, but Mycroft is in the spotlight because it has more versatility. At first, the mobility was a humble web server with a D-pad, but now it leverages a distance sensor and vision, and can even follow you with a voice command.

The screen up top gives it a personable look, but it is slated to become a display for everything you’d want to see on your robot assistant, like weather, recipes, or a video chat that can walk around with you. [Erik] would like to make something that assists the elderly who might need help with chores and help connect people who are stuck inside like him.

Expressive robots have long since captured our attention and we’re nuts for privacy-centric personal assistants.

Continue reading “OPARP Telepresence Robot”

Telepresence Robot Navigates Upgrades

As time marches on and a good percentage of us are still isolating from society at large, the progress of technology isn’t kept as stagnant. Earlier this year we featured a project about a much-needed small telepresence robot with an exceptionally low barrier for entry, and with the progress of time it has received several upgrades and some crowdfunding, all while preserving its original intent of a simple and easily-operated way of keeping in contact with others.

The new robot is still based on the cardboard design that holds a smartphone and drives it around using a microcontroller platform, but thanks to its small size and low power requirement this seems to suit it nicely. Improvements over the original design include a more robust one-size-fits-all phone mount and a more refined cardboard body. Also, since the small size is a little bit of a downside when navigating anywhere that isn’t a desk or counter, the new version makes it easier to make modifications such as adding a pedestal which can elevate the phone and improve the experience of the remote driver. A number of other optional modifications are possible as well, including a grabbing arm.

While telepresence robots unfortunately are needed now more than ever, we are happy to see people like [Ross] take on projects like this which will hopefully help improve our shared situation by allowing us to have a more involved level of contact with people we would otherwise prefer to see in person. If you’d like to build your own without waiting on the crowdfunding, be sure to check out the original project we featured back in April.

Get Back Out There, Robotically

When interacting with reality at a distance is the best course of action, we turn to robots. Whether that’s exploring the surface of Venus, the depths of the ocean, or (for the time being) society at large, it’s often better to put a robot out there than an actual human being. We can’t all send robots to other planets, but we can easily get them in various other places with telepresence robots.

This tiny telepresence robot comes to us from [Ross] at [Crafty Robot] who is using their small Smartibot platform as a basis for this tiny robot. The smartibot drives an easily-created cardboard platform, complete with wheels, and trucks around a smartphone of some sort which handles the video and network capabilities. The robot can be viewed and controlled from any other computer using a suite of web applications that can be found on the project page.

The Smartibot platform is an inexpensive platform that we’ve seen do other things like drive an airship, and the creators are hoping that as many people as possible can get some use out of this quick-and-easy telepresence robot if they really need something like this right now. The kit seems like it would be useful for a lot of other fun projects as well.

Continue reading “Get Back Out There, Robotically”

Hackaday Links Column Banner

Hackaday Links: October 6, 2019

“If you or someone you love has been exposed to questionable quality electrolytic capacitors, you could be entitled to financial compensation.” Perhaps that’s not exactly the pitch behind this class action lawsuit against capacitor manufacturers, but it might as well be. The suit claims that the defendants, a group of capacitor manufacturers that includes Nichicon, Matsuo, ELNA, and Panasonic, “engaged in an unlawful conspiracy to fix, raise, maintain, or stabilize the prices of Capacitors.” Translation: if you bought capacitors between 2002 and 2014 from a distributor, you paid too much for them. The suit aims to recover a bunch of money from the defendants and divide it up between all the class members, so make sure you go back through all your receipts from Mouser and DigiKey over the last 17 years so you can file a claim that could be worth several dozen cents.

When are people going to learn that posting pictures of their illegal activities online is an Official Bad Idea? One SpaceX fan earned a night in jail after posting selfies he took with Starhopper, the SpaceX test article currently residing at Elon Musk’s would-be spaceport at Boca Chica, Texas. JB Wagoner, a SpaceX super-fan, made the pilgrimage from California to Texas — in his Tesla of course — to see the recent Starship Mark 1 unveiling, and decided to take a side trip to see the Starhopper. He parked at a beach, climbed a dune, and was able to walk right up to Starhopper and go selfie-crazy. After posting the pictures on Facebook, he was arrested, interviewed by Homeland Security, charged with criminal trespass, and thrown in a cell overnight. Wagoner has since been bonded out, but the charges might not stick, since Texas trespassing law requires clear signage or verbal notification of trespass, neither of which Wagoner encountered. SpaceX had even let the fence between the beach and the Starhopper collapse, so Wagoner seems to have had no way of knowing he was trespassing. Still, posting the pictures online was probably asking for trouble.

As satire and dark comedy, the 1987 cyberpunk classic RoboCop can’t be beat. But it also managed to accurately foreshadow a lot of what was to come in the world in terms of technology. No, we don’t have cyborg law enforcement — yet — but we do have something predicted by one throwaway scene: robotic realtors. In the movie, kiosks were set up around Murphy’s old house to extol the various virtues of living there, which ended up triggering the cyborg and starting the film’s climactic rampage. The real-life robotic realtor is a little more flexible, more like a telepresence robot — described aptly as “a Segway with an iPad on top.” The robotic realtor is not autonomous; it only lets a remote realtor interact with potential homebuyers without having to travel to multiple homes. It seems a little gimmicky to us, but the robots are reported to have made 25 sales in their first year on the job.

We’ve been seeing a lot of cheap resin printers these days, enough to make us want to jump into the market and start playing with them. But the cheap ones are all cheap for the same reason — they’re so dang small! They all use LCD screens from phones to mask off the UV light used to cure the resin, and the resulting print volume is tiny. Clem Mayer from MayerMakes has bigger ideas, though: he wants to make a giant resin printer using an LCD monitor as the mask. It’s not as simple as using a bigger screen, though; the film used between the screen and the resin, a fluoropolymer film called FEP, gets deformed when used on larger screens. So Clem is looking at a new built-plate interface that floats the resin on a layer of denser, immiscible liquid. It’s an interesting idea that is still clearly in the proof-of-concept phase, but we look forward to seeing what progress Clem makes.