Soon You’ll Sit Inside A Robot’s Head At Work

MIT’s Computer Science and Artificial Intelligence Lab, CSAIL, has created a process of teleoperating a Baxter humanoid robot with an Oculus Rift VR headset. This project is partially aimed towards making manufacturing jobs a hell of a lot of fun telecommutable. It could even be a way to supervise robot workers from a distance.

In a nutshell, the user controls the robot remotely in a virtual reality environment. The user does this specifically in a VR environment modeled like a control room with multiple sensor displays, making it feel like they are sitting inside the robot’s head. By using hand controllers, users can match their movements to the robot’s to complete various tasks. If you’ve seen Pacific Rim, you are probably envisioning a Jaegar right about now — minus the psychic linking.

The whitepaper (PDF) walks through the choices the team made in developing this system. In the past, teleoperation was usually approached with a direct model or a cyper-physcial model.

In a “direct” model, the user’s vision is directly coupled to the robot’s state but a delayed signal could lead to nausea and headaches because of a lack of vestibular stimulation with visual representation of motion. In a “cyber-physical” model, the user is separate from the robot and interacts with a virtual copy of the robot and the environment. While this method was better for the driver it required much more data and specialized spaces.

Like any successful hodgepodge, the CSAIL team’s system has combined the two previous methods into something a bit more advanced. It solves the delay problem as the user is constantly receiving visual feedback from the virtual world. It additionally solves the cyber-physical issue of feeling distinct from the robot: once a user puts on the headset and logs into the system, it will seem as if they are inside Baxter’s head. The brilliance of this system is that it mimics the “homunculus model of mind” — the idea that there is a tiny human inhabiting our brains, controlling our actions, viewing the images we see, and understanding them for us.

Unsurprisingly, users with gaming experience had much more ease with the system as well. Much like the military shows an interest in skilled FPS players, we might soon see a wave of recruitment for video gamers in the manufacturing industry. Maybe all of those hours poured into the claw machine was worth it after all.

[Thanks Adam]

28 thoughts on “Soon You’ll Sit Inside A Robot’s Head At Work

  1. this is beautiful for specialists like surgeons. You don’t have to fly them in, but they can operate remotely with a few assistants. Throw in a self-learning neural network and the robot can take over more and more parts of the job.

        1. Yeah. Same thing with those ‘horseless-carriages’ people keep going on about. There might be a niche for them, but they’ll never really take off.

          See? I can take technologies that already exist and are widely used and pretend they aren’t also!

          1. You could also ask yourself what real-life problem is this actually trying to solve? How many automation jobs are actually crying out for a remote worker? Hopefully better than what white-collar workers are doing.

    1. Yeah, the surgeon can now be in China making $7.25/hour and the rest of the jobs can be outsourced to China for $0.10/hour. Meanwhile we Americans all get tossed out like empty beer bottles.

  2. If a robot is being hand controlled from the pendulum it’s usually because it’s either receiving a first teach in, testing or something went wrong and it needs to be recovered to safe position, meaning local presence is needed for safety reasons. Tele-operation doesn’t work for any of these situations. I really fail to see the application for this. Once teach in and testing is completed a robot should be doing work on it’s own without needing supervision.

    1. I should add, if it’s being hand controlled at all it’s because it needs to be done locally for greater precision and safety in case of system failure. None of that can be achieved with the system shown here.

      If it’s for tele-presence the controls through the VIVE controllers seem clumsy at best

  3. I see this as a great solution for factory owners looking to cut down on costs in about 30 years time. Just go to somewhere where the wages are cheap and the internet reliable, cobble together this hardware on your line, and bam, ‘Outsourcing 3.0’.

  4. Seems once latency issues are overcome and the opposite, artificial intelligence algorithms or some sort of logic if there is a movement that is to fast maybe by using some sort of object recognition database library to identify patterns that are not conducive to the procedures/methods to the roles operation, then this would be great for mission critical operations in environments that are too hostile to be exposed to. Since the 90’s I though we should have robots under the ocean developing and in space or other planets. I’ve even thought we could have geothermal power plants on volcano’s as barges using external combustion engines. Can do on geothermal vents too using subs as should the plants in general can be to prevent natural disaster issues. I think we need to use more heat and waste instead of manufacturing that.

    1. The problem I see is that AI may get to the point were a remote operator at best is rarely needed. That’s the ultimate goal being pursued. Kind of like cars without operators. ;-)

    1. Interesting the proposals for “taxes” on robots, the implications of society utilization of business implements if required for survival and industry business leadership ethics, moral and social values. Seems there is a point where a business turns to not really business operation and more dangerous to society entity. Maybe, the worst case scenario is derived from hostile groups hostile take over of a business and stakeholders/shareholders capital. I think you can be concerned the same with government who would regulate and process the taxes.

      Almost seems like the individual from birth can be valued and brokered based on credit derived from birth. Hostile in ways too.

      Thanks for the Western Space and Marine reference also.

    1. Some days… seems like. Test score performance determines what you can and can’t do unless you can compel or have the inside connections. Kind of like logical and required versus emotionally captivating/annoying (begging)… then there are the militias that are not all regular yet. Or do you mean pedo drunks and junkies (like “nobody’s”) driving around to other dangerous places? They’re definitely in the official acting positions already.

  5. I sympathize with your Lag statement, but I’m not convinced that it can’t be compensated with human intelligence. Unexpected lag is stressful but if you know it is there and consistent, I think you can learn to acclimate. The big test of the concept will be when the users and vr equipment are on the earth and the robots are on the moon. It will be good to hire highly coordinated gamers to do the work but they will have to adapt to the >1 second lag, but there is no doubt in my mind that they can do it and even adapt to the point of being comfortable with it. Another plus is that you could commute to your basement to go to work. The downside is you could find yourself in job competition with millions of people on the other side of the planet. We have a problem with outsourcing too many IT jobs. This kind of tech could cause the same problem with manufacturing, but I bet there are ways to make it acceptable to everyone. It should be cool to watch this technology develop.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.