Kinect

154 Articles

Cutting Ribbons With Robots And A Oculus Rift

July 21, 2014 by 12 Comments

PR2-GrandOpening

On June 26th, 2014, Clearpath Robotics opened up the doors to their brand new 12,000 square foot robot lair by bringing out a PR2 to cut the ceremonial ribbon and welcome everyone inside. And instead of just programming the ‘locate and destroy’ ribbon sequence, the co-founders opted to use an Oculus Rift to control the robot tearing through the material with flailing arms.

This was accomplished having Jake, the robot, utilize a Kinect 2.0 that fed skeleton tracking data via rosserial_windows, a windows-based set of extension for the Robot Operating System which we heard about in January. The software gathers in a stream of data points each with an X,Y,Z component allowing [Jake] to find himself within a 3D space.Then, the data was collected and published directly into the PR2’s brain. Inject a little python code, and the creature was able to route directions in order to move it’s arms.

Thus, by simply stepping in front of the Kinect 2.0, and putting on the Oculus Rift headset, anyone could teleoperate [Jake] to move around and wave its arms at oncoming ribbons. Once completed, [Jake] would leave the scene, journeying back into the newly created robot lair leaving pieces of nylon and polyester everywhere.

An earlier (un-smoothed) version of the full system can be seen after the break:

Continue reading “Cutting Ribbons With Robots And A Oculus Rift” →

Virtual Physical Reality With Kintinuous And An Oculus Rift

June 21, 2014 by 3 Comments

oculus

The Kinect has long been able to create realistic 3D models of real, physical spaces. Combining these Kinect-mapped spaces with an Oculus Rift is something brand new entirely.

[Thomas] and his fellow compatriots within the Kintinuous project are modeling an office space with the old XBox 360 Kinect’s RGB+D sensors. then using an Oculus Rift to inhabit that space. They’re not using the internal IMU in the Oculus to position the camera in the virtual space, either: they’re using live depth sensing from the Kinect to feed the Rift screens.

While Kintinuous is very, very good at mapping large-scale spaces, the software itself if locked up behind some copyright concerns the authors and devs don’t have control over. This doesn’t mean the techniques behind Kintinuous are locked up, however: anyone is free to read the papers (here’s one, and another, PDF of course) and re-implement Kintinuous as an open source project. That’s something that would be really cool, and we’d encourage anyone with a bit of experience with point clouds to give it a shot.

Video below.

Continue reading “Virtual Physical Reality With Kintinuous And An Oculus Rift” →

Ecstatic Computation: Exploring Technoshamanism With Virtual Reality

June 12, 2014 by 62 Comments

A Digital Shaman

Here’s a really clever use for the Oculus Rift — Ecstatic Computation, a virtual reality spirit journey.

[Michael Allison] began his university career as an artist and musician… and somehow down the line, became a Technoshaman. His thesis, presented at ITP 2014, is on computational art, virtual reality, cognitive psychology and his research on various religious, spiritual and scientific methods that try to explain the relationship between our bodies, minds and the universe itself.

Using virtual reality, Ecstatic Computation is a ritual that explores the merging of consciousness and quantum energy in the physio-chemical registration of state within the computer’s memory. The moment when human and computer become one; the moment when thought becomes bit and electrons become ideas.

Sound crazy? Maybe — but check out the video demonstrations after the break. To create this experience he’s using an Oculus Rift, a Microsoft Kinect, a fan, a small keyboard and of course a computer to render it all. During the participant’s journey, [Michael] leads them in flight, passing through a quantum tunnel, merging with quantum energy inside of state registration within the computer’s memory and finally ending by falling into infinity.

All the graphics and effects are generated on the fly using GLSL generation using a robust graphics rendered called Smolder which he wrote himself, which is built on top of Cinder.

Continue reading “Ecstatic Computation: Exploring Technoshamanism With Virtual Reality” →

The Race Is On To Build A Raspi Kinect 3D Scanner

June 3, 2014 by 29 Comments

pinect

The old gen 1 Kinect has seen a fair bit of use in the field of making 3D scans out of real world scenes. Now that Xbox 360 Kinects are winding up at yard sales and your local Goodwill, you might even have a chance to pick one up for pocket change. Until now, though, scanning objects in 3D has only been practical in a studio or workshop setting; for a mobile, portable scanner, you’d need to lug around a computer, a power supply, and it’s not really something you can fit in a back pack.

Now, finally, that may be changing. [xxorde] can now get depth data from a Kinect sensor with a Raspberry Pi. And with just about every other ARM board out there as well. It’s a kernel driver that’s small, fast, and does just one thing: turns the Kinect into a webcam that displays depth data.

Of course, a portabalized Kinect 3D scanner has been done before, but that was with an absurdly expensive Gumstix board. With a Raspi or BeagleBone Black, this driver has the beginnings of a very cheap 3D scanner that would be much more useful than the current commercial or DIY desktop scanners.

Robot-Army IRL Plus A Massive Build Log

May 30, 2014 by 8 Comments

We went to “the dark room” at Maker Faire once more for an interview with [Sarah] of Robot-Army. She and [Mark], who handles software development for the project, were showing off 30 delta robots who know how to dance. Specifically they’re dancing in unison to the movements of another faire-goer. A Kinect sensor monitors those movements and translates them to matching motions from the deltabots.

You should remember seeing this project back in November. Now that the standards for this model have been worked out it was just a matter of sinking about three-weeks into assembling the army. We’re happy to see that the Kickstarter made it to 250% of the goal at the beginning of March, and with that there are even bigger plans. [Sarah] says the goal remains to fill a room with the robots and a we may even see a much larger version some day.

The interview is a bit short since the Robot-Army booth was right next to Arc Attack (hence the noise-cancelling headphones) and we had to try to get in and out between their ear-drum-shattering interruptions. But you can see a ton more about the project in this huge build log post over on Hackaday.io. Also check out the Robot-Army webpage. There’s a nice illustration of their adventures at MFBA and the foam Jolly Wrencher made it into the piece!

Oculus Rift And Wii Balance Board Make Hoverboards A (Virtual) Reality

May 22, 2014 by 12 Comments

It’s almost 2015 and still don’t have the futuristic technology promised to us by Back to the Future Part II. Where are the flying cars, Mr. Fusions, or 19 Jaws movies? Most importantly, where are our hoverboards?

[cratesmith] got tired of waiting around and decided to take matters into his own hands. He combined the Oculus Rift virtual reality headset with the Wii Fit Balance Board to create a virtual hoverboard experience. He used the Unity3D engine (a favorite among Rift developers) to program the game engine. It’s a very rough demo right now, but the game comes complete with a simulated town to float around in and of course includes a model DeLorean.

Before you try to play this demo, you should know that it’s not without its faults. The primary problem [cratesmith] has experienced is with simulation sickness. His virtual reality system has no way to track body motion, which means that leaning back and forth on the Wii Fit board does not get translated to the equivalent virtual movement. The game must assume that the player stands straight up at all times, which is not an intuitive way to control something similar to a skateboard. The result is an off-putting experience that can break immersion and lead to a feeling of nausea.

A possible solution to this problem would be to use a camera style motion detector like the Microsoft Kinect. In fact, another Reddit user has recently posted a teaser video of another hoverboard simulator that uses the Oculus Rift, Wii Fit Board, and Kinect. Not much information is available about this second project, but we look forward to seeing updates in the future.

[createsmith] has not published the code for his demo because it’s still in the very early stages, but he has stated that he’s been giving it out to anyone who goes out of their way to ask. The hoverboard is probably the most coveted fictional technology from the 1989 adventure film. We know this because we’ve seen multiple projects over the years that were inspired by the movie.  We’re excited to see it come to fruition in any form.

[via Reddit]

Virtual Reality Gets Real With 3 Kinect Cameras

May 14, 2014 by 25 Comments

No, that isn’t a scene from a horror movie up there, it’s [Oliver Kreylos’] avatar in a 3D office environment. If he looks a bit strange, it’s because he’s wearing an Oculus Rift, and his image is being stitched together from 3 Microsoft Kinect cameras.

[Oliver] has created a 3D environment which is incredibly realistic, at least to the wearer. He believes the secret is in the low latency of the entire system. When coupled with a good 3D environment, like the office shown above, the mind is tricked into believing it is really in the room. [Oliver] mentions that he finds himself subconsciously moving to avoid bumping into a table leg that he knows isn’t there. In [Oliver’s] words, “It circumnavigates the uncanny valley“.

Instead of pulling skeleton data from the 3 Kinect cameras, [Oliver] is using video and depth data. He’s stitching and processing this data on an i7 Linux box with an Nvidia Geforce GTX 770 video card. Powerful hardware for sure, but not the cutting edge monster rig one might expect. [Oliver] also documented his software stack. He’s using Vrui VR Toolkit, the Kinect 3D Video Capture Project, and the Collaboration Infrastructure.

We can’t wait to see what [Oliver] does when he gets his hands on the Kinect One (and some good Linux drivers for it).

Continue reading “Virtual Reality Gets Real With 3 Kinect Cameras” →