It’s been more than a year since we first heard about Leap Motion’s new, Open Source augmented reality headset. The first time around, we were surprised: the headset featured dual 1600×1440 LCDs, 120 Hz refresh rate, 100 degree FOV, and the entire thing would cost under $100 (in volume), with everything, from firmware to mechanical design released under Open licenses. Needless to say, that’s easier said than done. Now it seems Leap Motion is releasing files for various components and a full-scale release might be coming sooner than we think.
Leap Motion first made a name for themselves with the Leap Motion sensor, a sort of mini-Kinect that only worked with hands and arms. Yes, we’re perfectly aware that sounds dumb, but the results were impressive: everything turned into a touchscreen display, you could draw with your fingers, and control robots with your hands. If you mount one of these sensors to your forehead, and reflect a few phone screens onto your retinas, you have the makings of a stereoscopic AR headset that tracks the movement of your hands. This is an over-simplified description, but conceptually, that’s what Project North Star is.
The files released now include STLs of parts that can be 3D printed on any filament printer, files for the electronics that drive the backlight and receive video from a laptop, and even software for doing actual Augmented Reality stuff in Unity. It’s not a complete project ready for prime time, but it’s a far cry from the simple spec sheet full of promises we saw in the middle of last year.
You may remember that earlier this year Leap Motion revealed Project North Star, a kind of open-source reference design for an Augmented Reality (AR) headset. While it’s not destined to make high scores in the fashion department, it aims to be hacker-friendly and boasts a large field of view. There’s also an attractive element of “what you see is what you get” when it comes to the displays and optical design, which is a good thing for hackability. Instead of everything residing in a black box, the system uses two forward-facing displays (one for each eye) whose images are bounced off curved reflective lenses. These are essentially semitransparent mirrors which focus the images properly while also allowing the wearer to see both the displays and the outside world at the same time. This co-existence of both virtual and real-world visuals are a hallmark of Augmented Reality.
A serious setback to the aspiring AR hacker has been the fact that while the design is open, the lenses absolutely are not off the shelf components. [Smart Prototyping] aims to change that, and recently announced in a blog post that they will be offering Project North Star-compatible reflective lenses. They’re in the final stages of approving manufacture, and listed pre-orders for the lenses in their store along with downloadable 3D models for frames.
When Leap Motion first announced their open-source AR headset, we examined the intruiguing specifications and the design has since been published to GitHub. At the time, we did note that the only option for the special lenses seemed to be to CNC them and then spring for a custom reflective coating.
If the lenses become affordable and mass-produced, that would make the design much more accessible. In addition, anyone wanting to do their own experiments with near-eye displays or HUDs would be able to use the frame and lenses as a basis for their own work, and that’s wonderful.
We just wrapped up the Human Computer Interface challenge in this year’s Hackaday Prize, and this project is pushing boundaries we’ve hardly seen before. [Giovanni Leal] is using a Leap Motion controller to move a robotic arm around in space.
The robot arm in question comes from Owi, and it is by every measure not a good robot arm. It is, however, an excellent toy filled with motors and plastic linkages that serves as a good stand-in for a proper robotic arm.
Control of this toy robot arm is done through a Leap Motion controller. While the Leap Motion is a few years old at this point, it is a very effective way to ‘measure’ the position and rotation of a hand in 3D space. The only thing that’s required is the Leap Motion controller itself and a tabletop.
The end result is a robot that can be controlled by a hand. While this robot arm is really just a toy, it was fun to assemble and a little bit of hardware hacking with an Arduino turned this into a working robot arm controlled by a human. Scale this up, establish an island lair, and you’re on your way to taking over the world.
Leap Motion just dropped what may be the biggest tease in Augmented and Virtual Reality since Google Cardboard. The North Star is an augmented reality head-mounted display that boasts some impressive specs:
- Dual 1600×1440 LCDs
- 120Hz refresh rate
- 100 degree FOV
- Cost under $100 (in volume)
- Open Source Hardware
- Built-in Leap Motion camera for precise hand tracking
Yes, you read that last line correctly. The North Star will be open source hardware. Leap Motion is planning to drop all the hardware information next week.
Now that we’ve got you excited, let’s mention what the North Star is not — it’s not a consumer device. Leap Motion’s idea here was to create a platform for developing Augmented Reality experiences — the user interface and interaction aspects. To that end, they built the best head-mounted display they could on a budget. The company started with standard 5.5″ cell phone displays, which made for an incredibly high resolution but low framerate (50 Hz) device. It was also large and completely unpractical.
The current iteration of the North Star uses much smaller displays, which results in a higher frame rate and a better overall experience. The secret sauce seems to be Leap’s use of ellipsoidal mirrors to achieve a large FOV while maintaining focus.
We’re excited, but also a bit wary of the $100 price point — Leap Motion is quick to note that the price is “in volume”. They also mention using diamond tipped tooling in a vibration isolated lathe to grind the mirrors down. If Leap hasn’t invested in some injection molding, those parts are going to make the whole thing expensive. Keep your eyes on the blog here for more information as soon as we have it!
You could have said this at any time in the last couple of decades: the world of virtual reality peripherals does not yet feel as though it has fulfilled its potential. From the Amiga-powered Virtuality headsets and nausea-inducing Nintendo Virtual Boy of the 1990s to today’s crop of advanced headsets and peripherals, there has always been a sense that we’re not quite there yet. Moments at which the shortcomings of the hardware intrude into the virtual world may be less frequent with the latest products, but still the goal of virtual world immersion seems elusive at times.
One of the more interesting peripherals on the market today is the Leap Motion controller. This is a USB device containing infra-red illumination and cameras which provide enough resolution for its software to accurately calculate the position of a user’s hands and fingers in three-dimensional space. This ability to track finger movement gives it the function of a controller for really complex interactions with and manipulations of objects in virtual worlds.
Even the Leap Motion has its shortcomings though, moments at which it ceases to be able to track. Rotating your hand, as you might for instance when aiming a virtual in-game weapon, confuses it. This led [Florian Maurer] to seek his own solution, and he’s come up with a hand peripheral containing a rotation sensor.
Inspired by a movie prop from the film Ender’s Game, it is a 3D-printed device that clips onto the palm of his hand between thumb and index finger. It contains both an Arduino Pro Micro and a bno055 rotation sensor, plus a couple of buttons for in-game actions such as triggers. It solves the problem with the Leap Motion’s rotation detection, and does not impede hand movement so much that he can’t also use his keyboard and mouse while wearing it. Sadly he does not yet seem to have posted any code, but he does treat us to a video demonstration which we’ve posted below the break.
Continue reading “VR Feels More Real with Leap Motion and This Rotation Sensor”
[Florian] wants to browse the web like an internet cowboy from a cyberpunk novel. Unfortunately, VR controllers are great for games but really incapacitate a hand for typing. A new input method was needed, one that would free his fingers for typing, but still give his hands detailed input into the virtual world.
Since VR goggles have… hopefully… already reached peak ridiculousness, his first idea was to glue a Leap Motion controller to the front of it. It couldn’t look any sillier after all. The Leap controller was designed to track hands, and when combined with the IMU built into the VR contraption, did a pretty good job of putting his hands into the world. Unfortunately, the primary gesture used for a “click” was only registering 80% of the time.
The gesture in question is a pinching motion, pushing the thumb and middle finger together. He couldn’t involve a big button without incapacitating his hands for typing. It took a few iterations, but he arrived at a compact ring design with a momentary switch on it. This is connected to an Arduino on his wrist, but was out of the way enough to allow him to type.
It’s yet another development marching us to usable VR. We personally can’t wait until we can use some technology straight out of Stephenson or Gibson novel.
Did [TobiasWeis] build a mirror that’s better at reflecting his image? No, he did not. Did he build a mirror that’s better at reflecting himself? We think so. In addition to these philosophical enhancements, the build itself is really nice.
The display is a Samsung LCD panel with its inconvenient plastic husk torn away and replaced with a new frame made of wood. We like the use of quickly made 3D printed brackets to hold the wood at a perfect 90 degrees while drilling the holes for the butt joints. Some time with glue, band clamps, and a few layers of paint and the frame was ready. He tried the DIY route for the two-way mirror, but decided to just order a glass one after some difficulty with bubbles and scratches.
A smart mirror needs an interface, but unless you own stock in Windex (glass cleaner), it is nice to have a way to keep it from turning into an OCD sufferer’s worst nightmare. This is, oddly, the first justification for the Leap Motion controller we can really buy into. Now, using the mirror does not involve touching the screen. [Tobias] initially thought to use a Raspberry Pi, but instead opted for a mini-computer that had been banging around a closet for a year or two. It had way more go power, and wouldn’t require him to hack drivers for the Leap Motion on the ARM version of Linux.
After that is was coding and installing modules. He goes into a bit of detail about it as well as his future plans. Our favorite is programming the mirror to show a scary face if you say “bloody mary” three times in a row.