HoloLens Brings Video Game Kart Racing To Life

January 3, 2020 by Tom Nardi 12 Comments

There aren’t a lot of video game experiences we can easily recreate in the physical realm. You’ll quickly find that jumping on mushrooms in the real world doesn’t have nearly the same appeal as it does in Super Mario, and we won’t even get into the dangers of trying to recreate Frogger on your local multi-lane. But video game style go-kart racing? We have all the technology to pull that off, somebody just has to put all the pieces together.

Which is precisely what [Ian Charnas] is trying to do with his latest project. Using Microsoft’s HoloLens augmented reality headset, electric go-karts, 433 MHz wireless transceivers, and some Arduinos sprinkled in, he’s created the closest thing to Mario Kart that us flesh and blood mortals are likely to experience anytime soon.

The HoloLens headset worn by each driver overlays the necessary graphical elements like pickups and weapon effects, as well as puts over-the-top cartoon heads on the other racers. But of course, that’s only half of the story. Seeing the pickups and gadgets doesn’t do you any good if they don’t have any effect on the actual race.

To that end, [Ian] has come up with a way to control the performance of the go-karts using an electronic “backpack” that mounts to each kart. So speed boosting pickups actually make the kart go faster, and if a driver gets hit with a weapon fired at them, they get slowed down.

That’s the high-level version, anyway. There’s obviously a lot going on behind the scenes, some of which are detailed on the Hackaday.io page. One of the interesting notes is that the HoloLens needs visual markers to orient itself, which in the video after the break can be seen as black and white posters dotting the walls alongside the track. As the project progresses, [Ian] is hoping that these can be camouflaged in creative ways (such as being made to look like audience members or checkered flags) to make the overall experience more immersive.

According to [Ian], the next step is to find partners who want to help elevate this from a one-off project to something that you might actually see at an amusement park. We wish him luck, if for no other reason than we really want to play the thing ourselves. In the meantime, we’ll have to settle for racing hacked Power Wheels.

Continue reading “HoloLens Brings Video Game Kart Racing To Life” →

Sharing Virtual And Holographic Realities Via Vive And Hololens

February 2, 2017 by Donald Papp 13 Comments

An experimental project to mix reality and virtual reality by [Drew Gottlieb] uses the Microsoft Hololens and the HTC Vive to show two users successfully sharing a single workspace as well as controllers. While the VR user draws cubes in midair with a simple app, the Hololens user can see the same cubes being created and mapped to a real-world location, and the two headsets can even interact in the same shared space. You really need to check ou the video, below, to fully grasp how crazy-cool this is.

Two or more VR or AR users sharing the same virtual environment isn’t new, but anchoring that virtual environment into the real world in a way that two very different headsets share is interesting to see. [Drew] says that the real challenge wasn’t just getting the different hardware to talk to each other, it was how to give them both a shared understanding of a common space. [Drew] needed a way to make that work, and you can see the results in the video embedded below.

Continue reading “Sharing Virtual And Holographic Realities Via Vive And Hololens” →

HoloLens NES Emulator For Augmented Retro Gaming

August 24, 2016 by Moritz Walter 5 Comments

[Andrew Peterson] was looking for a way to indulge in his retro gaming passions in a more contemporary manner. His 3D NES emulator “N3S” for Windows brings Nintendo classics to the HoloLens, turning pixels into voxels, and Super Mario into an augmented reality gingerbread man.

To run NES games on the HoloLens, [Andrew’s] emulator uses the Nestopia libretro core. Since AR glasses cry for an augmentation of the game itself, the N3S re-emulates the NES’ picture processing unit (PPU), allowing it to interpret a Nintendo game’s graphics in a 3D space. [Andrew] also put together a comprehensive explanation of how the original Nintendo PPU works, and how he re-implemented it for the HoloLens.

The current version of the N3S PPU emulator automatically generates voxels by simply extruding the original pattern data from the game’s ROM, but [Andrew] is thinking about more features. Users could sculpt their own 3D versions of the original graphic elements in an inbuilt editor, and model sets could then be made available in an online database. From there, players would just download 3D mods for their favorite games and play them on the HoloLens.

According to [Andrew], the emulator reaches the limits of what the current pre-production version of the HoloLens can render fluently, so the future of this project may depend on future hardware generations. Nevertheless, the HoloLens screen capture [Andrew] recorded makes us crave for more augmented retro gaming. Enjoy the video!

Hacklet 54 – Virtual Reality Projects

July 3, 2015 by Adam Fabio 4 Comments

Virtual Reality is finally coming of age. Hackers, Makers and Engineers have dreamed of creating immersive interfaces for years. From the first flight simulators to today’s cellphone powered head mounted displays, VR has always been an exciting field. Many of the advances today are being created by hackers who were inspired by systems like Virtuality from the early 1990’s. Now 25 years on, we’re seeing amazing advances – not only in commercial systems, but in open source VR projects. This week’s Hacklet is all about the best VR projects on Hackaday.io!

We start with [j0nno] and D.I.Y Virtual Reality. [J0nno] has become interested in VR, and decided to build his own head mounted display. His goal is to create a setup with full head tracking and an open source software stack. He’s hoping to do this within a budget of just $200 AUD. [J0nno] started with the Ritech3d-V2 VR Goggles, which are a plastic implementation of Google’s project cardboard. For display he’s using a 5.6 inch 1280 x 800 TFT LCD. Tracking is optical, using IR LEDs and a PS3 Eye camera. [J0nno’s] background is in software, so he’s doing great setting up OpenVR and Perception. The hardware side is a bit new to him. This isn’t stopping [J0nno] though! In true hacker spirit, he’s learning all about resistors and driving LEDs as he works on D.I.Y Virtual Reality.

Next up is [Josh Lindsay] with Digitabulum: The last motion-capture glove. Digitabulum is a motion capture glove designed to be able to emulate most other motion capture systems. It is also designed to be relatively low-cost. At $400 per hand, it is less expensive than most other offerings, though we’d still love to see something even cheaper. [Josh] is going with inertial sensors, and a lot of them. Specifically he’s using no less than 17 LSM9DS1 Inertial Measurement Unit (IMU) sensors from ST Microelectronics. IMU sensors like this combine multiple rate gyros, accelerometers, and magnetometers into a single unit. Essentially every segment of every finger has its own sensor suite. As you might imagine, that is quite a bit of data to crunch. An Altera Max II CPLD and an ST Arm processor help boil down the data to something which a VR engine can process. [Josh] has been working on this project for over a year now, and he’s making great progress. The prototype glove looks terrific!

[Thomas] brings augmented reality to the table with Oculus Rift featured Crane control. What started as a hobby experiment became [Thomas’] major project at university. He’s connected an Oculus Rift to a toy crane. A stereo camera on the crane sends a video image to the operator. The camera is mounted on a pan/tilt mechanism driven by the Rift’s head tracking unit. Simple joystick controls allow [Thomas] to move the boom and lower the line. On-screen displays show the current status of the crane. The use of the Rift makes this an immersive demonstration. One could easily see how moving this system into the real world would make crane operations safer for crane operators.

Finally we have [Arcadia Labs] with DIY Augmented Reality Device. This project, which is the [Arcadia Labs] entry in the 2015 Hackaday Prize, uses two 320 x 240 screens to create an augmented reality head mounted display. While the resolution can’t match that of the Oculus Rift or HTC Vive, [Arcadia Labs] is ok with that. They’re going for a lower cost open source alternative for augmented reality. Tracking is achieved with an IMU, while a PS3 Eye camera provides the video. A Raspberry Pi controls the show. [Arcadia Labs] was able to get 50 frames per second on the displays just using the Pi’s SPI interface, however the USB PS3 Eye camera limits things to around 10 FPS. This project is under heavy development right now, so follow along with us to see where [Arcadia Labs] ends up!

If you want VR goodness, check out our new virtual reality projects list! Did I miss your project? Don’t be shy, just drop me a message on Hackaday.io. If you’re on the left coast of the USA, check out SOCAL Virtual Reality Conference and Expo. Hackaday is a sponsor. The event happens on July 12 at the University of California Irvine.

That’s it for this week’s Hacklet, As always, see you next week. Same hack time, same hack channel, bringing you the best of Hackaday.io!

DIY Hololens Uses Pepper’s Ghost In A Box!

March 17, 2015 by James Hobson 20 Comments

Entirely too excited about Microsoft’s Hololens, the DIY community has leaped on the challenge to make some hardware before the real deal comes out. [Sean Hall] has an excellent 3D printed prototype that makes use of the Pepper’s Ghost illusion to create a “hologram” for this pair of unique VR goggles.

Similar to other DIY virtual reality goggles we’ve seen, [Sean] has 3D printed the enclosure — but instead of slapping the smart phone right in front of your eyes, it’s mounted above the goggles, reflecting off of a mirror and then a piece of transparent plexi-glass, which produces a hologram like effect thanks to the concept of Pepper’s Ghost illusion.

The problem with any of these reflection-based-holograms is they aren’t always that easy to see, so [Sean] is planning to try out some 1-way reflective car tint to get a more visible reflection while still being able to see through the image. He also plans to add gaze tracking with some open-source software called Project Haytham. It’s a depth sensor using a Kinect, head tracking using a Playstation Move and maybe even a leap motion controller for virtual object manipulation.

Check out the current state of this hack in the clip after the break.

Continue reading “DIY Hololens Uses Pepper’s Ghost In A Box!” →