There are many in the hacker community who would love to experiment with augmented reality (AR), but the hardware landscape isn’t exactly overflowing with options that align with our goals and priorities. Commercial offerings, from Google’s Glass to the Microsoft HoloLens and Magic Leap 2 are largely targeting medical and aerospace customers, and have price tags to match. On the hobbyist side of the budgetary spectrum we’re left with various headsets that let you slot in a standard smartphone, but like their virtual reality (VR) counterparts, they can hardly compare with purpose-built gear.
But there’s hope — Brilliant Labs are working on AR devices that tick all of our boxes: affordable, easy to interface with, and best of all, developed to be as open as possible from the start. Admittedly their first product, Monocle, it somewhat simplistic compared to what the Big Players are offering. But for our money, we’d much rather have something that’s built to be hacked and experimented with. What good is all the latest features and capabilities when you can’t even get your hands on the official SDK?
This week we invited Brilliant Lab’s Head of Engineering Raj Nakaraja to the Hack Chat to talk about AR, Monocle, and the future of open source in this space that’s dominated by proprietary hardware and software.
In the world of hardware hacking, you sometimes spend a ridiculous amount of time debugging a problem, only to find a simple solution that was right in front of you the whole time. [Zack Freedman] got a good dose of this while building the Optigon V2, a modified Epson Moverio wearable display he uses as a teleprompter in all his videos. He prefers having the teleprompter over his left eye only, but the newer version of the Moverio would shut off both sides if one is disconnected, so [Zack] needed a workaround.
Looking for some help from above, [Zack] requested developer documentation for the display module from Epson, but got declined because he wasn’t a manufacturer or product developer. Luckily, a spec sheet available for downloaded from the Epson website did contain a lot of the information he needed. An STM32 monitored the temperature of each display module over a pair of independent I2C interfaces, and would shut down everything if it couldn’t connect to either. This led [Zack] to attempt to spoof the I2C signals with an ATmega328, but it couldn’t keep up with the 400 kHz I2C bus.
However, looking at the logs from his logic analyzer, [Zack] found that the STM32 never talked to both display modules simultaneously, even though it is capable of doing so. Both displays use the same I2C address, so [Zack] could simply connect the two I2C buses to each other with a simple interface board, effectively making the left display “spoof” the signals from the right display.
Generating the real-time images required for augmented reality (AR) goggles usually requires a fair amount of processing power, to the point that DIY efforts based around the Raspberry Pi often have trouble keeping up. But what if your AR aspirations don’t require fancy high-resolution graphics? If text and the occasional icon is enough to get the job done, then these lo-fi AR goggles from [bobricius] might be the ideal solution.
As with previous homebrew AR rigs we’ve seen, this one starts with an affordable headset designed to project the display of a smartphone onto a pair of curved optical combiners. But instead of tucking a phone into the headset, [bobricius] is using a custom PCB that holds a pair of ST7789 1.3 inch 240 x 240 IPS displays. Connected over SPI and supported by just about any microcontroller you’d care to use, tossing some textual data over your field of vision can be accomplished in just a few lines of code.
[bobricius] has actually put together a couple different versions of the PCB for this project. One uses his custom ATSAMD21E18-based “ArmaBrain” module that packs the MCU and an array of common components onto a 28 mm square board that can be easily dropped into other projects. If you’d rather roll your own solution, the second version of the board that simply holds the two displays in the appropriate position and routes the SPI lines to a convenient header should do nicely.
After seeing the cheap transparent OLED displays that have recently hit the market, you might have thought of using them as an affordable way to build your own wearable display. To save you the inevitable disappointment that would result from such a build, [Zack Freedman] took it upon himself to test out the idea, and show why transparent wearable displays are a harder than it looks.
He put together a headband with integrated microcontroller that holds the transparent OLED over the user’s eye, but unfortunately, anything shown on the display ends up being more or less invisible to the wearer. As [Zack] explains in the video after the break, the human eye is physically incapable of focusing on any object at such a short distance. Contrary to what many people might think, the hard part of wearable displays is not in the display itself, but rather the optics. For a wearable display to work, all the light beams from the display need to be focused into your eyeball by lenses and or reflectors, without distorting your view of everything beyond the lens. This requires, lightweight and distortion-free collimators and beam splitters, which are expensive and hard to make.
Augmented reality (AR) and natural gesture input provide a tantalizing glimpse at what human-computer interfaces may look like in the future, but at this point, the technology hasn’t seen much adoption within the open source community. Though to be fair, it seems like the big commercial players aren’t faring much better so far. You could make the case that the biggest roadblock, beyond the general lack of software this early in the game, is access to an open and affordable augmented reality headset.
Which is precisely why [Graham Atlee] has developed the Triton. This Creative Commons licensed headset combines commercial off-the-shelf components with 3D printed parts to provide a capable AR experience at a hacker-friendly price. By printing your own parts and ordering the components from AliExpress, basic AR functionality should cost you $150 to $200 USD. If you want to add gesture support you’ll need to add a Leap Motion to your bill of materials, but even still, it’s a solid deal.
The trick here is that [Graham] is using the reflectors from a surprisingly cheap AR headset designed to work with a smartphone. By combining these mass produced optics with a six inch 1440 x 2560 LCD panel inside of the Triton’s 3D printed structure, projecting high quality images over the user’s field of view is far simpler than you might think.
If you want to use it as a development platform for gesture interfaces you’ll want to install a Leap Motion in the specifically designed socket in the front, but otherwise, all you need to do is plug in an HDMI video source. That could be anything from a low-power wearable to a high-end gaming computer, depending on what your goals are.
[Graham] has not only provided the STLs for all the 3D printed parts and a bill of materials, but he’s also done a fantastic job of documenting the build process with a step-by-step guide. This isn’t some theoretical creation; you could order the parts right now and start building your very own Triton. If you’re looking for software, he’s also selling a Windows-based “Triton AR Launcher” for the princely sum of $4.99 that looks pretty slick, but it’s absolutely not required to use the hardware.
Of course, plenty of people are more than happy to stick with the traditional keyboard and monitor setup. It’s hard to say if wearable displays and gesture interfaces will really become the norm, of they’re better left to science fiction. But either way, we’re happy to see affordable open source platforms for experimenting with this cutting edge technology. On the off chance any of them become the standard in the coming decades, we’d hate to be stuck in some inescapable walled garden because nobody developed any open alternatives.
TVs are usually something you sit and passively watch. Not so for [Nate Damen’s] interactive, wearable TV head project, aka Atltvhead. If you’re walking around Atlanta, Georgia and you see him walking around with a TV where his head should be, introduce yourself! Or sign into Twitch chat and take control of what’s being displayed on the LEDs which he’s attached to the screen. Besides being wearable technology, it’s also meant to be an interactive art piece.
For this, his third version, the TV is a 1960’s RCA Victor Portable Television. You can see some of the TVs he found for previous versions on his hackaday.io page. They’re all truly vintage. He gutted this latest one and attached WS2812 LED strips in a serpentine pattern inside the screen. The LEDs are controlled by his code and the FastLED library running on an ESP8266. Power comes from four NiMH AA-format batteries, giving him 5 V, which he regulates down to 3.3 V. His phone serves as a WiFi hotspot.
[Nate] limits the commands so that only positive things can be displayed, a heart for example. Or you can tweak what’s being displayed by changing the brightness or make the LEDs twinkle. Judging by the crowds we see him attracting in the first video below, we’d say his project was a huge success. In the second video, Nate does a code walkthrough and talks about some of his design decisions.
If you think of wearable electronic projects, in many cases what may come to mind are the use of addressable LEDs, perhaps on strips or on sewable PCBs like the Neopixel and similar products. They make an attractive twinkling fashion show, but there remains a feeling that in many cases once you have seen one project, you have seen them all.
So if you are tiring of static sewable LED projects and would like to look forward to something altogether more exciting, take a look at some bleeding-edge research from a team at KAIST, the Korean Advanced Institute of Science & Technology. They have created OLED fibres and woven them into fabric in a way that appears such that they can be lit at individual points to create addressable pixels. In this way there is potential for fabrics that incorporate entire LED displays within their construction rather than in which they serve as a substrate.
The especially interesting feature of the OLED fibres from the KAIST team is that their process does not require any high temperatures, meaning that a whole range of everyday textile fibres can be used as substrates for OLEDs. The results are durable and do not lose OLED performance under tension, meaning that there is the possibility of their becoming practical fabrics for use in garments.
While this technology is a little way away from a piece of clothing you might buy from a store, the fact that it does not rely on special processes during weaving means that when the fibres become commercially available we are likely to see their speedy adoption. Meanwhile you can buy conductive fabric, but you might have to take a multimeter to the store to find it.