Hackaday Links Column Banner

Hackaday Links: February 11, 2024

Apple’s Vision Pro augmented reality goggles made a big splash in the news this week, and try as we might to resist the urge to dunk on them, early adopters spotted in the wild are making it way too easy. Granted, we’re not sure how many of these people are actually early adopters as opposed to paid influencers, but there was still quite a bit of silliness to be had, most of it on X/Twitter. We’d love to say that peak idiocy was achieved by those who showed themselves behind the wheels of their Teslas while wearing their goggles, with one aiming for an early adopter perfecta, but alas, most of these stories appear to be at least partially contrived. Some people were spotted doing their best to get themselves killed, others were content to just look foolish, especially since we’ve heard that the virtual keyboard is currently too slow for anything but hunt-and-peck typing, which Casey Niestat seemed to confirm with his field testing. After seeing all this, we’re still unsure why someone would strap $4,000 worth of peripheral-vision-restricting and easily fenced hardware to their heads, but hey — different strokes. And for those of you wondering why these things are so expensive, we’ve got you covered.

Continue reading “Hackaday Links: February 11, 2024”

Dr. Niels Olson uses the Augmented Reality Microscope. (Credit: US Department of Defense)

Google’s Augmented Reality Microscope Might Help Diagnose Cancer

Despite recent advances in diagnosing cancer, many cases are still diagnosed using biopsies and analyzing thin slices of tissue underneath a microscope. Properly analyzing these tissue sample slides requires highly experienced and skilled pathologists, and remains subject to some level of bias. In 2018 Google announced a convolutional neural network (CNN) based system which they call the Augmented Reality Microscope (ARM), which would use deep learning and augmented reality (AR) to assist a pathologist with the diagnosis of a tissue sample. A 2022 study in the Journal of Pathology Informatics by David Jin and colleagues (CNBC article) details how well this system performs in ongoing tests.

For this particular study, the LYmph Node Assistant (LYNA) model was investigated, which as the name suggests targets detecting cancer metastases within lymph node biopsies. The basic ARM setup is described on the Google Health GitHub page, which contains all of the required software, except for the models which are available on request. The ARM system is fitted around an existing medical-grade microscope, with a camera feeding the CNN model with the input data, and any relevant outputs from the model are overlaid on the image that the pathologist is observing (the AR part).

Although the study authors noted that they saw potential in the technology, as with most CNN-based systems a lot depends on how well the training data set was annotated. When a grouping of tissue including cancerous growth was marked too broadly, this could cause the model to draw an improper conclusion. This makes a lot of sense when one considers that this system essentially plays ‘cat or bread’, except with cancer.

These gotchas with recognizing legitimate cancer cases are why the study authors see it mostly as a useful tool for a pathologist. One of the authors, Dr. Niels Olsen, notes that back when he was stationed at the naval base in Guam, he would have liked to have a system like ARM to provide him as one of the two pathologists on the island with an easy source of a second opinion.

(Heading image: Dr. Niels Olson uses the Augmented Reality Microscope. (Credit: US Department of Defense) )

Beautifully Rebuilding A VR Headset To Add AR Features

[PyottDesign] recently wrapped up a personal project to create himself a custom AR/VR headset that could function as an AR (augmented reality) platform, and make it easier to develop new applications in a headset that could do everything he needed. He succeeded wonderfully, and published a video showcase of the finished project.

Getting a headset with the features he wanted wasn’t possible by buying off the shelf, so he accomplished his goals with a skillful custom repackaging of a Quest 2 VR headset, integrating a Stereolabs Zed Mini stereo camera (aimed at mixed reality applications) and an Ultraleap IR 170 hand tracking module. These hardware modules have tons of software support and are not very big, but when sticking something onto a human face, every millimeter and gram counts.

Continue reading “Beautifully Rebuilding A VR Headset To Add AR Features”

Chatting About The State Of Hacker-Friendly AR Gear

There are many in the hacker community who would love to experiment with augmented reality (AR), but the hardware landscape isn’t exactly overflowing with options that align with our goals and priorities. Commercial offerings, from Google’s Glass to the Microsoft HoloLens and Magic Leap 2 are largely targeting medical and aerospace customers, and have price tags to match. On the hobbyist side of the budgetary spectrum we’re left with various headsets that let you slot in a standard smartphone, but like their virtual reality (VR) counterparts, they can hardly compare with purpose-built gear.

But there’s hope — Brilliant Labs are working on AR devices that tick all of our boxes: affordable, easy to interface with, and best of all, developed to be as open as possible from the start. Admittedly their first product, Monocle, it somewhat simplistic compared to what the Big Players are offering. But for our money, we’d much rather have something that’s built to be hacked and experimented with. What good is all the latest features and capabilities when you can’t even get your hands on the official SDK?

This week we invited Brilliant Lab’s Head of Engineering Raj Nakaraja to the Hack Chat to talk about AR, Monocle, and the future of open source in this space that’s dominated by proprietary hardware and software.

Continue reading “Chatting About The State Of Hacker-Friendly AR Gear”

NVIDIA Jetson Powers Real-Time Iron Man HUD

If you could recreate any of the capabilities of Tony Stark’s Iron Man suit in real life, it would probably be the ability to fly, the super strength, or maybe even the palm-mounted lasers that can cut through whatever obstacle is in your path. But let’s be real, all that stuff is way too hard to try and pull off. Plus you’ll probably just end up accidentally killing yourself in the backyard.

But judging by the videos he’s been posting, [Kris Kersey] is doing one hell of a job creating a functional heads-up display (HUD) similar to the one Tony uses in the films. He’s even building it into a 3D printed Iron Man helmet, with the NVIDIA Jetson board that’s powering the show inside a chest-mounted “Arc Reactor”. He goes into a bit more detail about the project and his goals in an interview recently published on NVIDIA’s own blog. Continue reading “NVIDIA Jetson Powers Real-Time Iron Man HUD”

Supercon 2022: Aedan Cullen Is Creating An AR System To Beat The Big Boys

There’s something very tantalizing about an augmented reality (AR) overlay that can provide information in daily life without having to glance at a smartphone display, even if it’s just for that sci-fi vibe. Creating a system that is both practical and useful is however far from easy, which is where Aedan Cullen‘s attempt at creating what he terms a ‘practical augmented reality device’.

In terms of requirements, this device would need to have a visual resolution comparable to that of a smartphone (50 pixels/degree) and with a comparable field of view (20 degrees diagonal). User input would need to be as versatile as a touchscreen, but ‘faster’, along with a battery life of at least 8 hours, and all of this in a package weighing less than 50 grams.

Continue reading “Supercon 2022: Aedan Cullen Is Creating An AR System To Beat The Big Boys”

Inspect The RF Realm With Augmented Reality

Intellectually, we all know that we exist in a complex soup of RF energy. Cellular, WiFi, TV, public service radio, radar, ISM-band transmissions from everything from thermometers to garage door openers — it’s all around us. It would be great to see these transmissions, but alas, most of us don’t come from the factory with the correct equipment.

Luckily, aftermarket accessories like RadioFieldAR by [Manahiyo] make it possible to visualize RF signals. As the name suggests, this is an augmented reality system that lets you inspect the RF world around you. The core of the system is a tinySA, a pocket-sized spectrum analyzer that acts as a broadband receiver. A special antenna is connected to the tinySA; unfortunately, there are no specifics on the antenna other than it needs to have a label with an image of the Earth attached to it, for antenna tracking purposes. The tinySA is connected to an Android phone — one that supports Google’s ARCore — by a USB OTG cable, and a special app on the phone runs the show.

By slowly moving the antenna around in the field of view of the phone’s camera, a heat map of signal strength at a particular frequency is slowly built up. The video below shows it in action, and the results are pretty cool. If you don’t have a tinySA, fear not — [Manahiyo] has a version of the app that supports a plain old RTL-SDR dongle too. That should make it easy for just about anyone to try this out.

And if you’re feeling deja vu about this, you’re probably remembering the [Manahiyo]’s VR spectrum analyzer, upon which this project is based.

Continue reading “Inspect The RF Realm With Augmented Reality”