Some readers may recall the Lynx-R1 headset — it was conceived as an Android virtual reality (VR) and mixed reality (MR) headset with built-in hand tracking, designed to be open where others were closed, allowing developers and users access to inner workings in defiance of walled gardens. It looked very promising, with features rivaling (or surpassing) those of its contemporaries.
Founder [Stan Larroque] recently announced that Lynx’s 6DoF SLAM (simultaneous location and mapping) solution has been released as open source. ORB-SLAM3, modified for Android-based hardware (GitHub repository), takes in camera images and outputs a 6DoF pose, and does so effectively in real-time. The repository contains some added details as well as a demo application that can run on the Lynx-R1 headset.
As a headset the Lynx-R1 had a number of intriguing elements. The unusual optics, the flip-up design, and built-in hand tracking were impressive for its time, as was the high-quality mixed reality pass-through. That last feature refers to the headset using its external cameras as inputs to let the user see the real world, but with the ability to have virtual elements displayed and apparently anchored to real-world locations. Doing this depends heavily on the headset being able to track its position in the real world with both high accuracy and low latency, and this is what ORB-SLAM3 provides.
A successful crowdfunding campaign for the Lynx-R1 in 2021 showed that a significant number of people were on board with what Lynx was offering, but developing brand new consumer hardware is a challenging road for many reasons unrelated to developing the actual thing. There was a hands-on at a trade show in 2021 and units were originally intended to ship out in 2022, but sadly that didn’t happen. Units still occasionally trickle out to backers and pre-orders according to the unofficial Discord, but it’s safe to say things didn’t really go as planned for the R1.
It remains a genuinely noteworthy piece of hardware, especially considering it was not a product of one of the tech giants. If we manage to get our hands on one of them, we’ll certainly give you a good look at it.
They didn’t make orbslam-3. It was published back in 2020 (https://arxiv.org/abs/2007.11898) and was open sourced in 2021 (https://github.com/UZ-SLAMLab/ORB_SLAM3). I’ve personally used it in projects.
Just looking at the actual Lynx article linked, they say “We took ORB-SLAM3 code and made it compatible for Android, and more precisely Android running with a Qualcomm chipset.” So the Android-compatible implementation is what they are releasing here. Definitely should be clarified in the HaD article.
Thanks for highlighting that, I’ve made it more clear it’s Lynx’s own implementation that is being released here.
I looked into it but never got off the ground, what was your experience with it?
What do you mean “never got off the ground”? It’s the single most popular SLAM system ever written, except for ORBSLAM2. When people ask how a SLAM system performs, it’s always relative to either ORBSLAM2 or 3.
Thanks for that! Important distinction since the original is GPL and they are obligated to release the derivative
No relation to Atari Lynx handheld?
Orbslam3 is full of bugs and performs poorly. Just see the backlog of bugs in the official repo and the forks. I was able to compile it and run for android but it crashed after minutes. It’s also designed to have different set of input configurations, with mono, stereo or depth cameras with or without Imu. Trying to use Imu data from an intel stereo camera made it crash immediately. Moreover, the code is a uni project quality and definitely not consumer grade.