An Introduction To Valve’s Tracking Hardware

[Alan Yates] brought a demo of Valve’s new VR tech that’s the basis of the HTC Vive system to Maker Faire this year. It’s exceptionally clever, and compared to existing VR headsets it’s probably one of the best headtracking solutions out there.

With VR headsets, the problem isn’t putting two displays in front of the user’s eyes. The problem is determining where the user is looking quickly and accurately. IMUs and image processing techniques can be used with varying degrees of success, but to do it right, it needs to be really fast and really cheap.

[Alan] and [Valve]’s ‘Lighthouse’ tracking unit does this by placing a dozen or so IR photodiodes on the headset itself. On the tracking base station, IR lasers scan in the X and Y axes. By scanning these IR lasers across the VR headset, the angle of the headset to the base station can be computed in just a few cycles of a microcontroller. For a bunch of one cent photodiodes, absolute angles and the orientation to a base station can be determined very easily, something that has some pretty incredible applications for everything from VR to robotics.

Remember all of the position tracking hacks that came out as a result of the Nintendo Wii using IR beacons and a tracking camera? This seems like an evolutionary leap forward but in the same realm and can’t wait to see people hacking on this tech!

42 thoughts on “An Introduction To Valve’s Tracking Hardware

  1. That’s very clever, but the problem I can see here is the lack of standards in vr can make the developer’s life a living hell. I can see some clever person making bucketloads of cash by creating a unified vr api, which can detect what headset you are using and go from there.

      1. This is also what valve is doing with the SteamVR api, which will not only support the HTC vive, but also “all” other vr headsets for computers. For mobile the “Works for Google Cardboard” programme can help with one unified api (not including gear vr though, which has their own. Point is that t standards are on their way, and if you plan to work for pc the SteamVR api is already there :)

    1. This is also what valve is doing with the SteamVR api, which will not only support the HTC vive, but also “all” other vr headsets for computers. For mobile the “Works for Google Cardboard” programme can help with one unified api (not including gear vr though, which has their own. Point is that t standards are on their way, and if you plan to work for pc the SteamVR api is already there :)

    2. In the AR realm at least Microsofts taken the early lead on that – apparently the Hololens is just tapped into native Windows 10 functionality. Microsoft said they want other developers making HMDs for their OS, which can tap into the same functionality.
      For example, much like many different pointer devices can fire a “click” action in older Windows versions, potentially many different devices can fire a click 3D click in space using Hololens. – and applications built for W10 could understand that 3d click regardless of the handgesture/stylus/whatever used to make it.

      Sure, its still Microsoft so its still going to be pretty restrictive – but compared to other devices out there it seems a solid step towards a “standard”.
      Hopefully eventually HMDs will be as interchangeable as monitors are. No one worries about software compatibility with their 2D output device, they shoudn’t have to with their 3D either.

    1. Ah, looks like it is. That’s pretty slick! Basically flashes the LED array for sync and then can compute position and orientation based on the time the various photosensors sense the laser. I’m sort of curious as to how it can differentiate between multiple units in one room. Maybe the sync pulse is encoded with a sort of ID?

  2. Correct me if I’m wrong, but as far as I’ve understood the wall units are the ir laser emitters and it is the sensors on the headset itself which does the measuring, thus allowing the headset to read many of the wall units, even when they are placed in multiple rooms (i.e. walking from one room to the other with a set of wall units in each.

    In the article you make it sound like the headset emit the light and the wall units do the measuring.

      1. What do you mean you misread? Your description is accurate, the lasers sweeps, the diodes read the laser and the timing signals of the received pulse gives location info..And in fact valve actually mentioned the addition of more sweep units during a presentation I think to recall hearing from a video I saw.

    1. That is how CastAR works. The cheap emiters are located on the walls and the expensive camera sensors to track the head’s pose in the glasses so the latency is minimal and there is no problem to put the cheap tracking emitters in every room.

      The Valve system is just the opposite. You would need to put cameras in every room to be able to freely roam between them.and deal with the increased latency.

      1. Are the CastAR markers modulated? Otherwise it would be a bit hard to determine what visible marker corresponded to what known marker outside of the more limited single constellation use case.

  3. Ok so having read it and some related articles (and not able to edit my reply) wow, gotta love how incredibly simple it actually is, that’s some smart stuff.

  4. Made the same mistake here on HaD in a comment some time ago regarding the Valve system, but corrected it: The lasers do not ‘scan’ across the objects/room but ‘sweep’, the scanning is done by the diodes after all.

    It might seem trivial but I see so much confusion all over as people more and more misrepresent things by mistake leading to others thinking wrongly they know how it works and causing a lot of annoyance later on when discussing these things. And I don’t just mean the Valve system but many technical things.

    1. This battle was probably fought when people started to use the word scan not just for the detection of images in CRTs but also for it’s reproduction … and the battle was lost.

      1. I think the fight continues, it’s a recoverable thing and it only takes a little effort by a select few writers/editors to remember to say sweep and not scan.

        There have been other such battles that seemed lost or stalemate for years suddenly turn around and be won.

  5. Hmm, how can we do this with an arduino? With 32ms for an x sweep, that is 60Hz for both x and y. At 16Mhz, that lets us use a lot of cycles (512000) of code per x and y sweep of the “laser”. Red Laser diodes are only $0.25.

        1. A robot positioning app probably doesn’t have the speed and latency requirements this Valve hardware addresses. Your app would be far simpler with just a camera on the robot and a few dozen LED beacons scattered around the house: far, far simpler, and no moving parts. And latency isn’t that bad either.

          1. You’re right, but I’m thinking this setup would actually be cheaper: no need to process an image (ie use a cheap msp micro), just some photodiodes using low power on the robot.

            Beacons…how would a camera get an ID from a beacon ir signal? I think the frame grabs would be too slow for sending pulse data. But I’m intrigued!

          2. With modest constraints you can determine position and orientation of a camera looking at as few as just four unmodulated beacons in a single image frame, and update at video frame rate. This has been done in real time on consumer-grade webcams since before 1995 (yes, they existed then) — one guy in particular at Waterloo (his name escapes me at the moment) did some fantastic work around that time in this field. There was also a huge amount of work at CMU for autonomous indoor position determination with beacons for robots, AR and the Ubiquitous Computing movement, though they tended to throw more hardware at the problem.

            You can add modulation on the beacons to improve detectability, increase range, reduce ambiguity, add redundancy, transmit information, etc., but that’s not necessary. If a room ID or beacon disambiguation is required, you can easily transmit several bits per second per beacon even without frame synchronization. A robot might need to stop a second or two for a fix in that case.

          3. the system won’t let me reply to Paul below.

            Ok, the couple seconds is what I was thinking to do a beacon ID. Thanks for the response.
            I think a swarm of cheap bots could use a single sweeping IR laser and not use a lot of power (vi and cpu cycles).

            The camera tech will win in the long run because of less setup /wires and free cpu cycles to spare. (And no need for artificial ir becons with opencv on future cheap micro controllers).

    1. The lasers should not be visible though, it would drive you nuts, hence the IR laser. And the laser also must have a certain output, since it needs to be clearly detected and distinguishable from other light sources and it’s sweeping and spread out so its power is spread out too and the moment of detection is short.

    2. In an earlier post a few weeks back i was mentioning that for hackers on HaD piggybacking on the Valve sweeps might be interesting. Since you already paid for and installed the laser sweep modules you might as well use the system in-place and simply use that for some home-made experiments.
      Apart from the detection system maybe you can also use it to get 3D scans of objects placed in front if you were to place a camera without IR filter on top of the laser unit for instance? I know people used linelasers to get 3D scans done before, but I’m not sure the speed won’t be too high with the Valve thing.
      Anyway, it might be interesting to see what people will come up with.

      1. The ir diodes aren’t just going to flag a gpio/hw interrupt line. (They always receive some amount of IR light). So I was thinking to have the micro do the analysis of the modulated ir light per angle of sweep (the modulation would need to be slower than the valve version though).

        Or were you thinking that separate circuits would filter/bias the ir signal and those circuits could also trigger an interrupt?

  6. Please tell me the IR lasers are below the power level at which, should the sweeping mechanism fail (or get turned off by some glitch/malware), the lasers can’t blind you, even if you stare at them.

    1. First of all it’s a line laser, so that spreads out the danger, and secondly it’s trivial really to detect a stall and have it switch off. And Valve is a commercial outfit that has people dedicated to not getting sued. Also, Valve isn’t a car company (those don’t seem to give a damn really we found out the last few years.)

      Personally I trust they thought of such things.

Leave a Reply to PointyOintmentCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.