[Jeri] Spills The Beans On Her AR Glasses


In the last year, [Jeri Ellsworth] has been very busy. She was hired by Valve, started development of an augmented reality system, fired by Valve, and started a new company with [Rick Johnson] to bring her augmented reality glasses to the market. On the last Amp Hour podcast she spilled the beans on what went down at Valve, how her glasses work, and what her plans for the future are.

[Jeri] and [Rick]’s castAR glasses aren’t virtual reality glasses like the Oculus Rift or other virtual reality glasses that cut you off from the real world. The castAR glasses preserve your peripheral vision by projecting images and objects onto a gray retro-reflective mat and allows you to interact with a virtual environment with an electronic wand. So far, there are a few demos for the castAR system; a Jenga clone, and a game of battle chess called Team For Chess, a wonderful reference to Valve’s hat simulator.

The electronics inside the castAR glasses are fairly impressive; new frames are drawn on the retro-reflective surface at 100 Hz, positioning accuracy is in the sub-millimeter range, and thanks to [Jeri]’s clever engineering the entire system should be priced at about $200. Not too bad for an awesome device that can be used not only for D&D and Warhammer, but also for some very cool practical applications like visualizing engineering models of 3D prints before they’re printed.

52 thoughts on “[Jeri] Spills The Beans On Her AR Glasses

    1. you are still blind from the world in front of you with this. google glass is transparent. this is better the most VG units in that you have peripheral vision but you are still blind to the world in front.

  1. Jeri! We Love you! Despite Valve’s “great clensing” of their hardware department they were all bat shit crazy to let you go! This project is awesome stick with it and we will back ya! <33

    1. I assume these retro reflective surfaces are Scotchlite(tm) type material which is a bunch of tiny glass spheres embedded in the surface. Can you get more than one pixel per sphere reflected and can it reflect beams that are not focused at infinity. I was wondering if it could be used in an enclosed headset, but with one projector focused at infinity for distance where parallax provides depth cueing and two focused at maybe half as meter where binocular vision and focus can provide depth.

      1. Its a retro reflector so all issues to do with one pixel per sphere beomce redundant. It can be thought of as a mirror that is always oriented towards the incoming beam of light. Thereby reflecting it straight back.

        1. The one pixel per sphere problem is relevent with a series of retro reflective spheres because a retro reflector swaps the positions of entering light rays. If two or more pixels’ light enters one sphere, those two pixels will swap positions. So it’s important that the spheres be small enough that the border between pixels, where the two inputs swap places, be too small to notice.

    2. How is the patent situation? I don’t say this often, but this idea deserves one … did the researcher just throw it out there without patenting, does Jeri’s company own it or is it licensed for a cost which won’t make these glasses boutique regardless of Jeri’s engineering?

    3. I wonder if the retro-reflective surface also participates in the motion detection. Some sort of optical Fourier transform using light from the projectors, then extract the phase information for motion?

    1. Indeed, but the “screen” is orientated as if laid out flat on a coffee table (in most cases I suspect). Multiple gamers can use the same screen (sitting in a circle around the coffee table) but with an image that is unique to each person looking at that screen (the retro-reflective material ensures that each player sees only his/her personalised image). A third party without the glasses would see nothing meaningful when they look at the screen.

      I can’t wait to see this tech become available, I think it’ll be a huge success. The sub-millimetre viewpoint headtracking will be a game changer (please excuse the pun).

    2. Plus if you were clever you could probably generate a pretty realistic stereo image hologram-style with a device like this. The illusion would break down at the edge of the mat, but I think the overall coolness would easily win out.

      I’d love to play LoL on a 3d millenium falcon mat.

    1. My best guess would be that they just had too many engineers on the payroll, and since they didn’t have a use for the AR project themselves, decided cutting that project and the associated personnel would be the easy way to reduce head count.

    2. It happens, nobody’s fault really. Except maybe upper management for hiring them in the first place. But even them you can’t blame because sometimes they make a bad guess or prediction.

          1. Either don’t or doesn’t could be used correctly depending on what you mean by “valve”.

            “Doesn’t” is correct if you mean valve as a singular corporate entity, but “don’t” is correct if “valve” is being used to refer to the people who make up the corporation.

          2. I think his point was that Valve don’t have *any* management, not just upper. I don’t know anything about Valve, but I think that’s what he was pointing out, not trying to correct the they / it grammar thingy.

    3. If you look closer, it was a layoff.
      Firing often has a connotation of person having dome something wrong; laid off/let go/etc. is everything else (you may be good at what you do but [there’s no business|project is cancelled|we don’t have enough money to keep you|etc.])
      Editor might want to change the article to not say “fired” as that could be considered slander.

  2. I also can’t find any negative-criticism/”trolling”/”hating” for this. The price seems realistic for something with such software features and hardware, and it seem efficient.

    1. no, VR touch screen would be perfect for this. e.g. View VR object,
      VR touch manipulation of VR object, e.g. rotate; move from left side of viewing to right side of viewing; push those VR buttons; rotate the
      VR knobs for volume, brightness, etc (vs. touch the glass screen).

    2. I’d make some type of pointer with buttons and a camera in it. The pointer can use the same IR LEDS as the glasses use to see where it’s pointed at … then the system can create a virtual representation of whatever type of manipulator you want where you point at (a croupier stick or a disembodied hand for instance).

  3. Glasses and bitmap tracking are probably as advance as interfaces will get till someone with and imagination AND intelligence figures out how to control photons and molecules to track interaction with air molecules.. 3D air displays

  4. My definition of a bad day: You’ve just spent the last 6 months working on a new type of augmented reality glasses when you hear that Jeri Ellsworth is doing the same thing.

  5. I tried these out at MakerFaire a few weeks ago, and wasn’t very impressed. Did anyone else try them and come away with a more positive impression? If you did, please share it! I’d really like to like these glasses, but compared to the Rift they’re a much worse experience.

  6. Same here, it sounds intriguing to us FPS fans.

    I expect that a lot of medical applications exist, such as projecting 3D proteins etc.
    Make the whole thing a lot smaller and add a few more projectors and build a real life Holodeck :-)

  7. Hello Tony Stark style prototyping, albeit with glasses for now. But these babies mixed with some great motion/gesture tracking would be awesome. Maybe if the EEG (I hope I’m correct in that) thought interpretation catches up you you throw that in the mix as well, as an intuitive input system. I see the future that I imagine getting closer every day; and it gives me a warm, happy feeling inside. Plus I loved Daemon so these fit pretty well into that scenario.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.