Millimeter Wave RADAR Tracks Gestures

If we believe science fiction — from Minority Report to Iron Man, to TekWar — the future of computer interfaces belongs to gestures. There are many ways to read gestures, although often they require some sort of glove or IR emitter, which makes them less handy (no pun intended).

Some, like the Leap Motion, have not proved popular for a variety of reasons. Soli (From Google’s Advanced Technology and Projects group) is a gesture sensor that uses millimeter-wave RADAR. The device emits a broad radio beam and then collects information including return time, energy, and frequency shift to gain an understanding about the position and movement of objects in the field. You can see a video about the device, below.

You naturally think of using optical technology to look at hand gestures (the same way humans do). However, RADAR has some advantages. It is insensitive to light and can transmit through plastic materials, for example. The Soli system operates at 60 GHz, with sensors that use Frequency Modulated Continuous Wave (FMCW) and Direct-Sequence Spread Spectrum (DSSS). The inclusion of multiple beamforming antennas means the device has no moving parts.

Clearly, this is cutting-edge gear and not readily available yet. But the good news is that Infineon is slated to bring the sensors to market sometime this year. Planned early applications include a smart watch and a speaker that both respond to gestures using the technology.

Interestingly, the Soli processing stack is supposed to be RADAR agnostic. We haven’t investigated it, but we wonder if you could use the stack to process other kinds of sensor input that might be more hacker friendly? Barring that, we’d love to see what our community could come up with for solving the same problem.

We’ve seen Raspberry Pi daughter-boards (ok, hats) that recognize gestures used to control TVs. We’ve even built some crude gesture sensing using SONAR, if that gives you any ideas. Are you planning on using Soli? Or rolling your own super gesture sensor? Let us know and document your project for everyone over on Hackaday.io.

43 thoughts on “Millimeter Wave RADAR Tracks Gestures

  1. At my company we’ve had Soli in our sights for quite some time waiting for it to be released.

    It is very interesting the concepts google has already shown such as detecting material types and we can’t wait to get our hands on some.

    1. I wouldn’t worry. I spend most of the day in a lab with low-power RF sources sweeping between 10-90GHz.

      Everything will kill you eventually, everything causes cancer, and everything is unhealthy. Sunlight, water, oxygen, carbon, etc.

      1. Reduced transmit power is also your friend

        After all, wifi runs in the same frequency band as a microwave oven – but even if you dumped ALL of the power from a typical consumer-grade AP into your hand – you’d at most make it warm and maybe uncomfortable.

        Typical consumer-grade WLAN equipment is <100 mW

        That's less than if you just held a 1/4 watt resistor being pushed to its maximum rated power in your fingertips.

          1. Why would anyone do that? Seems like a waste of a perfectly good resistor.

            RF, like almost all other energy sources, “cook” from the outside in. It doesn’t have any effect your brain without also having an effect all the millions of nerve endings in your scalp that would cause you to involuntarily stop poking yourself with the antenna.

    2. “irradiation for 6 min led to an elevation of the corneal surface temperature (reaching 54.2 +/- 0.9 degrees C)”

      yes, i am fairly certain taking a torch to your eyes causes damage too.

      should be just fine at lower transmit power.

  2. Does anyone know of good references explaining using DSSS for radar purposes?

    I understand why it is used in communications systems but fail to see how radar profits from it?

    1. Google search for “pulse compression” is probably your best bet.

      DSSS is one of the less common approaches, but sometimes used for interference mitigation or, in military systems, to facilitate low observability.

      Cornell used a long PRNG code when mapping Venus from Arecibo – 8192 bits long I think? It’s been nearly two decades since my radar and antennas class.

  3. I like that it could be affordable (for some variable of ‘affordable’). The rest is reading up on all those math-heavy research papers on what to do with the output of those sensors.

      1. Sure, but his point is valid. Haptic feedback is already missing partially in touch screens, especially in e.g. a moving car. You can interact with a physical button in a much more controlled way. Trying to control something by motion in thin air is again more difficult.

  4. HaD has featured Soli a few times before. I never can help noticing all the participants in the video have hand jewelry when being interviewed (wedding rings, bracelets, watches, etc), but none of the demonstration hands have any. I wonder how well the delicate gesture sense works with the waves being scattered and refracted by small pieces of surrounding metal?

  5. Coolness factors on this very HIGH
    I’ve seen other similar applications but not exactly like this,
    From looking at this, it tells me this is capable of applications other than gestures. Such as detecting object shapes, not just in 2D, but maybe 3D shapes too. It seems that is capable of generating a signature pattern for different shapes.
    But I can see already one issue that is lacking. Just as with any motion sensor, if you stand still, that sensor won’t know the difference. As the hand in the demonstration. So in any situation you need to stimulate the sensor by moving first. Which is why vision recognition will still have it’s merits.
    But I still love to get one of these to experiment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s