In a world where smartphones have commoditized precision MEMS Sensors, the stage is set to reimagine clusters of these sensors as something totally different. That’s exactly what [chronopoulos] did, taking four proximity sensors and turning them into a custom gesture input sensor for sound generation. The result is Quadrant, a repurposable human-interface device that proves to be well-posed at detecting hand gestures and turning them into music.
At its core, Quadrant is a human interface device built around an STM32F0 and four VL6180X time-of-flight proximity sensors. The idea is to stream the measured distance data over as fast as possible from the device side and then transform it into musical interactions on the PC side. Computing distance takes some time, though, so [chronopoulos] does a pipelined read of the array to stream the data into the PC over USB at a respectable 30 Hz.
With the data collected on the PC side, there’s a spread of interactions that are possible. Want a laser harp? No problem, as [chronopoulos] shows how you can “pluck” the virtual strings. How about an orientation sensor? Simply spread your hand over the array and change the angle. Finally, four sensors will also let you detect sweeping gestures that pass over the array, like the swoosh of your hand from one side to the other. To get a sense of these interactions, jump to the video demos at the 2:15 mark after the break.
If you’re curious to dig into the project’s inner workings, [chronopoulos] has kindly put the firmware, schematics, and layout files on Github with a generous MIT License. He’s even released a companion paper [PDF] that details the math behind detecting these gestures. And finally, if you just want to cut to the chase and make music of your own, you can actually snag this one on Tindie too.
MEMs sensors are living a great second life outside our phones these days, and this project is another testament to the richness they offer for new project ideas. For more MEMs-sensor-based projects, have a look at this self-balancing robot and magic wand.
I really like this project… Seems like there is a huge amount you can do with that magic sensor setup, it has certainly given me some ideas.
That said I think I’d still rather have a real Theremin or Tannerin (think that is how its spelt – very similar to a theremin but slightly more predictable to play I understand) there is something satisfying about it not being computerised or digitised in any way.
Offtopic:
I have just discovered the fact that the author names and author entries into the HAD
website is not alike.
Case in point the author for this article.
https://hackaday.com/author/doublejumpelectric/
Goodbye next 30 minutes as I look at all the authors…
The credit card shape seems a poor fit to a control surface of 4 variables like a joystick. It would be good to control sounds or effects but is not a musical instrument in and of itself. Old Baldwin organs had variable level under each key and careful playing could make those spacey sounds ages ago.
@Joshua Vasquez said: “MEMs sensors are living a great second life outside our phones these days, and this project is another testament to the richness they offer for new project ideas.”
The VL6180X Proximity and Ambient Light Sensing (ALS) Module from STMicroelectronics [1] is not a MEMS (Microelectromechanical Systems) [2] device, it is a Solid-State device with one IR (850 nm Infrared) Vertical-Cavity Surface-Emitting Laser (VCSEL) [3] and two sensors, one IR sensor for Time-of-Flight (ToF) [4] object proximity, and one ambient light sensor.
* References:
1. VL6180X Proximity and Ambient Light Sensing (ALS) Module @ STMicroelectronics
https://www.st.com/en/imaging-and-photonics-solutions/vl6180x.html
2. Microelectromechanical Systems
https://en.wikipedia.org/wiki/Microelectromechanical_systems
3. Vertical-Cavity Surface-Emitting Laser (VCSEL)
https://en.wikipedia.org/wiki/Vertical-cavity_surface-emitting_laser
4. Time of Flight (ToF)
https://en.wikipedia.org/wiki/Time_of_flight
https://www.youtube.com/watch?v=DnAfXK-hft8
Jarre’s laser harp was the primary inspiration for the second demo patch (plucking). one feature that time-of-flight sensors add to the experience is the ability to detect the *height* at which the beam is crossed. so, like on a real harp, you can map the plucking position along the virtual string to a tonal parameter like brightness or volume.
Nice! I love anything that operates hands-free. Some of the people I work with are so dirty! You can’t be too careful about germs these days.
Maybe someone would be interested by this project: https://youtu.be/ohGsdv1wRZk
I saw this in person a week ago and was instantly reminded of the NES U-Force accessory.
There’s a great YouTube documentary about it by The Gaming Historian: https://www.youtube.com/watch?v=EVszKSeSE-0
thanks for the write-up, Joshua! to those interested in obtaining a quadrant through the Tindie store: i’m currently sold out of inventory, but just started another manufacturing run, so i should be able to get to those on the waitlist over the next 2-3 weeks. thanks for your patience!