Sense All The Things With A Synthetic Sensor

What will it take to make your house smarter than you? Judging from the price of smart appliances we see in the home centers these days, it’ll take buckets of cash. But what if you could make your home smarter — or at least more observant — with a few cheap, general purpose “supersensors” that watch your every move?

Sounds creepy, right? That’s what [Gierad Laput] and his team at the Carnegie Mellon Human-Computer Interaction Institute thought when they designed their broadband “synthetic sensor,” and it’s why they purposely omitted a camera from their design. But just about every other sensor under the sun is on the tiny board: an IR array, visible light sensors, a magnetometer, temperature, humidity, and pressure sensors, a microphone, PIR, and even an EMI detector. Of course there’s also a WiFi module, but it appears that it’s only for connectivity and not used for sensing, although it clearly could be. All the raw data is synthesized into a total picture of the goings on in within the platform’s range using a combination of machine learning and user training.

The video after the break shows the sensor detecting typical household events from a central location. It’s a powerful idea and we look forward to seeing how it moves from prototype to product. And if the astute reader recognizes [Gierad]’s name, it might be from his past appearance on these pages for 3D-printed hair.

[muA] tipped us off on this one. Thanks!

44 thoughts on “Sense All The Things With A Synthetic Sensor

    1. As already mentioned there are sensors for specific molecules, for something general purpose you would need something like a mass spectrometer or ion mobility spectrometer. There has also been a class of chemical sensors that use the standing acoutic wave effect. I also read about a MEMS chemo sensor with piezo tuning fork elements treated with various reactants but I don’t think it has made it into commercial production.

      1. Electronic nose technology makes use of the fact that most chemical sensors are nonspecific, but respond differently to different components of an odor. Since we patented it in 1987, it’s been picked up by the wine industry and parts of the food industry, and is being validated for use in detection of cancer.

    1. Actually, it almost could with just one. Notice somewhere in the video where it detected the left and right faucets being used. If their software is as easy to use, registering appliance/tools should be as easy as recording the sensor-signature it makes, and label them as such. A row of faucets would sound the same, but the level of noise those make are different depending on it’s distance to the sensor. Theoretically, if the sensor is sensitive enough it should be able to cover the whole house.

    2. I would even say that with 3 sensors you can use triangulation, and with 4 it gives you the ability to find the height. After that the more you have, the better!

    1. I would be less concerned about them being tricked and more concerned about them being confused. However, with enough training it could do a pretty good job… but it requires a lot of training. :/

  1. When it was detecting the toothbrush it reminded me of the scene in Electric dreams when Edgar ( the computer) was trying to work out what the noise in the bathroom was ( electric razor)

    Is this thing going to lock me out of my house and try and steal my girlfriend :o

  2. It looks like an Industrial Psychologist’s dream.
    But would it be good for workers? Or could it manufacture a future of Tesla and Amazon wageslaves driven to exhaustion?
    Also, with the IOT, I worry about eventually non-updating devices connected to the internet which are powerful enough to cause harm if they’re pwnd and botnetted. We should always see a compromise as when, not if.

    1. Out of curiosity, what would you feel about setting something up similar via POE hardwired? That way the signal cannot be scraped, and you can choose how much info is release outside of the LAN or firewall.

      1. I’m working on something similar, running 1wire over Cat5/6. The sensor boards I have designed have temperature, humidity, 2 x PIR, IR TX/RX, Audio (locally stored chimes only, not much bandwidth available), light, air quality (MQ135), carbon monoxide (MQ7), and for bathrooms, hydrogen sulphide (MQ136 – these sensors cost a bit so they are optional). I’ll be open sourcing the boards, firmware, enclosures & submitting OWFS patches upstream shortly. I’ll post a project to Hackaday when it’s all ready.

  3. Cool. Very cool. I see it on battlefield. Couple of them will make picture of the area. Computer will process the data, make picture, AI analyze situation and propose strategies for commanders. Its compatibile with US strategy of giving more responsibility for lower levels of command. They need “eyes”.

  4. Detecting/identifying location of people and animals in the house should be pretty easy based on footstep vibration in the floor. In my house the floors squeak so it would be especially easy.

    In terms of training, this is the first compelling use for IOT I have seen.

    This is a great example of truly disruptive technology. I bet the refrigerator manufacturers are damned embarrassed that the best they could do was an app where you could tell the fridge to make more ice.

    1. Integrate all this into a new apartment complex… The building calls the cops if the music is too loud. Tracks the coming and going of tenants. The kitchen knows when you cook fish and your mailbox is submitted for targeted ads plus informs you when your mailbox is full. The plumbing bill for that clogged sewer is directed the the apartment that caused it. Mice and cockroaches are detected and the exterminator called. Your physician and employer can check your sleep patterns the picked up by the floor sensors. Grandma fell down and broke her hip, the ambulance is called.

      I like it but also see it’s 1984 on steroids. We’ll have to protect privacy rights starting right from the beginning…. which is now.

Leave a Reply to JoeCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.