Hey Google, Is My Heart Still Beating?

University of Washington researchers studying the potential medical use of smart speakers such as Amazon’s Echo and Google’s Nest have recently released a paper detailing their experiments with non-contact acoustic heartbeat detection. Thanks to their sensitive microphone arrays, normally used to help localize voice commands from the user, the team proposes these affordable and increasingly popular smart home gadgets could lead a double life as unobtrusive life sign monitors. The paper goes so far as to say that even with multiple people in the room, their technique can be used to monitor the heart and respiratory rate of a specific target individual.

Those are some bold claims, but they aren’t without precedent. Previous studies performed at UW in 2019 demonstrated how smart speaker technology could be used to detect cardiac arrest and monitor infant breathing. This latest paper could be seen as the culmination of those earlier experiments: a single piece of software that could not just monitor the vitals of nearby patients, but actually detect a medical emergency. The lifesaving potential of such a program, especially for the very young and elderly, would be incredible.

So when will you be able to install a heart monitor skill on the cheap Echo Dot you picked up on Prime Day? Well, as is often the case with this kind of research, putting the technique to work in the real-world isn’t nearly as easy as in the laboratory. While the concept is promising and is more than worthy of further research, it may be some time before our lowly smart speakers are capable of Star Trek style life-sign detection.

Continue reading “Hey Google, Is My Heart Still Beating?”

Amazon Echo Gets Open Source Brain Transplant

There’s little debate that Amazon’s Alexa ecosystem makes it easy to add voice control to your smart home, but not everyone is thrilled with how it works. The fact that all of your commands are bounced off of Amazon’s servers instead of staying internal to the network is an absolute no-go for the more privacy minded among us, and honestly, it’s hard to blame them. The whole thing is pretty creepy when you think about it.

Which is precisely why [André Hentschel] decided to look into replacing the firmware on his Amazon Echo with an open source alternative. The Linux-powered first generation Echo had been rooted years before thanks to the diagnostic port on the bottom of the device, and there were even a few firmware images floating around out there that he could poke around in. In theory, all he had to do was remove anything that called back to the Amazon servers and replace the proprietary bits with comparable free software libraries and tools.

Taping into the Echo’s debug port.

Of course, it ended up being a little trickier than that. The original Echo is running on a 2.6.x series Linux kernel, which even for a device released in 2014, is painfully outdated. With its similarly archaic version of glibc, newer Linux software would refuse to run. [André] found that building an up-to-date filesystem image for the Echo wasn’t a problem, but getting the niche device’s hardware working on a more modern kernel was another story.

He eventually got the microphone array working, but not the onboard digital signal processor (DSP). Without the DSP, the age of the Echo’s hardware really started to show, and it was clear the seven year old smart speaker would need some help to get the job done.

The solution [André] came up with is not unlike how the device worked originally: the Echo performs wake word detection locally, but then offloads the actual speech processing to a more powerful computer. Except in this case, the other computer is on the same network and not hidden away in Amazon’s cloud. The Porcupine project provides the wake word detection, speech samples are broken down into actionable intents with voice2json, and the responses are delivered by the venerable eSpeak speech synthesizer.

As you can see in the video below the overall experience is pretty similar to stock, complete with fancy LED ring action. In fact, since Porcupine allows for multiple wake words, you could even argue that the usability has been improved. While [André] says adding support for Mycroft would be a logical expansion, his immediate goal is to get everything documented and available on the project’s GitLab repository so others can start experimenting for themselves.

Continue reading “Amazon Echo Gets Open Source Brain Transplant”

“Alexa, Stop Listening To Me Or I’ll Cut Your Ears Off”

Since we’ve started inviting them into our homes, many of us have began casting a wary eye at our smart speakers. What exactly are they doing with the constant stream of audio we generate, some of it coming from the most intimate and private of moments? Sure, the big companies behind these devices claim they’re being good, but do any of us actually buy that?

It seems like the most prudent path is to not have one of these devices, but they are pretty useful tools. So this hardware mute switch for an Amazon Echo represents a middle ground between digital Luddism and ignoring the possible privacy risks of smart speakers.  Yes, these devices all have software options for disabling their microphone arrays, but as [Andrew Peters] relates it, his concern is mainly to thwart exotic attacks on smart speakers, some of which, like laser-induced photoacoustic attacks, we’ve previously discussed. And for that job, only a hardware-level disconnect of the microphones will do.

To achieve this, [Andrew] embedded a Seeeduino Xiao inside his Echo Dot Gen 2. The tiny microcontroller grounds the common I²S data line shared by the seven (!) microphones in the smart speaker, effective disabling them. Enabling and disabling the mics is done via the existing Dot keys, with feedback provided by tones sent through the Dot speaker. It’s a really slick mod, and the amount of documentation [Andrew] did while researching this is impressive. The video below and the accompanying GitHub repo should prove invaluable to other smart speaker hackers.

Continue reading ““Alexa, Stop Listening To Me Or I’ll Cut Your Ears Off””

Stay Smarter Than Your Smart Speaker

Smart speakers have always posed a risk to privacy and security — that’s just the price we pay for getting instant answers to life’s urgent and not-so-urgent questions the moment they arise. But it seems that many owners of the 76 million or so smart speakers on the active install list have yet to wake up to the reality that this particular trick of technology requires a microphone that’s always listening. Always. Listening.

With so much of the world’s workforce now working from home due to the global SARS-CoV-2 pandemic, smart speakers have suddenly become a big risk for business, too — especially those where confidential conversations are as common and crucial as coffee.

Imagine the legions of lawyers out there, suddenly thrust from behind their solid-wood doors and forced to set up ramshackle sub rosa sanctuaries in their homes to discuss private matters with their equally out-of-sorts clients. How many of them don’t realize that their smart speaker bristles with invisible thorns, and is even vulnerable to threats outside the house? Given the recent study showing that smart speakers can and do activate accidentally up to 19 times per day, the prevalence of the consumer-constructed surveillance state looms like a huge crisis of confidentiality.

So what are the best practices of confidential work in earshot of these audio-triggered gadgets?

Continue reading “Stay Smarter Than Your Smart Speaker”

Smart Speakers “Accidentally” Listen Up To 19 Times A Day

In the spring of 2018, a couple in Portland, OR reported to a local news station that their Amazon Echo had recorded a conversation without their knowledge, and then sent that recording to someone in their contacts list. As it turned out, the commands Alexa followed came were issued by television dialogue. The whole thing took a sitcom-sized string of coincidences to happen, but it happened. Good thing the conversation was only about hardwood floors.

But of course these smart speakers are listening all the time, at least locally. How else are they going to know that someone uttered one of their wake words, or something close enough? It would sure help a lot if we could change the wake word to something like ‘rutabaga’ or ‘supercalifragilistic’, but they probably have ASICs that are made to listen for a few specific words. On the Echo for example, your only choices are “Alexa”, “Amazon”, “Echo”, or “Computer”.

So how often are smart speakers listening when they shouldn’t? A team of researchers at Boston’s Northeastern University are conducting an ongoing study to determine just how bad the problem really is. They’ve set up an experiment to generate unexpected activation triggers and study them inside and out.

Continue reading “Smart Speakers “Accidentally” Listen Up To 19 Times A Day”

Wearable Cone Of Silence Protects You From Prying Ears

Careful,  the walls have ears. Or more specifically, the smart speaker on the table has ears, as does the phone in your pocket, the fitness band on your wrist, possibly the TV, the fridge, the toaster, and maybe even the toilet. Oh, and your car is listening to you too. Probably.

How does one fight this profusion of listening devices? Perhaps this wearable smart device audio jammer will do the trick. The idea is that the MEMS microphones that surround us are all vulnerable to jamming by ultrasonic waves, due to the fact that they have a non-linear response to ultrasonic signals. The upshot of that is when a MEMS hears ultrasound, it creates a broadband signal in the audible part of the spectrum. That creates a staticky noise that effectively drowns out any other sounds the microphone might be picking up.

By why a wearable? Granted, [Yuxin Chin] and colleagues from the University of Chicago have perhaps stretched the definition of that term a tad with their prototype, but it turns out that moving the jammer around does a better job of blocking sounds than a static jammer does. The bracelet jammer is studded with ultrasonic transducers that emit overlapping fields and result in zones of constructive and destructive interference; the wearer’s movements vary the location of the dead spots that result, improving jamming efficacy. Their paper (PDF link) goes into deeper detail, and a GitHub repository has everything you need to roll your own.

We saw something a bit like this before, but that build used white noise for masking, and was affixed to the smart speaker. We’re intrigued by a wearable, especially since they’ve shown it to be effective under clothing. And the effect of ultrasound on MEMS microphones is really interesting.

Continue reading “Wearable Cone Of Silence Protects You From Prying Ears”

Building A Smart Speaker From Scratch

Smart speakers have proliferated since their initial launch earlier this decade. The devices combine voice recognition and assistant functionality with a foreboding sense that paying corporations for the privilege of having your conversations eavesdropped upon could come back to bite one day. For this reason, [Yihui] is attempting to build an open-source smart speaker from scratch.

The initial prototype uses a Raspberry Pi 3B and a ReSpeaker microphone array. In order to try and bring costs down, development plans include replacing these components with a custom microphone array PCB and a NanoPi board, then implementing basic touch controls to help interface with the device.

There’s already been great progress, with the build showing off some nifty features. Particularly impressive is the ability to send WiFi settings to the device using sound, along with the implementation of both online and offline speech recognition capabilities. This is useful if your internet goes down but you still want your digital pal to turn out the lights at bed time.

It’s not the first time we’ve seen a privacy-focused virtual assistant, and we hope it’s not the last. Video after the break.

Continue reading “Building A Smart Speaker From Scratch”