Smart Assistants Need To Get Smarter

Science fiction has regularly portrayed smart computer assistants in a fanciful way. HAL from 2001: A Space Odyssey and J.A.R.V.I.S. from the contemporary Iron Man films are both great examples. They’re erudite, wise, and capable of doing just about any reasonable task that is asked of them, short of opening the pod bay doors.

Cut back to reality, and you’ll only be disappointed at how useless most voice assistants are. It’s been twelve long years since Siri burst onto the scene, with Alexa and Google Assistant following years later. Despite years on the market, their capabilities remain limited and uninspiring. It’s time for voice assistants to level up.

Continue reading “Smart Assistants Need To Get Smarter”

Why Did The Home Assistant Future Not Quite Work The Way It Was Supposed To?

The future, as seen in the popular culture of half a century or more ago, was usually depicted as quite rosy. Technology would have rendered every possible convenience at our fingertips, and we’d all live in futuristic automated homes — no doubt while wearing silver clothing and dreaming about our next vacation on Mars.

Of course, it’s not quite worked out this way. A family from 1965 whisked here in a time machine would miss a few things such as a printed newspaper, the landline telephone, or receiving a handwritten letter; they would probably marvel at the possibilities of the Internet, but they’d recognise most of the familiar things around us. We still sit on a sofa in front of a television for relaxation even if the TV is now a large LCD that plays a streaming service, we still drive cars to the supermarket, and we still cook our food much the way they did. George Jetson has not yet even entered the building.

The Future is Here, and it Responds to “Alexa”

An Amazon Echo Dot device
“Alexa, why haven’t you been a commercial success?” Gregory Varnum, CC BY-SA 4.0

There’s one aspect of the Jetsons future that has begun to happen though. It’s not the futuristic automation of projects such as Disneyland’s Monsanto house Of The Future, but instead it’s our current stuttering home automation efforts. We’re not having domestic robots in pinnies hand us rolled-up newspapers, but we’re installing smart lightbulbs and thermostats, and we’re voice-controlling them through a variety of home hub devices. The future is here, and it responds to “Alexa”.

But for all the success that Alexa and other devices like it have had in conquering the living rooms of gadget fans, they’ve done a poor job of generating a profit. It was supposed to be a gateway into Amazon services alongside their Fire devices, a convenient household companion that would help find all those little things for sale on Amazon’s website, and of course, enable you to buy them. Then, Alexa was supposed to move beyond your Echo and into other devices, as your appliances could come pre-equipped with Alexa-on-a-chip. Your microwave oven would no longer have a dial on the front, instead you would talk to it, it would recognise the food you’d brought from Amazon, and order more for you.

Instead of all that, Alexa has become an interface for connected home hardware, a way to turn on the light, view your Ring doorbell on models with screens, catch the weather forecast, and listen to music. It’s a novelty timepiece with that pod bay doors joke built-in, and worse that that for the retailer it remains by its very nature unseen. Amazon have got their shopping cart into your living room, but you’re not using it and it hardly reminds you that it’s part of the Amazon empire at all.

But it wasn’t supposed to be that way. The idea was that you might look up from your work and say “Alexa, order me a six-pack of beer!”, and while it might not come immediately, your six-pack would duly arrive. It was supposed to be a friendly gateway to commerce on the website that has everything, and now they can’t even persuade enough people to give it a celebrity voice for a few bucks.

The Gadget You Love to Hate

In the first few days after the Echo’s UK launch, a member of my hackerspace installed his one in the space. He soon became exasperated as members learned that “Alexa, add butt plug to my wish list” would do just that. But it was in that joke we could see the problem with the whole idea of Alexa as an interface for commerce. He had locked down all purchasing options, but as it turns out, many people in San Diego hadn’t done the same thing. As the stories rolled in of kids spending hundreds of their parents’ hard-earned on toys, it would be a foolhardy owner who would leave left purchasing enabled. Worse still, while the public remained largely in ignorance the potential of the device for data gathering and unauthorized access hadn’t evaded researchers. It’s fair to say that our community has loved the idea of a device like the Echo, but many of us wouldn’t let one into our own homes under any circumstances.

So Alexa hasn’t been a success, but conversely it’s been a huge sales success in itself. The devices have sold like hot cakes, but since they’ve been sold at close to cost, they haven’t been the commercial bonanza they might have hoped for. But what can be learned from this, other than that the world isn’t ready for a voice activated shopping trolley?

Sadly for most Alexa users it seems that a device piping your actions back to a large company’s data centres is not enough of a concern for them. It’s an easy prediction that Alexa and other services like it will continue to evolve, with inevitable AI pixie dust sprinked on them. A bet could be on the killer app being not a personal assistant but a virtual friend with some connections across a group of people, perhaps a family or a group of friends. In due course we’ll also see locally hosted and open source equivalents appearing on yet-to-be-released hardware that will condense what takes a data centre of today’s GPUs into a single board computer. It’s not often that our community rejoices in being late to a technological party, but I for one want an Alexa equivalent that I control rather than one that invades my privacy for a third party.

Photo of the Echo Dot PCB, highlighting the capacitor that needs to be shorted out for the exploit to work

Squeezing Secrets Out Of An Amazon Echo Dot

As we have seen time and time again, not every device stores our sensitive data in a respectful manner. Some of them send our personal data out to third parties, even! Today’s case is not a mythical one, however — it’s a jellybean Amazon Echo Dot, and [Daniel B] shows how to make it spill your WiFi secrets with a bit of a hardware nudge.

There’s been exploits for Amazon devices with the same CPU, so to save time, [Daniel] started by porting an old Amazon Fire exploit to the Echo Dot. This exploit requires tactically applying a piece of tin foil to a capacitor on the flash chip power rail, and it forces the Echo to surrender the contents of its entire filesystem, ripe for analysis. Immediately, [Daniel] found out that the Echo keeps your WiFi passwords in plain text, as well as API keys to some of the Amazon-tied services.

Found an old Echo Dot at a garage sale or on eBay? There might just be a WiFi password and a few API keys ripe for the taking, and who knows what other kinds of data it might hold. From Amazon service authentication keys to voice recognition models and maybe even voice recordings, it sounds like getting an Echo to spill your secrets isn’t all that hard.

We’ve seen an Echo hijacked into an always-on microphone before, also through physical access in the same vein, so perhaps we all should take care to keep our Echoes in a secure spot. Luckily, adding a hardware mute switch to Amazon’s popular surveillance device isn’t all that hard. Though that won’t keep your burned out smart bulbs from leaking your WiFi credentials.

Voice Without Sound

Voice recognition is becoming more and more common, but anyone who’s ever used a smart device can attest that they aren’t exactly fool-proof. They can activate seemingly at random, don’t activate when called or, most annoyingly, completely fail to understand the voice commands. Thankfully, researchers from the University of Tokyo are looking to improve the performance of devices like these by attempting to use them without any spoken voice at all.

The project is called SottoVoce and uses an ultrasound imaging probe placed under the user’s jaw to detect internal movements in the speaker’s larynx. The imaging generated from the probe is fed into a series of neural networks, trained with hundreds of speech patterns from the researchers themselves. The neural networks then piece together the likely sounds being made and generate an audio waveform which is played to an unmodified Alexa device. Obviously a few improvements would need to be made to the ultrasonic imaging device to make this usable in real-world situations, but it is interesting from a research perspective nonetheless.

The research paper with all the details is also available (PDF warning). It’s an intriguing approach to improving the performance or quality of voice especially in situations where the voice may be muffled, non-existent, or overlaid with a lot of background noise. Machine learning like this seems to be one of the more powerful tools for improving speech recognition, as we saw with this robot that can walk across town and order food for you using voice commands only.

Continue reading “Voice Without Sound”

Google Sheet showing wins and losses of sports team. Data automated by IFTTT, Alexa, and Particle

An Overly Complicated Method Of Tracking Your Favorite Sports Team

Much of the world appears to revolve around sports, and sports tracking is a pretty big business. So how do people keep up with their favorite team? Well, [Jackson] and [Mourad] decided to devise a custom IoT solution.

Their system is a bit convoluted, so bear with us. First, they tell Alexa whether or not the team won or lost that week. Alexa then sends that information to IFTTT where two different Particle Argon boards are constantly polling the results to decide how to respond next. One Particle responds by lighting up an LED, green for a win and red for a loss. Another Particle board displays the results on an LCD screen. But this is where things get tricky. One of the more confusing aspects of their design is one of the Particle boards then signals back to IFTTT, telling it to tally the number of wins and losses. This seems a bit roundabout since the system started with IFTTT in the first place. Regardless, they seemed to be happy with the result and I’m sure they learned something in the process.

This project might not fulfill any functional need given that Alexa knows everything about all our lives already and you could just ask her how your favorite team is doing whenever you want to. But hey, we’re all about learning by doing here at Hackaday and we’re all guilty of building useless projects here and there just because we can. In any case, their project could serve as a good intro to integrating your Particle with IFTTT or Alexa since there appears to be quite a bit of probably unnecessary handshaking going on here.

Continue reading “An Overly Complicated Method Of Tracking Your Favorite Sports Team”

Wall of video games

Consoles, Consoles On The Wall, Can Alexa Help Me Play Them All?

If you’ve got a collection of classic game consoles, finding the space to set them all up can be a challenge. But the bigger problem is figuring out how to hook them all up to a TV that, at best, might only have two or three inputs. [odelot] recently wrote in to tell us how he solved both problems with his voice-controlled wall of gaming history.

To mount the systems to the wall, [odelot] designed and printed angled brackets that attach to specially shaped pieces of 3 mm MDF. They do a pretty good job of holding the systems at a visually interesting angle while making themselves scarce, with only the notoriously slick-bottomed Wii needing some extra clips on the front to keep it from sliding off. He also printed up a series of blocks and pipes, no doubt a reference to Mario Bros., to hold the power and video cables for each system.

Prototype version of electronics on breadboardAs to connecting them all up to his TV, [odelot] picked up an eight-device Extron VGA switch that features a serial port for remote control. After getting all the systems adapted over to the appropriate video standard, he then wired an ESP8266 to the switch and wrote some code that ties it into Amazon’s Alexa voice assistant.

By just saying the name of the system he wants to play, the microcontroller will flick the switch over to the appropriate input and turn on a ring of blue LEDs under the appropriate shelf to signify which console has been selected. There’s even an array of solid state relays that will eventually control the mains power going to each system, though [odelot] hasn’t fully implemented it yet. Currently the electronics for this project live on a fairly packed breadboard, but it looks like he’s in the early stages of designing a proper PCB to clean it all up.

Not content to simply control a commercial A/V switch? In the past we’ve seen truly dedicated console collectors design their own custom switches from the ground up, complete with a display to show the currently selected system’s logo.

Continue reading “Consoles, Consoles On The Wall, Can Alexa Help Me Play Them All?”

Alexa, Bring Me A Beer!

Voice controlled home assistants are the wonder of our age, once you’ve made peace with the privacy concerns of sharing the intimacies of your life with a data centre owned by a massive corporation, anyway. They provide a taste of how the future was supposed to be in those optimistic predictions of decades past: Alexa and Siri can crack jokes, control your lights, answer questions, tell you the news, and so much more.

But for all their electronic conversational perfection, your electronic pals can’t satisfy your most fundamental needs and bring you a beer. This is something [luisengineering] has fixed, an he’s provided the appropriate answer to the question “Alexa: bring mir ein bier!“. The video which we’ve also put below the break is in German with YouTube’s automatic closed captions if you want them, but we think you’ll be able to get the point of it if not all his jokes without needing to learn to speak a bit of Deutsch.

As he develops his beer-delivery system we begin to appreciate that what might seem to be a relatively straightforward task is anything but. He takes an off-the-shelf robot and gives it a beer-bottle grabber and ice hopper, but the path from fridge to sofa still needs a little work. The eventual solution involves a lot of trial and error, and a black line on the floor for the ‘bot to follow. Finally, his electronic friend can bring him a beer!

We like [Luis]’s entertaining presentational style, and the use of props as microphone stands. We’ll be keeping an eye out for what he does next, and you should too. Meanwhile it may not surprise you that this is not the first beer-delivery ‘bot we’ve brought you.

Continue reading “Alexa, Bring Me A Beer!”