Voice-controlled appliances are nothing new. What might be new, however, is [Moonshine AI] running it all locally on a Raspberry Pi Pico 2 W!
The voice interface is roughly divided into three parts: voice activity detection, SpellingCNN speech-to-text and a neural text to speech. The speech to text supports up to 50 tokens, and can be re-trained to support any specific words you want. It runs a simple loop: detect voice activity, listen for (command) tokens, process them in C++, use the TTS to reply, and repeat.
Now, to be fair, it is a bit of a squeeze: 3.6 MiB of the available 4 MiB FLASH and 468 KiB SRAM on a stock Pi Pico 2 board. It leaves you with just about enough space to write a small amount of extra software, but it’ll be a challenge to fit anything substantial. Still, fitting three different types of AI model needed to make this possible in such a space is quite impressive.
This would be really interesting to hook up to Home Assistant.
I really like the idea of having voice recognition completely local and self contained. Having one of those voice assistants constantly listening in and sending every detail off to Google and the like isn’t appealing in the slightest.
I guess there’s nothing stopping you having several of these, each recognizing a different set of words, perhaps just outputting an index number for a command and letting another board interpret the output.
I wonder about converting this to run on a pi2 or a pi3. Perhaps putting it all into ram at startup rather than burning through the sd card write cycles, no doubt the extra speed and memory would allow for a quick response and a few more phrases.