Artificial intelligence is taking the world by storm. Rather than a Terminator-style apocalypse, though, it seems to be more of a useful tool for getting computers to solve problems on their own. This isn’t just for supercomputers, either. You can load AI onto some of the smallest microcontrollers as well. Tensorflow Lite is a popular tool for this, but getting it to work on your particular microcontroller can be a pain, unless you’re using an Espruino.
This project adds support for Tensorflow to this class of microcontrollers without having to fuss around with obtuse build tools. Basically adding a single line of code creates an instance, all without having to compile anything or even reboot. Tensorflow is a powerful software tool for microcontrollers, and having it this accessible now is a great leap forward.
So, what can you do with this tool? The team behind this build is using Tensorflow on an open smart watch that can be used to detect hand gestures and many other things. They also opened up these tools for use in a browser, which allows use of the AI software and emulates an Espruino without needing a physical device. There’s a lot going on with this one, and it’s a bonus that it’s open source and ready to be turned into anything you might need, like turning yourself into a Street Fighter.
Interesting, I’ve been keen on making a gesture interface for home automation for a while now.
I’d probably back this Kickstarter if the case and strap weren’t so ugly.
This AI code + espruino is open source and can run in any nrf52 device almost as is – including couple of cheap hackable fitness trackers from ebay/aliexpress.
The $4.99 DS-D6 is gone but there are other nrf52 ones that start at ~ US$14. Lot of them can be updated with custom firmware and some of them even over bluetooth without taking apart. Personally I have espruino running inside DS-D6, DS-D9 (HW02), HX03W and F07, but the list is longer e.g. F10 and HX03F should work and there are more see e.g. here https://github.com/fanoush/ds-d6#other-nrf52832-devices
If the strap is your only concern, I designed a simple adapter to accept standard straps :-)
https://www.thingiverse.com/thing:4010651
The idea of ML on sub-$3 MCUs like the NRF52832 is revolutionary to me. I’m also running TF Lite Micro for Arduino on an ESP8266, so that’s <$1 in volume.
In the next 12-24 months this is going to be everywhere in cheap consumer electronics. Bets on Sonoff etc adding it to their lightbulbs?
And Bangle.js just makes ML far more approachable for average dabblers like me. There is not a cross-compiler in sight on my computer but I can re-train a TF gesture model and load it (or any compatible TF Lite model) straight on to the watch in a Chrome window.
(Disclaimer, we worked with Gordon on Bangle.js for our NodeConf EU conference a few weeks back).
This is amazing.
I always thought artificially intelligent machines would take decades to produce and would take a lot of computational horsepower.
Good news.
I think the bad news is that it still takes a relative lot of computational horsepower to do the really cool stuff. You’ll notice their pre taught models are benchmarked on flagship phones which are using all cores, about 80-100 times as fast as the base Espruino so your 22ms on those is more like 2 seconds. Which could be bearable depending on application.
A pre-AI Explosion use case for convolutional neural networks that I ran into was a company in Houston that was doing intelligent surveillance cameras. The cameras processing system could be trained to signal if anything “unusual” for the area popped up. The signal was meant to get a human to look over the feed, so false positives were considered acceptable. Probably still too intensive for your average arduino to perform, but software that allows a pre-trained variant of the algorithm to be kept locally can seriously expand the power of a system like this.
I _still_ can’t find the article I read on running TensorFlow networks on microcontrollers, but the use case covered in said article was using a CNN discriminator to save on bandwidth (and transmission power) in sensor networks by only “phoning home” when something outside of the expected input happens. The cameras above are assumedly going to be tied into a physical network, or wi-fi with maintained connections, but being able to have the camera decide that it needs to save a copy of whatever is happening locally if network connectivity dies, sounds like a good product feature!