All of the machine language stuff coming out lately doesn’t affect you if you are developing with embedded microcontrollers, right? Perhaps not. Microsoft Research India wants you to use their EdgeML tool to do machine learning tasks such as gesture recognition in tiny devices like an Arduino Uno. According to the developers, you might need as little as 2 KB of RAM. There’s no network connection required and the work is using Tensorflow underneath, so it is compatible with much of what you’ll find for bigger computers.
If you add processing power, you can get more capability. For example, one of the demonstrations is a wake-word recognizer on a Raspberry Pi Zero (although the page for that demo seems to be missing at the moment; try the GesturePod, instead).
The system generally uses Python, but there are efficient C++ implementations for selected algorithms. The code lives on GitHub. There are also a number of research papers about each tool that you can find on the GitHub page. There’s also a recent paper on MinUn, an attempt to make things even more efficient for ARM microcontrollers. In particular, MinUn can store approximate numbers to save space, allows for variable precision of tensors, and tries to reduce memory fragmentation, an important feature for CPUs that don’t have memory management units.
If you haven’t studied TensorFlow yet, start here. Why use something like this with a microcontroller? How about smarter robots?
when it will run on a NE556 (dual-core NE555) I will have a look.
I did a MNIST character recognition with pytorch running on an Arduino Uno. Works kind of nice,
but the 2kB fills up fast. E.g. the MNIST dataset images are 28x28x8bit, so this means a third of the RAM just to buffer the input.
With quantisation to 8bit weights and down sampling the input data to 14×14, I could squeeze in two Conv2d and one fully connected layer. Just enough to make it work, sort of. (source available at https://github.com/kraiskil/onnx2c/tree/master/examples/atmega_mnist )
Lesson learned: don’t do image based AI on Arduino Unos :)
You can try a recurrent network that has a smaller input layer to process one row at a time instead. It won’t break records but at least 85% accuracy is possible with that approach.
“Microsoft Research India wants you”
No Sympathy for Redmond.
Would there be some way to compress something (such as datasets) to make the limited arduino uno storage & memory go further?
TinyML or EdgeML are pretty cool but you can’t run many inferences on the hardware and the accuracy is quite limited. There is a good certificate class on Edx by HarvardX on TinyML.