Much of the expense of developing AI models, and much of the recent backlash to said models, stems from the massive amount of power they tend to consume. If you’re willing to sacrifice some ability and accuracy, however, you can get ever-more-decent results from minimal hardware – a tradeoff taken by the Grove Vision AI board, which runs image recognition in near-real time on only 0.35 Watts.
The heart of the board is a WiseEye processor, which combines two ARM Cortex M55 CPUs and an Ethos U55 NPU, which handles AI acceleration. The board connects to a camera module and a host device, such as another microcontroller or a more powerful computer. When the host device sends the signal, the Grove board takes a picture, runs image recognition on it, and sends the results back to the host computer. A library makes signaling over I2C convenient, but in this example [Jaryd] used a UART.
To let it run on such low-power hardware, the image recognition model needs some limits; it can run YOLO8, but it can only recognize one object, runs at a reduced resolution of 192×192, and has to be quantized down to INT8. Within those limits, though, the performance is impressive: 20-30 fps, good accuracy, and as [Jaryd] points out, less power consumption than a single key on a typical RGB-backlit keyboard. If you want another model, there are quite a few available, though apparently of varying quality. If all else fails, you can always train your own.
Such edge AI projects as these are all about achieving better performance with limited resources; if your requirements aren’t too demanding, you can run speech recognition on much more limited devices. Of course, there are also some people who try to make image recognition less effective.
We’ve been doing stuff like that with wavelets since at least early 1990s.
Considering this is interesting stuff, please tell us more. Please do not respond with “you can google it” as that never is a helpful kind of response. Since you already seem to know the answer please enlighten us with a link of some kind as it would be fun if we could all learn from this. Assuming that you are actually willing or able to share this kind of knowledge.
Yes! I’d be interested in hearing the “wavelet” remark fleshed out, too.
This is is impressive, but a single LED on your keyboard does not pull 0.35W. That would imply that and RGB-lit 104-key keyboard pulls over 35 watts… nope. Cool project, but I hate it when people strain their credibility making idiot statements like that.
It stretches the statement a bit, but if we’re running an RGB LED at 100% duty cycle at 5V with 20mA per LED then that comes out to 0.3W (W = 5V * 0.020A * 3).
Keyboards get away with it by A: not running the LEDs at 100% duty cycle 20mA and 2: scanning through a matrix of LEDs so they’re not all on at once.
So not a downright “idiot statement” like you claim. But not entirely accurate either.