Artificial intelligence is taking the world by storm. Rather than a Terminator-style apocalypse, though, it seems to be more of a useful tool for getting computers to solve problems on their own. This isn’t just for supercomputers, either. You can load AI onto some of the smallest microcontrollers as well. Tensorflow Lite is a popular tool for this, but getting it to work on your particular microcontroller can be a pain, unless you’re using an Espruino.
This project adds support for Tensorflow to this class of microcontrollers without having to fuss around with obtuse build tools. Basically adding a single line of code creates an instance, all without having to compile anything or even reboot. Tensorflow is a powerful software tool for microcontrollers, and having it this accessible now is a great leap forward.
So, what can you do with this tool? The team behind this build is using Tensorflow on an open smart watch that can be used to detect hand gestures and many other things. They also opened up these tools for use in a browser, which allows use of the AI software and emulates an Espruino without needing a physical device. There’s a lot going on with this one, and it’s a bonus that it’s open source and ready to be turned into anything you might need, like turning yourself into a Street Fighter.
With the release of TensorFlow Lite at Google I/O 2019, the accessible machine learning library is no longer limited to applications with access to GPUs. You can now run machine learning algorithms on microcontrollers much more easily, improving on-board inference and computation.
[Brandon Satrom] published a demo on how to run TFLite on Particle devices (tested on Photon, Argon, Boron, and Xenon) making it possible to make predictions on live data with pre-trained models. While some of the easier computation that occurs on MCUs requires manipulating data with existing equations (mapping analog inputs to a percentage range, for instance), many applications require understanding large, complex sets of sensor data gathered in real time. It’s often more difficult to get accurate results from a simple equation.
The current method is to train ML models on specialty hardware, deploy the models on cloud infrastructure, and backhaul sensor data to the cloud for inference. By running the inference and decision-making on-board, MCUs can simply take action without backhauling any data.
He starts off by constructing a simple TGLite model for MCU execution, using mean squared error for loss and stochastic gradient descent for the optimization. After training the model on sample data, you can save the model and convert it to a C array for the MCU. On the MCU, you can load the model, TFLite libraries, and operations resolver, as well as instantiate an interpreter and tensors. From there you invoke the model on the MCU and see your results!
[Thanks dcschelt for the tip!]
Watching a sport can be a bit odd if you aren’t familiar with it. Most Americans, for example, would think a cricket match looked funny because they don’t know the rules. If you were not familiar with baseball, you might wonder why one of the coaches was waving his hands around, touching his nose, his ears, and his hat seemingly at random. Those in the know however understand that this is a secret signal to the player. The coach might be telling the player to steal a base or bunt. The other team tries to decode the signals, but if you don’t know the code that is notoriously difficult. Unless you have the machine learning phone app you can see in the video below.
If you are not a baseball fan, it works like this. The coach will do a number of things. Perhaps touch his cap, then his nose, brush his left forearm, and touch his lips. However, the code is often as simple as knowing one attention signal and one action signal. For example, the coach might tell you that if they touch their nose and then their lips, you should steal. Touching their nose and then their ear is a bunt. Touching their nose and then the bill of their cap is something else. Anything they do that doesn’t start with touching their nose means nothing at all. If the signal is this easy, you really don’t even need machine learning to decode it. But if it were more complicated — say, the gesture that occurs third after they touch their nose unless they also kick dirt at which point it means nothing — it would be much harder for a human to figure out.
Continue reading “AI Phone App Learns Baseball Signals”
A question: if you’re controlling the classic video game Street Fighter with gestures, aren’t you just, you know, street fighting?
That’s a question [Charlie Gerard] is going to have to tackle should her AI gesture-recognition controller experiments take off. [Charlie] put together the game controller to learn more about the dark arts of machine learning in a fun and engaging way.
The controller consists of a battery-powered Arduino MKR1000 with WiFi and an MPU6050 accelerometer. Held in the hand, the controller streams accelerometer data to an external PC, capturing the characteristics of the motion. [Charlie] trained three different moves – a punch, an uppercut, and the dreaded Hadouken – and captured hundreds of examples of each. The raw data was massaged, converted to Tensors, and used to train a model for the three moves. Initial tests seem to work well. [Charlie] also made an online version that captures motion from your smartphone. The demo is explained in the video below; sadly, we couldn’t get more than three Hadoukens in before crashing it.
With most machine learning project seeming to concentrate on telling cats from dogs, this is a refreshing change. We’re seeing lots of offbeat machine learning projects these days, from cryptocurrency wallet attacks to a semi-creepy workout-monitoring gym camera.
Continue reading “Arduino, Accelerometer, And TensorFlow Make You A Real-World Street Fighter”
What’s in your crypto wallet? The simple answer should be fat stacks of Bitcoin or Ethereum and little more. But if you use a hardware cryptocurrency wallet, you may be carrying around a bit fat vulnerability, too.
At the 35C3 conference last year, [Thomas Roth], [Josh Datko], and [Dmitry Nedospasov] presented a side-channel attack on a hardware crypto wallet. The wallet in question is a Ledger Blue, a smartphone-sized device which seems to be discontinued by the manufacturer but is still available in the secondary market. The wallet sports a touch-screen interface for managing your crypto empire, and therein lies the weakness that these researchers exploited.
By using a HackRF SDR and a simple whip antenna, they found that the wallet radiated a distinctive and relatively strong signal at 169 MHz every time a virtual key was pressed to enter a PIN. Each burst started with a distinctive 11-bit data pattern; with the help of a logic analyzer, they determined that each packet contained the location of the key icon on the screen.
Next step: put together a training set. They rigged up a simple automatic button-masher using a servo and some 3D-printed parts, and captured signals from the SDR for 100 presses of each key. The raw data was massaged a bit to prepare it for TensorFlow, and the trained network proved accurate enough to give any hardware wallet user pause – especially since they captured the data from two meters away with relatively simple and concealable gear.
Every lock contains the information needed to defeat it, requiring only a motivated attacker with the right tools and knowledge. We’ve covered other side-channel attacks before; sadly, they’ll probably only get easier as technologies like SDR and machine learning rapidly advance.
Sorting trash into the right categories is pretty much a daily bother. Who hasn’t stood there in front of the two, three, five or more bins (depending on your area and country), pondering which bin it should go into? [Alvaro Ferrán Cifuentes]’s SeparAItor project is a proof of concept robot that uses a robotic sorting tray and a camera setup that aims to identify and sort trash that is put into the sorting tray.
The hardware consists of a sorting tray mounted to the top of a Bluetooth-connected pan and tilt platform. The platform communicates with the rest of the system, which uses a camera and OpenCV to obtain the image data, and a Keras-based back-end which implements a deep learning neural network in Python.
Training of the system was performed by using self-made photos of the items that would need to be sorted as these would most closely match real-life conditions. After getting good enough recognition results, the system was put together, with a motion detection feature added to respond when a new item was tossed into the tray. The system will then attempt to identify the item, categorize it, and instruct the platform to rotate to the correct orientation before tilting and dropping it into the appropriate bin. See the embedded video after the break for the system in action.
Believe it or not, this isn’t the first trash-sorting robot to grace the pages of Hackaday. Potentially concepts like these, that rely on automation and machine vision, could one day be deployed on a large scale to help reduce how much recyclable material end up in landfills. Continue reading “Automate Sorting Your Trash With Some Healthy Machine Learning”
Who wouldn’t want a robot that can fetch them a glass of water? [Saral Tayal] didn’t just think that, he jumped right in and built his own personal assistant robot. This isn’t just some remote-controlled rover though. The robot actually listens to his voice and recognizes his face.
The body of the robot is the common “Rover 5” platform, to which [Saral] added a number of 3D printed parts. A forklift like sled gives the robot the ability to pick things up. Some of the parts are more about form than function – [Saral] loves NASA’s Spirit and Opportunity Mars rovers, so he added some simulated solar cells and other greebles.
The Logitech webcam up front is very functional — images are fed to machine learning models, while audio is processed to listen for commands. This robot can find and pick up 90 unique objects.
The robot’s brains are a Raspberry Pi. It uses TensorFlow for object recognition. Some of the models [Saral] is using are pretty large – so big that the Pi could only manage a couple of frames per second at 100% CPU utilization. A Google Coral coprocessor sped things up quite a bit, while only using about 30% of the Pi’s processor.
It takes several motors to control to robot’s tracks and sled. This is handled by two Roboclaw motor controllers which themselves are commanded by the Pi.
We’ve seen quite a few mobile robot rovers over the years, but [Saral’s] ‘bot is one of the most functional designs out there. Even better is the fact that it is completely open source. You can find the code and 3D models on his GitHub repo.
Check out a video of the personal assistant rover in action after the break.
Continue reading “DIY Personal Assistant Robot Hears And Sees All”