Twitter: It’s Not The Algorithm’s Fault. It’s Much Worse.

Maybe you heard about the anger surrounding Twitter’s automatic cropping of images. When users submit pictures that are too tall or too wide for the layout, Twitter automatically crops them to roughly a square. Instead of just picking, say, the largest square that’s closest to the center of the image, they use some “algorithm”, likely a neural network, trained to find people’s faces and make sure they’re cropped in.

The problem is that when a too-tall or too-wide image includes two or more people, and they’ve got different colored skin, the crop picks the lighter face. That’s really offensive, and something’s clearly wrong, but what?

A neural network is really just a mathematical equation, with the input variables being in these cases convolutions over the pixels in the image, and training them essentially consists in picking the values for all the coefficients. You do this by applying inputs, seeing how wrong the outputs are, and updating the coefficients to make the answer a little more right. Do this a bazillion times, with a big enough model and dataset, and you can make a machine recognize different breeds of cat.

What went wrong at Twitter? Right now it’s speculation, but my money says it lies with either the training dataset or the coefficient-update step. The problem of including people of all races in the training dataset is so blatantly obvious that we hope that’s not the problem; although getting a representative dataset is hard, it’s known to be hard, and they should be on top of that.

Which means that the issue might be coefficient fitting, and this is where math and culture collide. Imagine that your algorithm just misclassified a cat as an “airplane” or as a “lion”. You need to modify the coefficients so that they move the answer away from this result a bit, and more toward “cat”. Do you move them equally from “airplane” and “lion” or is “airplane” somehow more wrong? To capture this notion of different wrongnesses, you use a loss function that can numerically encapsulate just exactly what it is you want the network to learn, and then you take bigger or smaller steps in the right direction depending on how bad the result was.

Let that sink in for a second. You need a mathematical equation that summarizes what you want the network to learn. (But not how you want it to learn it. That’s the revolutionary quality of applied neural networks.)

Now imagine, as happened to Google, your algorithm fits “gorilla” to the image of a black person. That’s wrong, but it’s categorically differently wrong from simply fitting “airplane” to the same person. How do you write the loss function that incorporates some penalty for racially offensive results? Ideally, you would want them to never happen, so you could imagine trying to identify all possible insults and assigning those outcomes an infinitely large loss. Which is essentially what Google did — their “workaround” was to stop classifying “gorilla” entirely because the loss incurred by misclassifying a person as a gorilla was so large.

This is a fundamental problem with neural networks — they’re only as good as the data and the loss function. These days, the data has become less of a problem, but getting the loss right is a multi-level game, as these neural network trainwrecks demonstrate. And it’s not as easy as writing an equation that isn’t “racist”, whatever that would mean. The loss function is being asked to encapsulate human sensitivities, navigate around them and quantify them, and eventually weigh the slight risk of making a particularly offensive misclassification against not recognizing certain animals at all.

I’m not sure this problem is solvable, even with tremendously large datasets. (There are mathematical proofs that with infinitely large datasets the model will classify everything correctly, so you needn’t worry. But how close are we to infinity? Are asymptotic proofs relevant?)

Anyway, this problem is bigger than algorithms, or even their writers, being “racist”. It may be a fundamental problem of machine learning, and we’re definitely going to see further permutations of the Twitter fiasco in the future as machine classification is being increasingly asked to respect human dignity.

Boost Your Animation To 60 FPS Using AI

The uses of artificial intelligence and machine learning continue to expand, with one of the more recent implementations being video processing. A new method can “fill in” frames to smooth out the appearance of the video, which [LegoEddy] was able to use this in one of his animated LEGO movies with some astonishing results.

His original animation of LEGO figures and sets was created at 15 frames per second. As an animator, he notes that it’s orders of magnitude more difficult to get more frames than this with traditional methods, at least in his studio. This is where the artificial intelligence comes in. The program is able to interpolate between frames and create more frames to fill the spaces between the original. This allowed [LegoEddy] to increase his frame rate from 15 fps to 60 fps without having to actually create the additional frames.

While we’ve seen AI create art before, the improvement on traditionally produced video is a dramatic advancement. Especially since the AI is aware of depth and preserves information about the distance of objects from the camera. The software is also free, runs on any computer with an appropriate graphics card, and is available on GitHub.

Continue reading “Boost Your Animation To 60 FPS Using AI”

Let This Crying Detecting Classifier Offer Some Much Needed Reprieve

Baby monitors are cool, but [Ish Ot Jr.] wanted his to only transmit sounds that required immediate attention and filter any non-emergency background noise. Posed with this problem, he made a baby monitor that would only send alerts when his baby was crying.

For his project, [Ish] used an Arduino Nano 33 BLE Sense due to its built-in microphone, sizeable RAM for storing large chunks of data, and it’s BLE capabilities for later connecting with an app. He began his project by collecting background noise using Edge Impulse Studio’s data acquisition functionality. [Ish] really emphasized that Edge Impulse was really doing all the work for him. He really just needed to collect some test data and that was mostly it on his part. The work needed to run and test the Neural Network was taken care of by Edge Impulse. Sounds handy, if you don’t mind offloading your data to the cloud.

[Ish] ended up with an 86.3% accurate classifier which he thought was good enough for a first pass at things. To make his prototype a bit more “finished”, he added some status LEDs, providing some immediate visual feedback of his classifier and to notify the caregiver. Eventually, he wants to add some BLE support and push notifications, alerting him whenever his baby needs attention.

We’ve seen a couple of baby monitor projects on Hackaday over the years. [Ish’s] project will most certainly be a nice addition to the list.

Facial Detection With Pi + MATLAB

[Monica] wanted to try a bit of facial detection with her Raspberry Pi and she found some pretty handy packages in MATLAB to help her do just that. The packages are based on the Viola-Jones algorithm which was the first real-time object detection framework for facial detection.

She had to download MATLAB’s Raspbian image to allow the Pi to interpret MATLAB commands over a custom server. That setup is mostly pretty easy and she does a good job walking you through the setup on her project page.

With that, now she can control the Pi in MATLAB: configure the camera, toggle GPIO, etc. The real fun comes with the facial detection program. In addition to opening up a live video feed of the Pi camera, the program outputs pixel data. [Monica] was mostly just testing the stock capabilities, but wants to try detecting other objects next. We’ll see what cool modifications she’s able to come up with.

If MATLAB doesn’t quite fit your taste, we have a slew of facial detection projects on Hackaday.

Art Generated From The Dubious Comments Section

[8BitsAndAByte] are back, and this time they’re taking on the comments section with art. They wondered whether or not they can take something as dubious as the comments section and redeem it into something more appealing like art.

They started by using remo.tv, a tool they’ve used in other projects, to read comments from their video live feeds and extract random phrases. The phrases are then analyzed by text to speech, and a publicly available artificial intelligence algorithm that generates an image from a text description. They can then specify art styles like modern, abstract, cubism, etc to give their image a unique appeal. They then send the image back to the original commenter, crediting them for their comment, ensuring some level of transparency.

We were a bit surprised that the phrase dog with a funny hat generated an image of a cat, so I think it’s fair to say that their AI engine could use a bit of work. But really, we could probably say that about AI as a whole.

Continue reading “Art Generated From The Dubious Comments Section”

Engineers Develop A Brain On A Chip

Our abilities to multitask, to quickly learn complex maneuvers, and to instantly recognize objects even as infants are just some of the ways that human brains make use of our billions of synapses. Biologically, our brain requires fluid-filled cavities, nerve fibers, and numerous other cells and connections in order to function. This isn’t the case with a new kind of brain recently announced by a team of MIT engineers in Nature Nanotechnology. Compared to the size of a typical human brain, this new “brain-on-a-chip” is able to fit on a piece of confetti.

When you take a look at the chip, it is more similar to tiny metal carving than to any neurological organ. The technology used to design the chip is based on memristors – silicon-based components that mimic the transmissions of synapses. A concatenation of “memory” and “resistor”, they exist as passive circuit elements that retain a relationship between the time integrals of current and voltage across an element. As resistance varies, tiny read charges are able to access a history of applied voltage. This can be accomplished by hysteresis and other non-linear properties of passive circuitry.

These properties can be best observed at nanoscale levels, where they aren’t dwarfed by other electronic and field effects. A tiny positive and negative electrode are separated by a “switching medium”, or space between the two electrodes. Voltage applied to one end causes ions to flow through the medium, forming a conduction channel to the other end. These ions make up the electrical signal transmitted through the circuit.

In order to fabricate these memristors, the researchers used alloys of silver for the positive electrode, and copper alongside silicon for the negative electrode. They sandwiched the two electrodes along an amorphous medium and patterned this on a silicon chip tens of thousands of times to create an array of memristors. To train the memristors, they ran the chips through visual tasks to store images and reproduce them until cleaner versions were produced. These new devices join a new category of research into neuromorphic computing – electronics that function similar to the way the brain’s neural architecture operates.

The opportunity for electronics that are capable of making instantaneous decisions without consulting other devices or the Internet spell the possibility of portable artificial intelligence systems. Though we already have software systems capable of simulating synaptic behavior, developing neuromorphic computing devices could vastly increase the capability of devices to do tasks once thought to belong solely to the human brain.

Train All The Things Contest Update

Back in January when we announced the Train All the Things contest, we weren’t sure what kind of entries we’d see. Machine learning is a huge and rapidly evolving field, after all, and the traditional barriers that computationally intensive processes face have been falling just as rapidly. Constraints are fading away, and we want you to explore this wild new world and show us what you come up with.

Where Do You Run Your Algorithms?

To give your effort a little structure, we’ve come up with four broad categories:

  • Machine Learning on the Edge
    • Edge computing, where systems reach out to cloud resources but run locally, is all the rage. It allows you to leverage the power of other people’s computers the cloud for training a model, which is then executed locally. Edge computing is a great way to keep your data local.
  • Machine Learning on the Gateway
    • Pi’s, old routers, what-have-yous – we’ve all got a bunch of devices laying around that bridge space between your local world and the cloud. What can you come up with that takes advantage of this unique computing environment?
  • Machine Learning in the Cloud
    • Forget about subtle — this category unleashes the power of the cloud for your application. Whether it’s Google, Azure, or AWS, show us what you can do with all that raw horsepower at your disposal.
  • Artificial Intelligence Blinky
    • Everyone’s “hardware ‘Hello, world'” is blinking an LED, and this is the machine learning version of that. We want you to use a simple microprocessor to run a machine learning algorithm. Amaze us with what you can make an Arduino do.

These Hackers Trained Their Projects, You Should Too!

We’re a little more than a month into the contest. We’ve seen some interesting entries bit of course we’re hungry for more! Here are a few that have caught our eye so far:

  • Intelligent Bat Detector – [Tegwyn☠Twmffat] has bats in his… backyard, so he built this Jetson Nano-powered device to capture their calls and classify them by species. It’s a fascinating adventure at the intersection of biology and machine learning.
  • Blackjack Robot – RAIN MAN 2.0 is [Evan Juras]’ cure for the casino adage of “The house always wins.” We wouldn’t try taking the Raspberry Pi card counter to Vegas, but it’s a great example of what YOLO can do.
  • AI-enabled Glasses – AI meets AR in ShAIdes, [Nick Bild]’s sunglasses equipped with a camera and Nano to provide a user interface to the world. Wave your hand over a lamp and it turns off. Brilliant!

You’ve got till noon Pacific time on April 7, 2020 to get your entry in, and four winners from each of the four categories will be awarded a $100 Tindie gift card, courtesy of our sponsor Digi-Key. It’s time to ramp up your machine learning efforts and get a project entered! We’d love to see more examples of straight cloud AI applications, and the AI blinky category remains wide open at this point. Get in there and give machine learning a try!