Twitter: It’s Not The Algorithm’s Fault. It’s Much Worse.

September 26, 2020 by Elliot Williams 160 Comments

Maybe you heard about the anger surrounding Twitter’s automatic cropping of images. When users submit pictures that are too tall or too wide for the layout, Twitter automatically crops them to roughly a square. Instead of just picking, say, the largest square that’s closest to the center of the image, they use some “algorithm”, likely a neural network, trained to find people’s faces and make sure they’re cropped in.

The problem is that when a too-tall or too-wide image includes two or more people, and they’ve got different colored skin, the crop picks the lighter face. That’s really offensive, and something’s clearly wrong, but what?

A neural network is really just a mathematical equation, with the input variables being in these cases convolutions over the pixels in the image, and training them essentially consists in picking the values for all the coefficients. You do this by applying inputs, seeing how wrong the outputs are, and updating the coefficients to make the answer a little more right. Do this a bazillion times, with a big enough model and dataset, and you can make a machine recognize different breeds of cat.

What went wrong at Twitter? Right now it’s speculation, but my money says it lies with either the training dataset or the coefficient-update step. The problem of including people of all races in the training dataset is so blatantly obvious that we hope that’s not the problem; although getting a representative dataset is hard, it’s known to be hard, and they should be on top of that.

Which means that the issue might be coefficient fitting, and this is where math and culture collide. Imagine that your algorithm just misclassified a cat as an “airplane” or as a “lion”. You need to modify the coefficients so that they move the answer away from this result a bit, and more toward “cat”. Do you move them equally from “airplane” and “lion” or is “airplane” somehow more wrong? To capture this notion of different wrongnesses, you use a loss function that can numerically encapsulate just exactly what it is you want the network to learn, and then you take bigger or smaller steps in the right direction depending on how bad the result was.

Let that sink in for a second. You need a mathematical equation that summarizes what you want the network to learn. (But not how you want it to learn it. That’s the revolutionary quality of applied neural networks.)

Now imagine, as happened to Google, your algorithm fits “gorilla” to the image of a black person. That’s wrong, but it’s categorically differently wrong from simply fitting “airplane” to the same person. How do you write the loss function that incorporates some penalty for racially offensive results? Ideally, you would want them to never happen, so you could imagine trying to identify all possible insults and assigning those outcomes an infinitely large loss. Which is essentially what Google did — their “workaround” was to stop classifying “gorilla” entirely because the loss incurred by misclassifying a person as a gorilla was so large.

This is a fundamental problem with neural networks — they’re only as good as the data and the loss function. These days, the data has become less of a problem, but getting the loss right is a multi-level game, as these neural network trainwrecks demonstrate. And it’s not as easy as writing an equation that isn’t “racist”, whatever that would mean. The loss function is being asked to encapsulate human sensitivities, navigate around them and quantify them, and eventually weigh the slight risk of making a particularly offensive misclassification against not recognizing certain animals at all.

I’m not sure this problem is solvable, even with tremendously large datasets. (There are mathematical proofs that with infinitely large datasets the model will classify everything correctly, so you needn’t worry. But how close are we to infinity? Are asymptotic proofs relevant?)

Anyway, this problem is bigger than algorithms, or even their writers, being “racist”. It may be a fundamental problem of machine learning, and we’re definitely going to see further permutations of the Twitter fiasco in the future as machine classification is being increasingly asked to respect human dignity.

Dynamic Soaring: 545 MPH RC Planes Have No Motor

September 24, 2020 by Elliot Williams 59 Comments

The fastest remote-controlled airplane flight ever recorded took place in 2018, with a top speed of 545 miles/hour. That’s 877 km/h, or Mach 0.77!

What was the limiting factor, preventing the pilot-and-designer Spencer Lisenby’s plane from going any faster? The airstream over parts of the wing hitting the sound barrier, and the resulting mini sonic booms wreaking havoc on the aerodynamics. What kind of supercharged jet motor can propel a model plane faster than its wings can carry it? Absolutely none; the fastest RC planes are, surprisingly, gliders.

Dynamic soaring (DS) was first harnessed to propel model planes sometime in the mid 1990s. Since then, an informal international competition among pilots has pushed the state of the art further and further, and in just 20 years the top measured speed has more than tripled. But dynamic soaring is anything but new. Indeed, it’s been possible ever since there has been wind and slopes on the earth. Albatrosses, the long-distance champs of the animal kingdom, have been “DSing” forever, and we’ve known about it for a century.

DS is the highest-tech frontier in model flight, and is full of interesting physical phenomena and engineering challenges. Until now, the planes have all been piloted remotely by people, but reaching new high speeds might require the fast reaction times of onboard silicon, in addition to a new generation of aircraft designs. The “free” speed boost that gliders can get from dynamic soaring could extend the range of unmanned aerial vehicles, when the conditions are right. In short, DS is at a turning point, and things are just about to get very interesting. It’s time you got to know dynamic soaring.

Continue reading “Dynamic Soaring: 545 MPH RC Planes Have No Motor” →

Code For Hackers

September 19, 2020 by Elliot Williams 57 Comments

Mike and I were talking about two very similar clock projects we’d both built recently: they both use ESP8266 modules to get the time over WiFi and NTP, and they both failed. Mike’s failed because he was visiting relatives in a different timezone with different WiFi credentials, and mine failed because daylight savings time caught me off-guard. In both cases, we hard-coded stuff that could obviously change, but we drew vastly different conclusions.

Mike thought he’d solve his WiFi problem with a fallback to a captive portal, and maybe would have to figure out some web interface for configuring the timezone. A very clean, professional solution. Me? I’ve got good comments in the code, can find the UTC offset (or the WiFi creds) in a few minutes, and flash the new version up simply by fetching a USB cable, for something that happens twice a year. It’s hardly worth the trouble to cobble together a web interface.

We’ve accidentally embodied a quandary that spans both the hardware and software worlds: should flexibility be exposed to the end-user or to the hacker who can peer under the hood or open up the source code? (And what if the end-user is the hacker?) What are the tradeoffs, in project complexity and in ease of use?

And in this, Mike is on the side of right and good, and I’m the heretic. I don’t always write my code to be extensible or re-usable. I sometimes write it to be quickly re-edited and patched whenever I need to. Is it full of magic numbers? Sure! But I know just where they are and how to change them. Heck, most are even well documented in their own header file. You could probably figure it out just about as fast. Would my father-in-law be able to tweak the timezone? Nope! But this ain’t his project anyway.

Dare to code for hackers! Don’t over-generalize or over-abstract. Less is more. Don’t be afraid to edit code. Tweak, compile, and re-flash when the situation changes. After all, that’s how you got the code there in the first place.

And although I’m on the wrong end of history, in this case I was right. You see, before daylight savings time could come around again, and I could have made use of that captive portal that I didn’t bother coding up anyway, my son entered first grade. Everything needs to be changed, from the hardware to the software. Will I code up the next version with flexible time regimes? As flexible as I need it to be, but not more.

Plastic Prosthetics For Rubber Duckies

September 19, 2020 by Elliot Williams 14 Comments

Will someone please think of the rubber duckies?!

For decades they’ve been reduced to a laughing stock: a caricature of waterfowl. Left without a leg to stand on, their only option is to float around in the tub. And they don’t even do that well, lacking the feet that Mother Nature gave them, they capsize when confronted with the slightest ripple. But no more!

Due to the wonders of 3D printing, and painstaking design work by [Jan] from the Rubber Ducky Research Center, now you can print your own rubber ducky feet. We have the technology! Your ducks are no longer constrained to a life in the tub, but can roam free as nature intended. The video (embedded below) will certainly tug at your heartstrings.

OK, it’s a quick print and it made my son laugh.

The base and legs probably don’t fit your duck as-is, but it’s a simple matter to scale them up or down while slicing. (Picture me with calipers on the underside of a rubber ducky.) The legs were a tight press-fit into the body, so you might consider slimming them down a tiny bit when doing the scaling, but this probably depends on your printer tolerances.

It looks snazzy in gold-fleck PETG, and would probably work equally well for some more elaborate rubber duckies as well.

Continue reading “Plastic Prosthetics For Rubber Duckies” →

Mirror Turns Webcam Into Document Camera

September 17, 2020 by Elliot Williams 12 Comments

This is one of those so-simple-I-wish-I-invented-it hacks. Professor [Michael Peshkin] is teaching his engineering students remotely. While he has a nice second camera that he can use to transmit whatever he doodles on paper, most of his students just have the single webcam built into their laptops.

The solution is to put a mirror in front of the laptop cam, and flip the image left-to-right in software. They use Zoom, which has a mirror mode. Done.

The trick is making a nice frame. [Michael] has bent one out of wire, but suggests that a mirror compact works about as well in a pinch. It’s super important that his students can ask him questions backed up by drawings, and this reduces the startup cost to nearly nothing, making it universally useful.

[Prof. Peshkin] is not a stranger to mirror-based pedagogical hacks. Seven years ago, he showed us how to make a transparent whiteboard for video lectures, and it blew up on Hackaday. Since then, there are hundreds or thousands of Lightboards in the wild. We hope this idea catches on as well!

The Egg-laying Wool-Milk Pig

September 12, 2020 by Elliot Williams 32 Comments

Last week, I wrote about two recent projects of mine that serve as cautionary tales in keeping projects simple — you probably can’t simplify everything, so it’s worth the time to find out which simplifications have the most bang for the buck. This week, I’d like to share a tale of lack of design focus.

German has the eierlegende Wollmilchsau: a mystical animal that lays eggs, while producing wool, milk, and meat to boot. It’s a little bit like the English “jack of all trades, master of none” except that the eierlegende Wollmilchsau doesn’t do each job badly, it plainly can’t exist. This is obviously a bad way to start a design.

The first surfboard that I made by myself was supposed to be an eierlegende Wollmilchsau. It was going to be a longboard, because we had months with smaller waves that just weren’t all that suitable for shortboarding, but it was also going to turn sharply off the rails like a shortboard. To help it turn, it was going to have tons of camber (bend like a banana), and small fins. And along the way, I thought I’d make it thin to cut through the water.

Of course what I ended up with, not helped by my heavy fiberglassing hand, was a plow that dug into the water, would turn unexpectedly when you managed to get it onto the rails, and couldn’t pick up a small wave to save its life due to the camber and aforementioned plowing. I surfed it anyway, as a matter of pride, but I had no illusions of it being anything but the the worst board I owned. And that’s comparing it to the $30 used rasta-graphic plank that had been taking on water for at least five years, unrepaired, and was rotting out from the inside. At least it had design focus.

My surfboard didn’t suffer from feature creep, where you start piling on features until the project crumbles from overload, but rather from wanting to have my cake and eat it too. Or from failing to realize that certain design goals were necessarily tradeoffs. The “raily” behavior that I wanted when it was in bigger waves was necessarily “diggy” in small waves. Good boards trade off these features, and getting the balance between them is the art of shaping a board.

So when you start up a new project, think about which facets of your design are jointly achievable, and which are necessarily tradeoffs. Ignoring tradeoffs is a recipe for disaster, designing an eierlegende Wollmilchsau. But viewed constructively, it’s exactly these nuanced decisions that separates the simply possible from the truly marvelous. May you identify your trades, and make them well!

Keep It Simple, Smartly

September 5, 2020 by Elliot Williams 13 Comments

“Keep it simple” sounds like such good advice, but what exactly is the “it”; what parts of a project should you try to keep simple? You can’t always make everything simple, can you? Are all kinds of “simplicity” equally valuable, or are there aspects of a design where simplicity has multiplier effects on the rest of the project?

I ran into two seemingly different, but surprisingly similar, design problems in the last couple weeks, and I realized that focusing on keeping one aspect of the project simple had a multiplier effect on the rest — simplifying the right part of the problem made everything drastically easier.

The first example was a scratch-built airplane design. I’d made a few planes over the summer, focusing on plans on the Interwebs that emphasize simplicity of the actual build. Consequently, the planes were a bit heavy, maybe not entirely aerodynamic, and probably underpowered. And this is because the effort you expend building the plane doesn’t fundamentally have anything to do with flight. Keeping the build simple doesn’t necessarily get you a good plane.

Weight, on the other hand, is central. Wings produce lift, whether measured in grams or ounces, and anything heavier just isn’t gonna fly. But reducing weight has a multiplier effect. Less weight means smaller and lighter motors and batteries. Structures don’t need to be as stiff if they’re not subject to heavier bending forces. And, important to the noob pilot, planes with less weight per wing area fly slower, giving me (ahem, the noob pilot) more reaction time when something goes sideways. Trying to simplify the design by trimming weight has knock-on effects all around.

My latest fully-DIY design threw out anything that brought weight along with it, including some parts I thought were necessary for stiffness or crash resistance. But with the significantly lowered weight, these problems evaporated without needing me to solve them — in a way, the complexity of design was creating the problems that the complexity of design was supposed to solve. Ditching it meant that I had a slow plane, with simple-to-build wings, that’s capable of carrying a lightweight FPV camera. Done and done! Simply.

At the same time, I’m building a four-axis CNC foam cutter. I’ve built many 3D printers, and played around with other folks’ DIY CNC machines, so I had a few design ideas in my head starting out. My first iteration of an XY axis for the machine runs on metal angle stock with a whopping eight skate bearings per axis. It’s strong and rigid, and clumsy and overkill, in a bad way for this machine.

3D printers want to move a relatively light tool head around a small volume, but relatively quickly. CNC mills need to be extremely rigid and shoulder heavy side loads, subject to some speed constraints. A foam cutter has none of these needs. The hot wire melts the foam by radiation, so there are no loads on the machine because it doesn’t even contact the workpiece. And because it cuts by melting, it has to go slow. These are the places in the design where simplification will bear the most fruit.

I write this in retrospect, or at least from the perspective of a second prototype. I wanted the first design to hold the cutting filament taut, hence the rigid frame. But separating the tension from the motion, by using a lightweight external bow to keep the filament tight, meant that the machine could be dead simple. I could use smaller plastic sliders instead of complex bearings, on thin rods instead of bulky rails. In a day after having this realization, I got twice as far as I had on the previous machine design in a week, and it takes up a lot less space in my basement.

So take your KISS to the next level. Brainstorm a while about the binding constraints on your design, and what relaxing any of them can do. Do any particular simplifications enable further simplifications? Those are the ones that you want to start with. Keep it simple, smartly. And because it’s not always easy to find these multiplier effects, tell your friends!