Hackaday Links Column Banner

Hackaday Links: February 6, 2022

Last week, the news was filled with stories of Jack Sweeney and his Twitter-bot that tracks the comings and goings of various billionaires in their private jets. This caught the attention of the billionaire-iest of them all, one Elon Musk, who took exception to the 19-year-old’s feat of data integration, which draws from a number of public databases to infer the location of Elon’s plane. After Jack wisely laughed off Elon’s measly offer of $5,000 to take the bot down, Elon ghosted him — pretty childish behavior for the richest man on the planet, we have to say. But Jack might just have the last laugh, as an Orlando-based private jet chartering company has now offered him a job. Seems like his Twitter-bot and the resulting kerfuffle is a real resume builder, so job-seekers should take note.

Here’s hoping that you have a better retirement plan than NASA. The space agency announced its end-of-life plans for the International Space Station this week, the details of which will just be a run-up to the 2031 de-orbit and crash landing of any remaining debris into the lonely waters of Point Nemo. The agency apparently sees the increasingly political handwriting on the ISS’s aging and sometimes perforated walls, and acknowledges that the next phase of LEO space research will be carried out by a fleet of commercial space stations, none of which is close to existing yet. Politics aside, we’d love to dig into the technical details of the plan, and see exactly what will be salvaged from the station before its fiery demise, if anything. The exact method of de-orbiting too would be interesting — seems like the station would need quite a bit of thrust to put on the brakes, and might need the help of a sacrificial spacecraft.

“You break it, you fix it,” is a philosophy that we Hackaday types are probably more comfortable with than the general public, who tend to leave repairs of broken gear to professionals. But that philosophy seems to be at the core of Google’s new Chromebook repair program for schools, which encourages students to fix the Chromebooks they’re breaking in record numbers these days. Google is providing guidance for schools on setting up complete Chromebook repair facilities, including physical layout of the shop, organization of workflows, and complete repair information for at least a couple of popular brands of the stripped-down laptops. Although the repairs are limited to module-level stuff, like swapping power supplies, we still love the sound of this. Here’s hoping that something like this can trigger an interest in electronics for students that would otherwise never think to open up something as complicated as a laptop.

Back in July, we took note of a disturbing report of an RTL-SDR enthusiast in Crimea who was arrested for treason, apparently based on his interest in tracking flights and otherwise monitoring the radio spectrum. Now, as things appear to be heating up in Ukraine again, our friends at RTL-SDR.com are renewing their warning to radio enthusiasts in the area that there may still be risks. Then as now, we have little interest in the politics of all this, but in light of the previous arrest, we’d say it pays to be careful with how some hobbies are perceived.

And finally, aside from the aforementioned flight-tracking dustup, it’s been a tough week for Elon and Tesla. Not only have 817,000 of the expensive electric vehicles been recalled over something as simple as a wonky seatbelt chime, but another 54,000 cars are also being recalled for a software bug that causes them to ignore stop signs in “Full Self-Driving” mode. We’re not sure if this video of this Tesla hell-ride has anything to do with that bug, but it sure illustrates the point that FSD isn’t really ready for prime time. Then again, as a former Boston resident, we can pretty safely say that what that Tesla was doing isn’t really that much different than the meat-based drivers there.

Gaming Twitter’s Trending Algorithm To Make A Point

If you have ever taken to Twitter to gauge the zeitgeist, you’ll have noticed that among the trending hashtags related to major events of the day there are sometimes outliers of minority interest associated with single-issue causes. When a cause with a distasteful pedigree was cited one as proof of widespread public support in a debate in the UK’s House of Lords there were concerns raised that a flaw in the ranking algorithm might be responsible, and it was left to [Mallory Moore] to prove the hypothesis by getting a #ThisIsAnExploit hashtag trending without a groundswell of popular support.

Some previous detective work had established that equal ranking might be awarded equally not simply for Tweeting a hashtag but also for retweeting it. The exploit takes advantage of this by means of a relatively small cadre of people all Tweeting the tag a number of times, then retweeting all other instances of it. The resulting rank gain is then in the order of the square of the number of accounts interacting with the tag, and thus hugely inflated over the number of real participants. To test this she created the #ThisIsAnExploit tag and asked her followers to do just that: Tweet it and retweet all others containing it. In a short time the exploit succeeded, beating a very high-profile tag associated with the travails of the British Prime Minister in the process, and with most of the effort due to only 50 accounts.

Our world is now significantly influenced by social media because for many it appears more trustworthy than the old-style mass media with a print origin. Work like this is important because a reminder that transferring the message from newspaper proprietors to tech barons does not confer credibility is sorely needed. Meanwhile now the weakness is in the wild we wonder how Hackaday readers might have fun with it. Does anyone want to see a #RaiseTheJollyWrencher hashtag top the pile?

People in meeting, with highlights of detected phones and identities

Machine Learning Detects Distracted Politicians

[Dries Depoorter] has a knack for highly technical projects with a solid artistic bent to them, and this piece is no exception. The Flemish Scrollers is a software system that watches live streamed sessions of the Flemish government, and uses Python and machine learning to identify and highlight politicians who pull out phones and start scrolling. The results? Pushed out live on Twitter and Instagram, naturally. The project started back in July 2021, and has been dutifully running ever since, so by now we expect that holding one’s phone where the camera can see it is probably considered a rookie mistake.

This project can also be considered a good example of how to properly handle confidence in results depending on the application. In this case, false negatives (a politician is using a phone, but the software doesn’t detect it properly) are much more acceptable than false positives (a member gets incorrectly identified, or is wrongly called-out for using a mobile device when they are not.)

Keras, an open-source software library, is used for the object detection and facial recognition (GitHub repository for Keras is here.) We’ve seen it used in everything from bat detection to automatic trash sorting, so if you’re interested in machine learning applications, give it a peek.

DOOM Played By Tweet

Getting DOOM to run on hardware it was never intended to run on is a tradition as old as time. Old cell phones, embedded systems, and ancient televisions have all been converted to play this classic first-person shooter. This style of playing games on old hardware might be passé now as the new trend seems to be the ability to play this game on more ethereal platforms instead. This project brings DOOM to Twitter.

The gameplay is a little nontraditional as well. To play the game, a tweet needs to be sent with specific instructions for the bot. The bot then plays the game according to its instructions and then tweets a video. By responding to this tweet with more instructions, the player can continue the game tweet-by-tweet. While slightly cumbersome, it does have the advantage of allowing a player to resume any game simply by responding to the tweet where they would like to start. Behind the scenes of the DOOM-playing Twitter bot is interesting as well and the code is available on the project’s GitHub page.

While we’ve seen plenty of DOOM instances on all kinds of hardware, it’s safe to say we’ve never really seen a gameplay experience quite like this one. It may stay as a curiosity, but DOOM porters are always looking for something else to run this classic game so it may eventually branch out or develop into something more user-friendly like this cloud-based Atari 2600.

Access An 8-bit Atari Through Twitter

Building a retro computer, or even restoring one, is a great way to understand a lot of the fundamentals of computing. That can take a long time and a lot of energy, though. Luckily, there is a Twitter bot out there that can let you experience an old 8-bit Atari without even needing to spin up an emulator. Just tweet your program to the bot, and it outputs the result.

The bot was built by [Kay Savetz] and accepts programs in five programming languages: Atari BASIC, Turbo-Basic XL, Atari Logo, Atari PILOT, and Atari Assembler/Editor, which was a low-level assembly-type language available on these machines. The bot itself runs on a Raspberry Pi with the Atari 800 emulator, rather than original hardware, presumably because it’s much simpler to get a working network connection on a Pi than on a computer from the 80s. The Pi runs a python script that polls Twitter every two minutes and then hands the code off to the emulator.

[Kay]’s work isn’t limited to just Ataris, though. There’s also an Apple II BASIC bot for all the Apple fans out there that responds to programs written in AppleSoft BASIC. While building your own retro system or emulating one on other hardware is a great exercise, it’s also great that there are tools like these that allow manipulation of retro computers without having to do any of the dirty work ourselves.

Twitter: It’s Not The Algorithm’s Fault. It’s Much Worse.

Maybe you heard about the anger surrounding Twitter’s automatic cropping of images. When users submit pictures that are too tall or too wide for the layout, Twitter automatically crops them to roughly a square. Instead of just picking, say, the largest square that’s closest to the center of the image, they use some “algorithm”, likely a neural network, trained to find people’s faces and make sure they’re cropped in.

The problem is that when a too-tall or too-wide image includes two or more people, and they’ve got different colored skin, the crop picks the lighter face. That’s really offensive, and something’s clearly wrong, but what?

A neural network is really just a mathematical equation, with the input variables being in these cases convolutions over the pixels in the image, and training them essentially consists in picking the values for all the coefficients. You do this by applying inputs, seeing how wrong the outputs are, and updating the coefficients to make the answer a little more right. Do this a bazillion times, with a big enough model and dataset, and you can make a machine recognize different breeds of cat.

What went wrong at Twitter? Right now it’s speculation, but my money says it lies with either the training dataset or the coefficient-update step. The problem of including people of all races in the training dataset is so blatantly obvious that we hope that’s not the problem; although getting a representative dataset is hard, it’s known to be hard, and they should be on top of that.

Which means that the issue might be coefficient fitting, and this is where math and culture collide. Imagine that your algorithm just misclassified a cat as an “airplane” or as a “lion”. You need to modify the coefficients so that they move the answer away from this result a bit, and more toward “cat”. Do you move them equally from “airplane” and “lion” or is “airplane” somehow more wrong? To capture this notion of different wrongnesses, you use a loss function that can numerically encapsulate just exactly what it is you want the network to learn, and then you take bigger or smaller steps in the right direction depending on how bad the result was.

Let that sink in for a second. You need a mathematical equation that summarizes what you want the network to learn. (But not how you want it to learn it. That’s the revolutionary quality of applied neural networks.)

Now imagine, as happened to Google, your algorithm fits “gorilla” to the image of a black person. That’s wrong, but it’s categorically differently wrong from simply fitting “airplane” to the same person. How do you write the loss function that incorporates some penalty for racially offensive results? Ideally, you would want them to never happen, so you could imagine trying to identify all possible insults and assigning those outcomes an infinitely large loss. Which is essentially what Google did — their “workaround” was to stop classifying “gorilla” entirely because the loss incurred by misclassifying a person as a gorilla was so large.

This is a fundamental problem with neural networks — they’re only as good as the data and the loss function. These days, the data has become less of a problem, but getting the loss right is a multi-level game, as these neural network trainwrecks demonstrate. And it’s not as easy as writing an equation that isn’t “racist”, whatever that would mean. The loss function is being asked to encapsulate human sensitivities, navigate around them and quantify them, and eventually weigh the slight risk of making a particularly offensive misclassification against not recognizing certain animals at all.

I’m not sure this problem is solvable, even with tremendously large datasets. (There are mathematical proofs that with infinitely large datasets the model will classify everything correctly, so you needn’t worry. But how close are we to infinity? Are asymptotic proofs relevant?)

Anyway, this problem is bigger than algorithms, or even their writers, being “racist”. It may be a fundamental problem of machine learning, and we’re definitely going to see further permutations of the Twitter fiasco in the future as machine classification is being increasingly asked to respect human dignity.

Community Testing Suggests Bias In Twitter’s Cropping Algorithm

With social media and online services are now huge parts of daily life to the point that our entire world is being shaped by algorithms. Arcane in their workings, they are responsible for the content we see and the adverts we’re shown. Just as importantly, they decide what is hidden from view as well.

Important: Much of this post discusses the performance of a live website algorithm. Some of the links in this post may not perform as reported if viewed at a later date. 

The initial Zoom problem that brought Twitter’s issues to light.

Recently, [Colin Madland] posted some screenshots of a Zoom meeting to Twitter, pointing out how Zoom’s background detection algorithm had improperly erased the head of a colleague with darker skin. In doing so, [Colin] noticed a strange effect — although the screenshot he submitted shows both of their faces, Twitter would always crop the image to show just his light-skinned face, no matter the image orientation. The Twitter community raced to explore the problem, and the fallout was swift.

Continue reading “Community Testing Suggests Bias In Twitter’s Cropping Algorithm”