Claude Plays DOOM

Large language models (LLMs) are generally thought of as machines that accept textual prompts and spit out textual content. However, if you’re creative in the way you interface with them, you can get them to do a wider range of tasks. For example, [Andrea Ricci] figured out how to get one to play DOOM.

For this project, [Andrea] began by porting the game to the SCINTIX P4. It’s a rather interesting device, being a single board designed in the Raspberry Pi CM4/CM5 form factor, but carrying an ESP32-P4 and an ESP32-C6 instead. The game runs on the P4 and is displayed on a 1024×600 MIPI DSI panel, but it’s only stepped through a few frames at a time. These frames are then passed to Claude Sonnet via a WebSockets setup. With only the same information as a human player would get, the LLM has to figure out what it’s looking at, and then respond with movement and fire commands to play the game.

It’s quite interesting to watch the system play—the LLM mostly accurately describes the game world, navigates down corridors, opens doors, and shoots at enemies. There is a bit of work behind the scenes to enable it to see and understand the game world—namely, using a depth fan across the field of view so it can figure out where walls are and how not to bang into them. There’s also an ASCII automap used to allow the system to keep track of where it has already been. But fundamentally, the LLM is playing the game without any other sort of additional assistance.

We’ve seen some other great ways in which AIs have been whipped up to play various games, like Trackmania.

Continue reading “Claude Plays DOOM

Cheap AI Token Resellers: The Secret Ingredient Is Fraud

[Matt Lenhard] has an interesting writeup explaining exactly how fraudsters offer access to cutting-edge AI models at a tenth of the price. Perhaps unsurprisingly, the secret is to get tokens for free from anywhere they can and by any means necessary. Then wrap them in a pretty relay API, and sell access to it.

Relaying tokens is not by itself a shady practice. That distinction belongs to services that obtain tokens fraudulently, opening the door to selling them at rates far below market value. This practice is widespread and profitable, in part because the abuse is so hard to pin down and stop.

One source of tokens is free credits on new accounts. New accounts are spooled up as fast as possible, hammered until they’re empty, then it’s done all over again. Another method is to sign up as pay-after, possibly with a stolen card, and simply ensure the account has no valid payment method once the bill comes due. Or set up a temporary card, pay some minimum up front and consume as much as possible, then initiate a chargeback. It doesn’t matter if individually each of these doesn’t amount to much before they get flagged, because it’s being leveraged relentlessly on a massive scale by automated systems.

There are the shadier methods, too. Fraudsters don’t just target providers directly. Consumer software products with AI features get reverse-engineered, then the back ends hammered for all they are worth. Poorly-coded support chatbots can be highjacked into serving fraudsters’ traffic instead of just their own. It doesn’t actually matter where the tokens come from, after all. As long as the fraudsters are obtaining them for free (or at least below their costs) then it’s profit.

That last point is one [Matt] zeroes in on with advice on how to mitigate this abuse. He goes into detail in his writeup but what it comes down to is recognizing that it’s a numbers game. Fraudsters depend entirely on obtaining tokens for free, or nearly free. So just like using an AI to keep phone scammers tied up, anything that raises friction increases the fraudster’s costs, in turn encouraging them to find an easier target.

AI Book Scanning: Just What Is A Rare Book?

One of the stories of the last few weeks has been that AI companies have been scanning books in very large numbers in order to train their models with content guaranteed to have been written before 2002, and thus AI free. It’s caused some outrage, because of the size of the operation, and because the scanning process is destructive. In particular the phrase being bandied around is that these are rare books, and it’s this phraseology I find problematic. I think it’s time to unpack why that is the case.

It’s Not Book Burning, Folks

Before I worked for Hackaday I had a long career in and around the publishing industry, mostly on the electronic side, but from time to time crossing paths with my colleagues in the world of paper-based publishing. I understand the appeal of a good book, I’ve spend a lot of my life among bibliophiles, and let’s just say I own a few books myself. In particular I understand the symbolism of destroying books, bringing to mind as it does the actions of repressive regimes. I have stood in Bebelplatz in Berlin where the photo of Nazi student organisation members burning the library of Magnus Hirschfeld’s institute was taken in 1933, and if you know me, you’ll have an idea why that’s close to home. But for all that, what the AI companies are doing is not the same thing.

In this case they’re destroying the books for two reasons. Firstly, as I remember from a previous employer in the publishing world, it’s much easier to digitise a stack of papers than it is a bound book. Thus I’m pretty sure that’s one reason they remove the binding before digitising the pages. Then secondly, as I understand it it’s a copyright issue. If they buy a book, digitise it, and destroy the physical copy, they can legitimately claim that only one copy of it exists, and they hope, sidestep copyright claims from publishers. Continue reading “AI Book Scanning: Just What Is A Rare Book?”

Artificial Intelligence As It Once Was

One day, people will look back at what we call Artificial Intelligence and laugh. We do the same thing today, as chat bots totally outstrip what the computer industry called AI up until pretty recently. This didn’t escape curators at the Internet Archive, apparently, and [Jason Scott] tells us about a recent collection, “Vintage Artificial Intelligence.”

These are old software titles that will run for you in emulated machines right in your browser, ranging from somewhere in the 1970s to the 1990s. There’s Eliza, of course. Actually, there are several copies of Eliza. Given how simple it was to write Eliza, it did a pretty good job. Then there are adventure games that are pretty conversational, Lisp, Prolog, which was going to spawn expert systems to replace us all, and Racter, which tried to write fiction.

There was even Alter Ego that was supposed to help you explore life decisions, maybe? There are a couple different versions of even a few versions of Conway’s Game of Life. We aren’t sure that’s ever really been AI, but perhaps it depends on your definition. We’re happy to see Sargon, the chess program, represented.

We didn’t see Hexapawn, which is a shame. We also didn’t see Parry (the paranoid counterpart to Eliza) or that elusive software we remember but can never find that built word chains from text called George.

We’ve written about Eliza before. If you want to experiment with Prolog and you like Pokémon, you’ll appreciate this tutorial.

Machine Learning COFFIES “Hears” Sunspots Before We Can See Them

In this age of neural net “AI”, even the most skeptical of Butlerians have to agree that these machine learning models can be very, very good at pattern recognition if nothing else. NASA is on the same page, and to take advantage of that pattern recognition, they’ve built a machine learning module called COFFIES, which stands for Consequence Of Fields and Flows in the Interior and Exterior of the Sun, because at NASA everything is an acronym, or at least a backronym. Like most such names, this one is at least vaguely descriptive: the model is trying to predict what’s going on in the material flows and magnetic fields deep within our local star, and using those inferences is able to predict active regions– that’s sunspots to us chickens — up to 12 hours before they visibly form.

Continue reading “Machine Learning COFFIES “Hears” Sunspots Before We Can See Them”

ShieldFont: Bludgeoning AI Scrapers That Disrespect Robots.txt

In the more innocent days of the World Wide Web you could simply put a robots.txt file in the root of your website that search engine indexing bots and similar would consult for the indexing wishes of the site owner. In this brave new world of LLM training data indexing such pleasantries are however rarely respected, leaving site owners to resort to increasingly more involved ways to bludgeon so-called AI scrapers, with ShieldFont being one of the most recent methods.

Its basic functioning is detailed in the white paper, explaining their use of ligatures. These are normally used to join multiple graphemes or letters into a single glyph which are rendered in the final text. By substituting about a quarter of the words in a text with such ligature-based versions in an intelligent, dictionary-based manner, the HTML version – as typically parsed by a scraper – will read as grammatically valid but nonsensical text, while the rendered font version will look normal.

Naturally, there are some disadvantages to this, such as screen readers for the visually impaired needing to also use the rendered font version, and it’s just as effective on legitimate search engine indexing bots. That said, if you apply this to static, archived content, or content marked as ‘do not follow’ in said robots.txt, then it might just be one way to make ChatGPT and friends spit out really funny output in the future now that the novelty of wood glue on pizza and eating rocks has somewhat worn off.

While LLM scrapers can adapt to this by also parsing the rendered text, this makes the scraping effort significantly more expensive. Together with maze traps like Nepenthes and Cloudflare’s offerings that seek to keep these scrapers busy scraping dynamically generated content through infinite linked pages, the tools available to combat the menace of these scrapers keep developing.

Track Bird Visitors With A Raspberry Pi And A USB Mic

Avian Visitors is a lovely project by [Teddy Warner] that uses a Raspberry Pi and microphone to keep track of which birds have been visiting your home, and creates a colorful illustration of recent visitors on top of it all.

It reports on a web interface of its own making, but what really takes things to a new level is an optional, stylish E-Ink panel that shows the last 24 hours’ worth of visitors at a glance in a collage.

The key to identification is BirdNET (GitHub here), a deep learning classifier from Cornell that can reliably identify and classify more than 11,000 species worldwide based on sound alone.

Based on that information, the system pulls bird images from a reference set for the region and creates a collage representing the breadth and frequency of visitors in a single image. The larger the image of a bird, the more frequently it was heard.

That’s a cool project, but [Teddy] took things one step further by setting up a color E-Ink display to show a running summary of all the avian visitors the system identifies. [Teddy] has a knack for leveraging projects into wall-mounted art, as we saw with his generative art wall plotter. Continue reading “Track Bird Visitors With A Raspberry Pi And A USB Mic”