A graph showing the poisoning success rate of 7B and 13B parameter models

It Only Takes A Handful Of Samples To Poison Any Size LLM, Anthropic Finds

It stands to reason that if you have access to an LLM’s training data, you can influence what’s coming out the other end of the inscrutable AI’s network. The obvious guess is that you’d need some percentage of the overall input, though exactly how much that was — 2%, 1%, or less — was an active research question. New research by Anthropic, the UK AI Security Institute, and the Alan Turing Institute shows it is actually a lot easier to poison the well than that.

We’re talking parts-per-million of poison for large models, because the researchers found that with just 250 carefully-crafted poison pills, they could compromise the output of any size LLM. Now, when we say poison the model, we’re not talking about a total hijacking, at least in this study. The specific backdoor under investigation was getting the model to produce total gibberish.

Continue reading “It Only Takes A Handful Of Samples To Poison Any Size LLM, Anthropic Finds”

Weird Email Appliance Becomes AI Terminal

The Landel Mailbug was a weird little thing. It combined a keyboard and a simple text display, and was intended to be a low-distraction method for checking your email. [CiferTech] decided to repurpose it, though, turning it into an AI console instead.

The first job was to crack the device open and figure out how to interface with the keyboard. The design was conventional, so reading the rows and columns of the key matrix was a cinch. [CiferTech] used PCF8574 IO expanders to make it easy to read the matrix with an ESP32 microcontroller over I2C. The ESP32 is paired with a small audio output module to allow it to run a text-to-speech system, and a character display to replace the original from the Mailbug itself. It uses its WiFi connection to query the ChatGPT API. Thus, when the user enters a query, the ESP32 runs it by ChatGPT, and then displays the output on the screen while also speaking it aloud.

[CiferTech] notes the build was inspired by AI terminals in retro movies, though we’re not sure what specifically it might be referencing. In any case, it does look retro and it does let you speak to a computer being, of a sort, so the job has been done. Overall, though, the build shows that you can build something clean and functional just by reusing and interfacing a well-built commercial product.

Continue reading “Weird Email Appliance Becomes AI Terminal”

Why LLMs Are Less Intelligent Than Crows

The basic concept of human intelligence entails self-awareness alongside the ability to reason and apply logic to one’s actions and daily life. Despite the very fuzzy definition of ‘human intelligence‘, and despite many aspects of said human intelligence (HI) also being observed among other animals, like crows and orcas, humans over the ages have always known that their brains are more special than those of other animals.

Currently the Cattell-Horn-Carroll (CHC) theory of intelligence is the most widely accepted model, defining distinct types of abilities that range from memory and processing speed to reasoning ability. While admittedly not perfect, it gives us a baseline to work with when we think of the term ‘intelligence’, whether biological or artificial.

This raises the question of how in the context of artificial intelligence (AI) the CHC model translate to the technologies which we see in use today. When can we expect to subject an artificial intelligence entity to an IQ test and have it handily outperform a human on all metrics?

Continue reading “Why LLMs Are Less Intelligent Than Crows”

In Which I Vibe-Code A Personal Library System

When I was a kid, I was interested in a number of professions that are now either outdated, or have changed completely. One of those dreams involved checking out books and things to patrons, and it was focused primarily on pulling out the little card and adding a date-due stamp.

Of course, if you’ve been to a library in the last 20 years, you know that most of them don’t work that way anymore. Either the librarian scans special barcodes, or you check materials out yourself simply by placing them just so, one at a time. Either way, you end up with a printed receipt with all the materials listed, or an email. I ask you, what’s the fun in that? At least with the old way, you’d usually get a bookmark for each book by way of the due date card.

As I got older and spent the better part of two decades in a job that I didn’t exactly vibe with, I seriously considered becoming a programmer. I took Java, Android, and UNIX classes at the local junior college, met my now-husband, and eventually decided I didn’t have the guts to actually solve problems with computers. And, unlike my husband, I have very little imagination when it comes to making them do things.

Fast forward to last weekend, the one before Thanksgiving here in the US. I had tossed around the idea of making a personal library system just for funsies a day or so before, and I brought it up again. My husband was like, do you want to make it tonight using ChatGPT? And I was like, sure — not knowing what I was getting into except for the driver’s seat, excited for the destination.

Continue reading “In Which I Vibe-Code A Personal Library System”

Kubernetes Cluster Goes Mobile In Pet Carrier

There’s been a bit of a virtualization revolution going on for the last decade or so, where tools like Docker and LXC have made it possible to quickly deploy server applications without worrying much about dependency issues. Of course as these tools got adopted we needed more tools to scale them easily. Enter Kubernetes, a container orchestration platform that normally herds fleets of microservices in sprawling cloud architectures, but it turns out it’s perfectly happy running on a tiny computer stuffed in a cat carrier.

This was a build for the recent Kubecon in Atlanta, and the project’s creator [Justin] wanted it to have an AI angle to it since the core compute in the backpack is an NVIDIA DGX Spark. When someone scans the QR code, the backpack takes a picture and then runs it through a two-node cluster on the Spark running a local AI model that stylizes the picture and sends it back to the user. Only the AI workload runs on the Spark; [Justin] also is using a LattePanda to handle most of everything else rather than host everything on the Spark.

To get power for the mobile cluster [Justin] is using a small power bank, and with that it gets around three hours of use before it needs to be recharged. Originally it was planned to work on the WiFi at the conference as well but this was unreliable and he switched to using a USB tether to his phone. It was a big hit with the conference goers though, with people using it around every ten minutes while he had it on his back. Of course you don’t need a fancy NVIDIA product to run a portable kubernetes cluster. You can always use a few old phones to run one as well.

Continue reading “Kubernetes Cluster Goes Mobile In Pet Carrier”

An AI By Any Other Name

While there are many AI programs these days, they don’t all work in the same way. Most large language model “chatbots” generate text by taking input tokens and predicting the next token of the sequence. However, image generators like Stable Diffusion use a different approach. The method is, unsurprisingly, called diffusion. How does it work? [Nathan Barry] wants to show you, using a tiny demo called tiny-diffusion you can try yourself. It generates — sort of — Shakespeare.

For Stable Diffusion, training begins with an image and an associated prompt. Then the training system repeatedly adds noise and learns how the image degenerates step-by-step to noise. At generation time, the model starts with noise and reverses the process, and an image comes out. This is a bit simplified, but since something like Stable Diffusion deals with millions of pixels and huge data sets, it can be hard to train and visualize its operation.

The beauty of tiny-diffusion is that it works on characters, so you can actually see what the denoising process is doing. It is small enough to run locally, if you consider 10.7 million parameters small. It is pretrained on Tiny Shakespeare, so what comes out is somewhat Shakespearean.

Continue reading “An AI By Any Other Name”

A photo of the LEGO sorter

Making A Machine To Sort One Million Pounds Of LEGO

You know what’s not fun? Sorting LEGO. You know what is fun? Making a machine to sort LEGO! That’s what [LegoSpencer] did, and you can watch the machine do its thing in the video below.

[Spencer] runs us through the process: first, quit your day job so you can get a job playing with LEGO; then research what previous work has been done in this area (plenty, it turns out); and then commit to making your own version both reproducible and extensible.

A sorting machine needs three main features: a feeder to dispense one piece at a time, a classifier to decide the type of piece, and a distributor to route the piece to a bin. Of course, the devil is in the details.

Continue reading “Making A Machine To Sort One Million Pounds Of LEGO”