Artificial Intelligence

353 Articles

Local LLMs Can Work Better Than Claude, At Least For Some

September 9, 2026 by 59 Comments

This is going to be a very personal question, because when you’re talking cloud vs local anything, it comes down to this: how much are you willing to pay for independence? The local option might save you long term, or it might never pay off the capital investment. It will almost certainly cost you your time to set up and maintain your own system — but what you get back is independence. With LLMs, traditionally you lose quite a bit of performance, but as [Anurag Singh] points out on XDA Developers, a lesser model might actually let you get more done, depending on your workflow.

[Anurag] had been on the 20$/month plan with Anthropic when he decided that the scratch just wasn’t worth the sniff– he was hitting usage limits he couldn’t stand at that level, but couldn’t justify a higher tier of access. So he decided to try a local LLM, even though all he had was a 16 GB MacBook Air M5, not a beefy workstation. Since his workflow isn’t so much ‘vibe code the whole thing for me’ as ‘help me find where I went wrong here, electronic rubber duck’, Qwen2.5 Coder 14B proved more than adequate for his use case.

It can’t understand all the moving parts of a large project as well as Claude can — not surprising given how old it is and how much memory it has to work with — but that’s [Anurag]’s job. He’s the programmer, it’s just the assistant. For his use case, he can make use of his existing hardware and having the the LLM right in VS Code is allows for a speedy workflow.

Your millage may vary, but if you want to get into locally running LLMs, we can point you at the easy ways to get started. Depending on your hardware, you might want to grab another GPU.

How To Talk To A Machine Without Anthropomorphising It

September 7, 2026 by 45 Comments

LLMs remain a divisive topic in these times. Perhaps we all know someone who’s become over-infatuated with their new robotic friend, or who believes it has made them a genius. [Emily M. Bender] and [Nanna Inie] have written about how people anthropomorphise the LLMs they interact with, and suggested some language tips to avoid that. It’s a couple of months old, but we think Hackaday readers will find it interesting.

Their analysis is interesting, because it looks at the way people talk about LLMs and highlights the unconscious anthropomorphism. The LLM is a piece of software not a person, so why does it “recognise” when it does “speech recognition”, for example. They suggest “automatic transcription” instead. Even “hallucination” implies cognisance that evidently isn’t there. They admit that their suggestion of “undesirable output” isn’t entirely appropriate. They’re on safer ground with “input” and “output” instead of “prompt” and “response”.

Whatever your views on them, it’s evident that LLM usage will be a feature of the world for the forseeable future. The language surrounding them is however capable of evolving, and maybe some of the suggestions here are worth taking note of.

Grappling with our new electronic overlords? Have a look at our AI for Skeptics series.

Can AI Now Design PCBs That Just Work?

September 5, 2026 by 64 Comments

With the recent release of its GPT-6 Astra model, OpenAI explicitly pushed the claim that it is capable of designing complete circuit boards in KiCad, starting from a provided schematic and outputting a fully routed PCB that theoretically could be sent off to be manufactured. This of course raises the question whether this is just a nifty party trick that works under strictly controlled conditions like most auto-routing tools, or whether there’s more to it. In a recent [EEBench] blog post, OpenAI’s claims are put to the test.

Continue reading “Can AI Now Design PCBs That Just Work?” →

A grid of images shows pictures emerging from patches of random noise. To the left, images are more random, while to the right they become more recognizable.

Running Generative AI On An RP2350

September 5, 2026 by 22 Comments

Driven by a desire for privacy, customization, and lower costs, there’s growing interest in AI models which can be run on local hardware. Few of them go as far as [Tim], though, who built an image generation diffusion model which can run on an RP2350 microcontroller.

As might be expected, its capabilities are limited. The resolution is 128×128, it only generates images of human faces, and it takes about twenty seconds per image – still impressive for such limited hardware. It runs on a Waveshare RP2350 development board, and it can output the generated image over USB or display it with the aid of a VGA adapter board.

The generative model doesn’t directly create an image. Rather, it generates a distribution in a latent space, which a variational auto-encoder’s decoder component translates into an image. The auto-encoder was trained in two parts: an encoder which transforms an image into a latent-space distribution, and a decoder to transform that distribution back to an image; once this was trained, only the decoder was used.

The generative portion of the model uses a latent flow diffusion transformer; this takes in noise to start with, then iteratively predicts changes which bring it toward the desired image. It can also take in a output class, which guides the generator’s direction (toward a smiling face, for example). [Tim] trained two models, one larger and one faster, and quantized the weights for both to 8-bit integers. Both models, along with the inference program, then fit into 4 MB of flash memory.

For such a small model, the results are remarkably good; they don’t look quite natural, but they’re quite recognizable. For more on how diffusion image generators work, check out our article on Stable Diffusion.

The Birds Outside, Drawn For You Automatically

September 4, 2026 by 23 Comments

With artificial intelligence being the bête noir of the moment, there are some projects using it which maybe don’t bring much to the table. So it’s nice to see one that uses it in a creative way, and delivers something new. [arnegiacomo]’s e-paper screen is a great example, as it draws a picture in real time of whatever birds are outside.

Behind the quite large screen sits a Raspberry Pi 5, and on that is BirdNet-Go, an AI-powered birdsong classifier. A USB microphone catches the birdsong, and Birdnet comes up with the species. The birds on the display are then those species as pictures from 19th century bird spotters guides, assembled into a collage. You can even see what the current set of birds it hears are, live, and they are a representative cross section of the European birds you’d find in Norway where it’s located.

We like this project, both for the bird book vibe it gives, and the creative use of machine classification. Surprisingly this isn’t the first project in this field we have seen over the years.

From A Ten-Line Script To A Real Utility With Codex

September 3, 2026 by 68 Comments

I’m an experienced programmer, and I’ve worked in many different languages. Sometimes being a programmer is a two-edged sword. You want to accomplish something, and you can do it easily — but it can be a lot of work to do it right. Maybe more work than you want to do.

Normally, I’ll kick out a few lines of script for something I want and be done, accepting that it isn’t production-hardened. This time, however, I decided to try an AI tool to see whether they could do the work I was too lazy to do myself. While I’ve played with chatbots, I wanted to try one of the dedicated coding agents, in this case, Codex. Outside of asking ChatGPT to write a simple function or find the cause of an error message, I haven’t done much coding with AI assistance, so I was interested to see what these agents brought to the table.

A Radio Problem

The problem was simple: I wanted an easy way to put buttons on my Linux desktop that launched Internet radio stations. Sure, I could open a player and paste in a long URL, but I’m far too lazy to remember all those URLs.

I searched for a way to make Shortwave — an Internet radio player — open a URL from the command line. Apparently, you can’t. Google Gemini suggested writing a script that launches cvlc, the command-line VLC player, with the URL as an argument.

That’s easy, so I did it. Of course, then I had to find the stream URLs for all my favorite stations. It turns out that Radio Browser maintains an extensive database of stations. I considered scraping the site or using its API, but honestly, the little script was becoming too much of a project.

Besides, I was already struggling to manage the media player’s lifetime. I didn’t want a new station playing on top of one that was already running, and I wanted a command to stop playback, so the script had already grown larger than I first imagined.

My first version used a temporary file containing the player’s process ID so a future script execution could kill the old player. That usually works, but it isn’t very robust, and I knew it. But how much work did I really want to do here? I decided I had done enough and turned the rest over to Codex, OpenAI’s coding assistant.

Continue reading “From A Ten-Line Script To A Real Utility With Codex” →

LLM Moats Quickly Evaporating

August 30, 2026 by 55 Comments

In the business world, a moat is a quality of a business that makes it difficult for competitors to take that company’s profits. With how hard it is to train models for large language models (LLMs) and generative AI, it might seem like Anthropic, Open AI, and other LLM companies would have huge moats given the amount of compute it takes to build models. But open source models are quickly draining that moat, and now the only thing standing in the way of a customer using one of these models on their own hardware instead one from the larger companies is physical computing resources. [TerminalBytes] demonstrates a few of these models on personally owned computers to show the current state of the art.

[TerminalBytes] started off running the 27B version of the Qwen3.8 on a Mac Studio with 256 GB of unified RAM, which is plenty for this task. But it’s also enough to benchmark a few different models. Qwen3.6 is compared to 3.8, and then the different quants of each model are also compared. Quants are compressed versions of models that need fewer bits to store weights, meaning that the same models can run in less memory with smaller losses in fidelity. Many of these quants run on machines with 32 GB of RAM or less, encompassing many average gaming PCs. There’s even a 1-bit quant that [TerminalBytes] tested which can easily run on a machine with 16 GB, although with mixed results.

Keep in mind that this is just the current state of affairs with open LLMs. Future versions of these models are likely to optimize the number of tokens produced per unit time, or otherwise increase quality of responses while requiring less computer resources. We don’t really think that the ease of running local models will be the sole reason that the AI bubble pops, though. The fact that not every computer user is running Linux is proof enough of that.