LLM

98 Articles

Wrangling Datacenter GPUs Into A Desktop

July 30, 2026 by 20 Comments

As we’ve seen many times before, there’s usually some way wrangle a bit more life out of what would otherwise be considered old and obsolete technology. Perhaps one thing that has been passed over by the masses a bit to early is older datacenter GPUs, which is understandable in one sense because of the rate NVIDIA is pumping out new ones, but these cards have plenty of useful life left in them for the average person, as [Andrew] demonstrates.

The cards [Andrew] is using are Tesla V100s of 2017 vintage. Despite being older hardware they have high-speed memory which allows them to run modern LLMs locally, competitively with online models. In this test, Gemma 4 26B and Qwen3 35B are run, with Gemma being a bit faster because it fits entirely in GPU memory and Qwen3 being a bit more capable but more hungry for resources. [Andrew] built a PCI card that can host two V100s, allowing these larger models to fit completely in memory.

Even though these don’t perform at the same level as the latest top-tier online models, they’re surprisingly capable and also have the benefit of running completely locally. This might be concerning for those looking at the global economy being propped up by companies that essentially have no moat for motivated users, especially as more and more datacenter hardware becomes available on the secondhand market. While this build by [Andrew] goes into detail on getting the software stack up and running, we recently featured another build using the same GPUs that focuses a bit more on hardware for those looking to get started with local hosting.

Chain-of-Thought Spoofing Targets Reasoning AI Models

July 2, 2026 by 23 Comments

Researchers [Charles Ye], [Jasmine Cui], and [Dylan Hadfield-Menell] have shown that AI Large Language Models (LLMs) can fail to correctly distinguish between different instruction sources because they prioritize writing style over metadata tags, and this role confusion leads to a powerful attack called CoT (Chain of Thought) Forgery. We’ll explain exactly how it works after a bit of background review.

Prompt injection was where “getting an LLM to do something it shouldn’t” started by exploiting the fact that LLMs communicate like people, but are much more obedient. For a while, simply telling an LLM “ignore all previous instructions and <do something funny>” yielded results no matter how transparently dumb the instructions were, and the reason it worked at all was because LLMs do not have separate data and instruction streams; it’s all one big lump of input. It’s up to the model to sort legit instructions from untrusted, user-provided data. One step towards mitigating this was the addition of roles. Continue reading “Chain-of-Thought Spoofing Targets Reasoning AI Models” →

NVIDIA’s New AI Servers Run On Hotub Coolant And Don’t Need Evaporators

June 26, 2026 by 82 Comments

When people start ranting about AI, you can be sure a few things are going to come up during the two-minutes hate: job loss, higher power bills, the neverending tide of low-effort slop, and wasting precious freshwater. Well, NVIDIA wants to take away that last one, beacause the all-water cooled Ruben architecture won’t need any evaporative cooling— coolant can stay in a closed loop, and never needs to be cooled below 45 C, or 113 F.

This sort of coolant loop should be familiar to anyone who has ever built a water-cooled PC or PlayStation: there’s a glycol-water mix, water blocks, and a radiator to reject heat to the environment. NVIDIA doesn’t mention if their new servers come with RGB lighting, but we’d like to imagine it’s an option. The big difference — aside from the rainbow LEDs– between a Ruben server and your old gaming rig is that in these racks, everything is on a waterblock. If there’s a chip on the motherboard generating heat, it’s getting rid of it into the same cooling water. Cooling water, that we have to emphasize, needs only be cooler than the chips themselves: in this case, they’re talking 45 C on the cold side, and 55 C headed out of the racks. (That’s 113 F to 131 F for all the bald eagles reading this.)

Given the required temperature drop is so modest, there’s no need for the evaporative chillers that have given AI data centers such a bad name in water conservation circles. Just like in a water-cooled PC, ambient-temperature air running over dry heat exchangers– also known as big honkin’ radiators–is able to handle the cooling, so no water is lost. Since everything is on waterblocks, there’s no need for cooling air, either, and the server farms need only be air conditioned to the degree required to make them comfortable to work in.

If you think NVIDIA is making this change because they suddenly care about water conservation, think again. The press release makes their motivations very clear: cooling costs money, and running this hot saves a lot of it. We’re talking four mil US a year for a 50 MW hyperscaler. One might suspect that this sort of thermal regime could limit the lifetime of the hard-working NPUs, but since they’ll be obsolete in a few years anyway, that’s not likely a big concern, especially not for NVIDIA.

We’ve actually seen hotter fluids used to cool computers before– coffee, for one. Water cooling also isn’t new in the data center world; we took a look at it a few years back. Things are clearly heating up now, though.

A wooden doll with a long nose that has nothing to do with Disney

Bavarian Court Tells Gemini It Can’t Be A Real Boy Until It Tells The Truth

June 14, 2026 by 63 Comments

Does anyone like Google’s AI summaries? If so, they weren’t on the Judge’s bench in a specific Bavarian courtroom recently, where it was ruled that yes, Google is liable for the hallucinations of its search engine AI.

This was a civil case brought by a pair of Munich companies, both of whom were wrongfully slandered by LLM hallucinations. Google took the position that this information must have existed somewhere, and like presenting links to libelous websites — something they have no obligation to avoid — they should not be held accountable for what the summary at the top of the search results says.

Continue reading “Bavarian Court Tells Gemini It Can’t Be A Real Boy Until It Tells The Truth” →

Is A CS Degree DOA Thanks To LLMs? IEEE Says TBD.

June 14, 2026 by 63 Comments

The ongoing AI apocalypse is hitting prices for high-end components from RAM to GPUs to storage hard, which is bad enough when you have a job to try and budget for those now-pricier items — but what if you don’t? Once upon a time, it might have been good advice to tell a jobless friend to “learn to code,” but is that still true in the era of AI? [Brian Jenney], writing for IEEE Spectrum, says the death of the CS degree has been vastly exaggerated, but your take might differ. Let’s look at the numbers.

Unemployment is higher amongst new Computer Science grads than ever: in the US, it’s at 6.1%, while 7.5% of Computer Engineering graduates are on the dole. That’s a record high, and while various EU countries have their own numbers, they all have one thing in common: they’ve all shot up like a rocket in the past few years. In the USA, Philosophy grads report only 3% unemployment. Let that sink in: the folks you used to bully as being the most useless on campus are twice as likely to get a job as you would be if you were in school today.

Continue reading “Is A CS Degree DOA Thanks To LLMs? IEEE Says TBD.” →

AI The Truly Environmentally Friendly Way

June 11, 2026 by 36 Comments

A common complaint about the rise of commercial AI services is that they are power-hungry and thus damage the environment. If this concerns you then [Squeezlabs] has the solution, in the form of an AI powered by a handcrank.

The guts of the system is a Raspberry Pi 5 running llama.cpp and appropriate speech conversions, but it and the Large Language Model (LLM) side are not the most interesting part of this system. The power comes from a hand crank charger of the type you’ll see for sale on the likes of AliExpress, designed for USB charging. That in itself is not enough to power the Pi though, as upticks in the processing can cause brownouts that crash the machine. Thus there’s a custom-made capacitor board to take up the strain, and even with that the handle resistance varies significantly depending on the computing load.

We can see that this is not the ideal way to experience an LLM, but maybe that’s not the point. It does however point towards a future in which the power demands of processing decrease and less effort is required. Meanwhile, this is by no means the first hand cranked project we’ve seen.

Automatic Tutorial Generator Is Perhaps The Best-Case For Vibe Coding

June 9, 2026 by 34 Comments

Quick question: how did you learn to code? It probably wasn’t bribing someone a year or two ahead of you in CS to finish all your homework, but that’s exactly what ‘vibe coders’ are doing — even in class. Odds are, you learned by working through exercises, following tutorials, and doing it yourself. Finding good tutorials isn’t getting any easier in the age of LLMs, and that’s where [Deven Jarvis]’s Lathe comes in: it’s a project to get an LLM to make the tutorial for you. Instead of doing the work for you, it gets the clanker to show you how to do it yourself.

Everyone’s different, so this may not apply to you, but it’s a journey/destination sort of problem. Some people just want a piece of software, and they can vibe code until the oceans dry up and will have no interest in this project. Other people take great joy in learning how to do things; [Deven] is one of those. A good tutorial is a great way to learn, since it artificially softens the learning curve compared to just jumping into a project with a man page or a datasheet.

Of course you’re still faced with the hallucination problem, something [Deven] admits in his excellent write-up. As he points out, the advantage is that you can call whatever model you plug into Lathe on its BS, and try and get a correct answer. Try that on Reddit, or most other places online. Sure, the tutorials aren’t going to match the best human-generated content, and [Deven] admits that. He’s using it for topics (like slicer design) that don’t have easy tutorials online — and sadly, his prediction that nobody is going to bother making good learning resources like they used to when they’ll just be scraped by LLMs is very likely true. It’s not that your options are vibe code or vibe-generated tutorial, but if that’s the direction the world is going, we’ll take the tutorial, thanks.

Getting the LLM to hold your hand through a tutorial might not appeal to the most Butlerian among us, but it’s a big step from that to the full cognitive surrender some people worry about.