As technologies change and adapt, we’re often left with seemingly useless junk that has nowhere to go. Certainly anyone still sitting on a pile of floppy disks feels this way sometimes, but odds are anyone who owns a mining ASIC or an NFT can attest to that as well. The trillions of dollars flowing into GPU-based data centers will likely become the next victim of this trend, so if you want to capitalize on the losses of some venture capitalist you’ll want to figure out a way to get GPUs meant for a server into your desktop doing useful work.
Of course, calling these devices GPUs is a bit of a stretch compared to the Radeon and GeForce cards many of us are used to using for gaming. These don’t even have a PCIe slot or video output, after all. But, as [] notes, the VRAM and GPU cores are very real and can still do useful work. An adapter board is able to mate a Tesla V100 SXM2 16 GB GPU to a standard PCIe slot, which solves the first problem, but the major downside from there is that the cooling fan for this unit was literally deafeningly loud. At 82 dB it was about as loud as a lawnmower, which is fine in a server rack but not great in a bedroom. [Oscar] found a way to tamp down the fan speed, making it usable in a home.
Without video output, the utility of these cards mainly comes from adding VRAM and compute for tasks that benefit from parallel computing. Using tensor splitting, [Oscar] is running a local LLM with this card alongside his RTX 4080, providing 32 GB of VRAM on his NixOS system. With his benchmarking tests, the LLM sports impressive stats for a self-hosted model, ranking somewhere around Claude Sonnet 4.6. What’s even more impressive is that this is all done for around £200, and with the rate the various LLM companies are ratcheting up pricing could pay itself back very quickly. If trading off performance for cost is acceptable, though, it’s possible to run local models on much less powerful hardware as well.
On one hand, im a sucker for unique and quirky hardware, on the other hand during winter months I love running my PC running Folding@Home and Boinc work units, so if these GPUs become reasonable in price and they can crunch the same work units that a regular desktop GPU can, I’ll be down for one or two!
That’s literally the same hand
Ever seen Fiddler on the roof? (Yes, this is related to your comment).
“On one hand…”
“On the other hand…”
“But on the other hand…”
One Hand, Other Hand, Gripping Hand…
Argghh!! The Moties have escaped their star-system!
Figure it could also be used for self-hosting and adding more video processing power to VM’s for gaming and video
Tesla V100 hah. Its way too old for modern AI tasks. No FP4/FP8 support, Volta architecture from 2017, high power consumption even at idle. Totally not worth it.
You can run them from solar. So who cares for hobby use. They are cheap and you can combine up to 4 of them easily on a chinese carrier board.
You don’t need a last generation GPU for everything. Simple translation tasks or such can be done with much less.
The “totally not worth it” is unworthy for a hacker mentality.
I’m running qwen2.5 and gemma4 on CMP50hx and CMP90HX and even that’s stretching it , V100 is even more ancient
I’m running gemma4:12b-it-qat on a Quadro K2200, which is 12 years old at this point. Is it fast? No – 1.8-3.4 tk/sec for any real tasks. Do I care? Also no. It’s fun, and I can do it.
Note that smaller models, such as minicpm-v4.6 run at 15-28 tk/sec on this hardware, which is perfectly usable for simple tasks.
But the V100 comes with more VRAM, also its memory bus is faster, and it draws less power.
Which GPU is the best option will depend on your application and goals.
meanwhile I “borrowed” a 5070 and while everything says pytorch & tensorflow all updated last year that is as far as updates have gone. Everything past that was just errors. I held up the white flag, back to the 3080. Now I am upgrading to a p40 and pascal… I don’t give a fig about any gaming ‘stuff’ (radio edit). I am a research scientist and I need compute.. just a smidge of vram for the monitor so I can what I am doing, though, maybe it would be more fun without it ;)
not even for mining ..
I have 4 x V100 each with 32GB of VRAM (got them when they were cheaper), Qwen35B-A3B UD-Q8_K_XL can run on two of them with full context at 80+t/s. Solar offsets the “high power consumption” which is open to interpretation 1 x 5080 at full load can chew through 280+W. Opencode and ClaudeCode are very performant at that token rate. One of the things that I do with this setup is run ClaudeCode pointed to it, it can run through what I need it to do and if that local model get stuck a quick switch to Sonnet or Opus to get over whatever hump was there and then back to local. I save a ton tokens that way and it allows me to run multiple projects maximizing the paid-for tokens. Before anyone asks, yes I hit the weekly token limits in Claude, again always maximize what you pay for.
Very scrappy of you! I love hearing stories like this!
True, but it can take over other duties like video output to free up space in a second gpu. It’s got fp16. It’s too old for updates or support though. If you’re broke these might come in handy. It would be great if the open source community could give them some software support so they could continue to be useful somehow. I was considering getting some for a node…
I’ve also had some really useful results with v100’s, written up here: https://notes.alelec.net/posts/datacentre-under-the-desk/
There was a fairly recent video by hardware haven who also used one of the V100 cards for his local AI inference work. That in itself is no different from what had been performed here, however, he reduced the power target of the card to 100W. Seemingly, this does not throttle the HBM clock speed, thus only taking some power out of the compute which is second to the inference speed.
I believe he got by with 100W and only “paid” around 5-10% of tok/s. I wanted to leave this for completeness. (https://www.youtube.com/watch?v=7DAPd5MGodY)
datacentre gpu like t4 or l4 come in pci variants. you just need a blowthru cooler, because they’re designed for it so they have no fans… That said, at < 100w (t4 is ~65w) you can probably do some sort of less obnoxiously loud cooling setup.
What I wouldn’t do to get a cheap used 24GB card. All I can run with my 12GB 3060 are 13B parameter Q4 models. I want smarter models
Wel it isn’t hard, just pay up. You can buy them :) or is that what i would not do so limited :)
Bound to be some when the AI craze pops, and it will pop, because things are heavily subsidized and when investors demand profit…..
Sell everything to a foreign nation or company that can actually afford leftovers.
This hardly news. Months old at best. The Chinese deliver al you need. plug and play basically.
industrial. GPU are the best bang for the buck , specially second market, I got 2 dozens with “exotic” diy closed loop , built like tanks
huh but if this is going to be possible, wouldn’t the market already be flooded with 5 year old computers being sold from datacenters? like google and amazon must each decomission some thousands of machines per day, no?
what i’m getting at is that these guys with their proprietary / custom solutions already seem to be in the habit of disposing of them destructively, rather than letting their work flow into secondary markets after they’re done with it?
i know the mining ecosystem produces a good amount of e-waste that makes it onto ebay but my impression is that the larger more ‘official’ corporations don’t seem to let this happen?
if i’m wrong, i’d love to know about it :)
They don’t decommission thousands of machines per day, their machines are designed to run as long as possible until they become functionally obsolete or fail. Obsolete and failed components of the machines are sold to likely 1 or 2 companies, then those companies likely sell them to smaller companies – some of which may refurbish and resell on eBay or elsewhere.
Additionally, these corporations know exactly where the technology is heading and already have a plan of what to use their GPUs for once GPUs are no longer necessary for AI inferencing – so there won’t be an influx of GPUs on the market, unless the idea of a GPU is obsolete in every form.
Where is the evidence that corporations have a plan of what to use GPUs for other than AI inference?
They are buying GPUs with a 6 year useful life, building data centers with loans that have to pay for 20 years, building compute capacity to like 15 times the market demand.
Analysts like Ed Zitron have looked hard to find what is the plan, how will they make those loan payments? No answers so far.
A lot of people’s pensions depend on this. So what is the evidence that they have a plan? (Concrete documentation, not just wishful thinking.)
When building my Team Hackaday Folding@Home boxes in 2020, I bought a couple of what were then 8 year old server boards with built in video (Supermicro X8DTi-F). This saved me from juggling which PCIe card was going to drive a display, instead just focusing on CUDA chores. No HDMI output from an SXM2 GPU? No worries.