When people start ranting about AI, you can be sure a few things are going to come up during the two-minutes hate: job loss, higher power bills, the neverending tide of low-effort slop, and wasting precious freshwater. Well, NVIDIA wants to take away that last one, beacause the all-water cooled Ruben architecture won’t need any evaporative cooling— coolant can stay in a closed loop, and never needs to be cooled below 45 C, or 113 F.
This sort of coolant loop should be familiar to anyone who has ever built a water-cooled PC or PlayStation: there’s a glycol-water mix, water blocks, and a radiator to reject heat to the environment. NVIDIA doesn’t mention if their new servers come with RGB lighting, but we’d like to imagine it’s an option. The big difference — aside from the rainbow LEDs– between a Ruben server and your old gaming rig is that in these racks, everything is on a waterblock. If there’s a chip on the motherboard generating heat, it’s getting rid of it into the same cooling water. Cooling water, that we have to emphasize, needs only be cooler than the chips themselves: in this case, they’re talking 45 C on the cold side, and 55 C headed out of the racks. (That’s 113 F to 131 F for all the bald eagles reading this.)
Given the required temperature drop is so modest, there’s no need for the evaporative chillers that have given AI data centers such a bad name in water conservation circles. Just like in a water-cooled PC, ambient-temperature air running over dry heat exchangers– also known as big honkin’ radiators–is able to handle the cooling, so no water is lost. Since everything is on waterblocks, there’s no need for cooling air, either, and the server farms need only be air conditioned to the degree required to make them comfortable to work in.
If you think NVIDIA is making this change because they suddenly care about water conservation, think again. The press release makes their motivations very clear: cooling costs money, and running this hot saves a lot of it. We’re talking four mil US a year for a 50 MW hyperscaler. One might suspect that this sort of thermal regime could limit the lifetime of the hard-working NPUs, but since they’ll be obsolete in a few years anyway, that’s not likely a big concern, especially not for NVIDIA.
We’ve actually seen hotter fluids used to cool computers before– coffee, for one. Water cooling also isn’t new in the data center world; we took a look at it a few years back. Things are clearly heating up now, though.
the amazing thing is that this wasn’t done from the start with LLM data centres ..
What is amazing is that they still reject waste heat in the air instead of providing to the community around (co-generation)
100%
They are a certain kind of people who don’t think about others often.
Who in Texas is going to want your hot air?
Low temperature waste heat is almost unusable.
No it’s not, you just need a decent high COP heat pump to boost it to usable. All well understood.
The heat pump often costs more (and requires more energy to make) than the alternative heat would have. This is especially true for low-delta heat sources. If you really want a lot of heat, just embed pipe in asphalt. In the summer, in Texas, half a mile of asphalt is enough to give you a fairly large quantity of live steam…
Most likely the data center is built on the cheapest land around, which means it won’t be next to any dense community of housing. That being the case, it would cost a whole lot to actually pipe the warm water to anyone needing it, and it would cool down significantly along the way.
Most common solution it that case is a bunch of greenhouses.
Common w thermal powerplants, but only in cold climates.
Actually I saw a news clip about data centers that are roughly the size of a sports stadium and just as tall and noise levels were also a concern along with rising electricity bills and cost of water. This link is to Virginia’s “Data Center Ally” Source: YouTube https://share.google/95YLkmkWMtfA2GtE0
Water consumption probably didn’t register as a concern until people started protesting it. There are a lot of industries that use similar or higher amounts of water.
It’s like they made the plans, mentioned water as a coolant, and just got to building and never looked back over it.
Quantity over quality
“Waste” in the sense that it’s not usable anymore? Or just cleaning it and in the sink?
I think the major complaint is that they’re taking water form an easily accessible location and passing it down to some less usable location, like taking water upstream a river and passing it downstream, so if someone else needs it they’d have to pump it back up the grade. There’s only so much water flowing down the river, so tapping some of it elsewhere can hurt local ecosystems or communities that depend on the water.
Or, the whole point about water is just an excuse to rally people to protest and sign petitions. It may be a strategic grievance.
As I understand it the current state of affairs is that if almond farming uses a gallon of water, data centers use a half cup or 32x less. By 2030 data centers are expected to use 1 cup or 16 times less than almonds. Everyone benefits from data centers but not everyone likes almonds.
Data centers need to use zero. Farming has to use water.
Also “everyone benefits from data centers” is LOLx1000.
Anyone that uses the internet benefits from a data center in some way
A major issue was when the consequences of fixed-price commercial-contract water from municipal sources showed up.
While not everyone is playing in good faith, a lot of this is just “assuming” at scale: datacenter assumes that “as much water as you need” is literally open-ended, and muni assumes that there’s no way anyone could need fifty times the previous yearly consumption by themselves.
Add a bit of long-term legal obligation, and you have all the ingredience for a nice pitchfork-and-torch parade.
And in most (US) jurisdictions, the ratepayers are on the hook to pay for the necessary new power (and sometimes water) infrastructure over the next 20 years even if the commercial tenant goes bankrupt and leaves the facility abandoned the next year. A few jurisdictions are different, but for the rest, this problem alone tends to generate significant anger once people realize what’s happening to their utility bills.
Most industrial air conditioning evaporates water off of the outdoor coils. It saves electricity but costs water.
A lot of things were designed in an era where water was considered to be cheap and plentiful.
Generally, water remains cheap and plentiful east of the Mississippi river, in the USA.
It’s so GD plentiful in northern California we let 40,000 cfs flow to sea every wet season, without any effort to collect it…That’s about 3 cubic km per month.
LA can survive on the typical yearly flow of the LA river, but they just let it go and steal water from others, because politics.
Wet season water is INXS, in GD California…
Half wits love to conflate wet and dry season water, also conflate the sac river valley w death valley, then bitch about almond trees.
‘Move to where the food is.’ / Kinison
But not here, that other country up the road has better government tits.
They forgot to ask the AI how a closed liquid cooling system works until now
And for the optimists among us looking for an answer to the increase in power/electricity, remember: one of the most direct measures of the development of a civilization is the portion of the energy output of a star they are able to harness on it’s way into the black. Yes, we’ll need more power to do more things, but that’s a good thing!
Now if only they were required to pay for enough rollout of green power generation to offset their load.
I suspect they’re going to make the pro-nuclear lobby very happy.
They deserve a day in the sun too
The question becomes : do you need to do more things or just enough things properly ? Also thats cool to harness more power from the stars, but honestly what’s the point if everyone is starving and earth is becoming a nightmare to live on ? I find this metric missleading because it conveniently ignores everything else than raw growth, which I believe has proven to not automatically lead to prosperity these last few years.
Kardashev scale is a highly speculative and mostly naive measure, which only applies in niche cases where the priority is on output over efficiency.
There probably should be tiers within the scale for efficiency measures. A species uses the entire output of a star is impressive but you’d still think they are lame if it’s super inefficient. (Kinda like some of the species in Star Trek that have warped but are sketchy as heck)
This being HAD, you’d think people here would have the basic understanding that there are more ways to generate power than just the sun.
This is what’s wrong with the world. Small minds that think small….
sun and nuclear. i think that’s all
And nuclear is still from /A/ sun.
ZPM around here somewhere.
And the AI is the BLM.
And the AI is the BLU, my oops
Sun or processing matter yourself. That’s really all. Maybe ZPM in the future, we still barely even aknowledge zero-point in science.
What is “hotub coolant?”
ethelene glycol. its also the stuff they use to de-ice airplanes, its used in antifreeze, pretty common coolant.
Is “hotub” supposed to be a brand name or something? Searches only bring up misspellings of “hot tub”, and I’ve never heard of ethylene glycol being used in hot tubs as a coolant, only as a toxic antifreeze.
yea idk why they used hot tubs, i suppose if they have a heat pump you need a working fluid. but more people own cars than hot tubs.
Had a hot tub; it didn’t use a coolant. Just heated water, had an air pump.
Glycol based fluids are sometimes use to winterize the unit for shutdown.
I thought I was the hot tub coolant!
Hm. It’s was supposed to reference the temperature of the fluid, like hottub-warm or something, but evidently we messed that up.
still want to see supercritical co2 used in a computer cooling application.
Why? that’s probably the worst thing to use for a small cooling appliance.
i dont want to see it because its good, i want to see it because its hard.
Is that one of “the other things”, beside going to the Moon ? I always wondered what they were.
No love for liquid salt?
I don’t know why they don’t run even hotter, honestly – more of that sweet, sweet ∆T – Then you could have more of these beasts in deserts on solar and wind farms.
“coolant can stay in a closed loop, and never needs to be cooled below 45 C, or 113 F.”
This temperature might still be hard to achieve in many sunny places.
There is, albeit significantly less. Wherever there is electric current, there is heat. Every single wire generates heat and given the amount of wires in a data center, I’d guess even Ethernet cables give 100 W which needs to be removed.
This is just window dressing.
They have to get rid of megawatts of heat. They have to get rid of it to the environment. The environment of server farms will inevitably heat up significantly. Whether it’s the air or a river or what else. It might be that this system saves the owner of the server farm a few bucks but to the environment it will be no change. Heat does not evaporate into nothing.
Yes, the servers themselves still generate the same amount of waste heat, and the server farm is still dumping it into the surrounding environment, but this a lot more than window dressing.
You eliminate almost all the need for heat pump chillers and/or evaporative cooling, the first of which consumes a ton of additional electricity, and the second of which wastes an absurd amount of water.
Well my state is producing 5000 megawatts of nuclear power anyway so may as well use it
Sometimes I wonder if we could produce chips that happily ran at well over 100C so we could recapture some energy from the steam we could produce.
Why aren’t they using the sky-cooling paint that NightHawkInLight came up with? There are even commercial versions available as a roll of film (no affiliation): https://coldrays.com/
Isn’t about 80% of the planet’s surface, covered with water? And, doesn’t freshwater rain down from the sky daily? Least in Florida… Seems ridiculous that we still have people whining about water shortage, drought. Where there is a need, there is an opportunity. Someone would have cashed in long time ago. We used to have rainmakers, the indian rain dance… Didn’t work, but people paid. Those same desperate people still exist. Water is abundant. Just a matter of getting it to where it’s need. California is a sad joke. Lot of ocean to desalinate. Humidity can be extracted from the air. They have both, a dry season, called wildfire season. And, they have a wet season, causing flooding, mudslides. Excess water channeled into the ocean, instead of stored in retaining ponds, reservoirs. We have massive pipelines for oil and gas, but not freshwater, which has been a ‘crisis’ for such a long time. And water cover most of the planet surface…
We actually do need the data centers. Building them isn’t new. The high demand, as most everything has gone digital these days, just increased. AI is only part of it. Everyone wanted cell phones, just not a tower in their neighborhood. But, more towers were needed, so there was dead zones, and weak signals. People accepted the towers, for more reliable service. Data centers are no different. We all mostly use smart phones, which access the internet. We pay digitally. Lot of home security video stored in the ‘cloud’, to be view anywhere, anytime, any device. By anyone with a password. We use our phones for GPS navigation in our cars. Instead of a dedicated unit, with maps stored internally. The smart phone maps are the most current, and include traffic, closures, detours, accidents in real-time. We all pretty much use commercial data processing daily. Lots of peoples jobs depend on it. How many people using rideshare apps, or food delivery apps? I never have, but know quite a few who do weekly, even some pretty much daily. It’s a digital era, and data processing needs to expand to supply what consumers demand. People demand seamless steaming of music and video. I remember the delays when watching online videos. The pause, while a cache caught up.
Mostly corrosive salt water. Full of living things that love to stick to warm surfaces.
“radiator to reject heat” should be “radiator to eject heat” if you insist on using the incorrect “eject”. Check your thermo: heat isn’t “ejected”.
Not sure what else a radiator would do other than radiate. When I read it I heard “Radiator to reject heat, with Lemon!”
Generally radiators dont radiate much, they mostly convect.
Except in space. One of the problem with high power systems in space is that you can only radiate (or transfer heat into a medium that you then discard, only useful for very short lifecycle designs).
That one fluid spray system was actually a trick question, though. The intent was to recapture the spray, making it not-very-lossy, and just taking advantage of the massive increase in radiative area while it was temporarily being sprayed between the sprayer and catcher. Difficult and very niche, but an interesting out of the pipes, er box, solution :)
Its said every 10C halves the operational life of electronics by half. So 45C really, isn’t that bad at all, I personally don’t mind if my desktop PC is at 45C. Anything below 60C is quite okay. The GPU however may get up to 80C, but most of the time its 65C during heavy use.
If I don’t like my PC temps, I just take off the side panel, and place a kitchen exhaust fan that I have laying around blowing on the mobo+GPU. Instant 10C+ drop. Airflow matters A LOT. I would’ve tried liquid cooling if I wasn’t so worried about the water leaking and killing my machine. Sadly water is pretty much the best liquid cooling fluid on this planet
I am not ranting about AI (well, I kind of do), but I am asking very, VERY simple question – what’s the win? Not for the investors, for me, average Sam.
Insofar I don’t really see many – I do not see anything getting cheaper/better/simpler, on a contrary, it is getting more expensive overall, quality is okay to my liking, tolerable, but simpler? When AI will it start delivering simpler things? Not in 25 years from now, like nuclear power, tomorrow or next month? AI is speedy, so by now we should already be seeing things unimaginable just a decade ago, right? RIGHT?
As far as engineering challenges go, I am really so NOT inspired to discuss these – similarly how I am not fond of discussing local politicians’ dumping all the grunt work on us, tax-paying citizens, pretending we are “involved in the discussion”. Most of the times we are skipped, private deals rule, and we are given trifle non-critical things to munch over and discuss as if these make a difference. Say, property tax, aha, sure, yeah, it only goes up, no matter how many million-dollar houses are sold in the area, it is still never enough to cover municipal level needs, taxes NEVER go down. AI, invent me lower-taxes neighborhood where I can move to and forget about that part of my life. No? Why not?
Back to the topic at hand. Lower cooling requirements, good, AI invent, make and sell me more efficient central cooling that will cost less, require cheaper maintenance, lasts LONGER and doesn’t need expensive machinery to install. Because that would be the direct result of something better/cheaper/smarter, better technology spilling over to the consumer-grade infrastructure. Not 2 years from now when patents expire, TODAY.
No one will ever do or build anything because you will benefit.
People / organizations only create things to take your money. They will create the minimum quality of product / service in order for people to open their wallets.
RE: No one will ever do or build anything because you will benefit.
I am perfectly fine with companies making better products sold for lower prices making sustainable profit. That’s the classical (ie naive) definition of idealistic/proper liberal/competitive capitalism and it works well when it does – thus directly benefiting average Sam IN ADDITION TO the usual profit-strip-mining entities.
If for-profit entities are in no hurry to trickle-down anything of value to my bottom of the food chain, then WHY do I need them around? I do NOT want to give my hard-earned little money to them so that they can exist – I’d very much give these moneys to my kids instead, or buy myself a new pair of shoes. Non-profit entities can (and should) fill the void, because it is clear demand that should be driving the supply, idealistic invisible hand waving the market, backed up by quite a LOT of keynesian economy infusions (tax-funded Farmer Bill, Infrastructure bill, Social Security payments – they don’t go to some kind of publicly-owned entities, they go to private providers, again PRIVATE PROVIDERS, etc etc).
My naive solipsism is simpler than that – it comes from the ideal economics 101, the one that trickles down the better-made cheaper products ALL THE WAY to the bottom. Presently it delivers some things okay (cars), not terribly well other ones (residential real estate and healthcare) and stop products I could use somewhere before they arrive at my level ($15K car – I’ve already rumbled about that – or $25/barrel crude oil – absolute minimum at which petrol companies STILL make good/sustainable profit).
My needs are rather humble – If I never wanted MCMansion, nor I needed garage full of Koenigseggs or bunch of luxury yachts parked at private islands, then why is it so darn hard to find what I need for the price I can afford – that was my point. AI doesn’t seem to know the answer either, and it has been at it (bubbling up) since mid-2000s – WHERE ARE the results at my bottom of the food chain?
Better question:
Why do they need you around?
Make yourself useful and you won’t be at the bottom of the food chain for long.
If your expecting the bottom 10% to revolt, I’ve got bad news for you.
They’re incompetent, incapable or both.
Rather, the OTHER way around – I don’t exactly need for-profit entities, they NEED me and my moneys to exist. They also need to keep extracting value from the product/products I’ve been making for the last 35+ years working full-time job (sometimes two jobs). This is very simple idea, no need to make it overly complicated.
Not sure where “the bottom 10%” came from, must be corporate propaganda, The Other 90% is the real statistics (in the US at least, which is already in the top 10% of the world, btw), and the number of New Poor who keep joining the Old Poor kept growing since 1990s non-stop. New Poor who need to finance/loan/borrow to buy pretty much everything they need – even when they don’t own anything other than a car or two (rent). Same generation who is derided for “living in their parents’ basements” because no job can pay for a decent house without the horrible 30-year mortgage (or 50-year ones that are arriving already in the US of A – after copying this idiocy from Germany and Switzerland).
Whether the New Poor are more (politically) active then the Old Poor is not for me to figure out, I am just stating the obvious reality I see every day around me. I also see a LOT of people freshly thrown out of their jobs under various lame excuses.
Enjoyed mildly entertaining diversion, and pre-answering the “if not the Big Bad For-Profit Entities then WHO?” – “anyone else, once there is a void it will be filled with smaller/agile entities literally overnight, offering cheaper/better product with smaller overhead.” This is not new either, “Back to School” calls them “quick and dead”. Good movie, btw, some basic econ 101 as well, oh, bribes, too, are integral part of the economy, the larger the company … and I’ll stop for now.
Sammie Gee – your response proved HaHa’s point…
The upside is that hopefully a country that despises you mind, body, and soul doesn’t build AM from I Have No Mouth, etc. and lock your consciousness away in it forever to get revenge for the century of humiliation
mmm…this is probably rudimentary but the reason data centers are generally cool is for the PEOPLE who work there…a split system of different temperatures are doable for many more pennies…and the Beato
of utube fame did point out the future demise of data centers that are ai specific…between the pc style hardware makers this situation will surely change in fairly short order (they love retail mark-up)…what to do with the empty buildings oh my…
nvidia finally caught up to what is already standard in academic HPC clusters. Academia did already care about their cooling cost since universities usually get a fixed budget for compute, including the operating costs. So you get to buy more computers if you have the lowest operating costs possible. Althogh 45°C is hotter than the 40°C that is standard now, which is good.
What makes me laugh is that my whole section of my industry hire HVAC for contingency and reactive plans revolves around water cooling and heating, chillers and boilers, where we use (temporary) closed systems to cool server racks machinery ect ect with chillers direct to system or with AHU’s, Always wondered why we’ve not implemented direct system cooling using Mono propylene glycol or even Ethlene Glycoas we do in our systems like liquid cooled PC’s Or siblings
cooling on an air-cooled server breaks – the server dies/shuts down. Leak in a server at the top of a rack…..
Actually, on most of the systems, the rack loop is not water, often not electrically conductive at all (systems are sometimes submerged, especially for special-use one-off architectures). This loop dumps heat via liquid-liquid heat exchanger into the standard chilled-water standpipes that any large commercial building designs in anyway. The difference is that datacenters just massively scale up the cooling, since racks of servers generate a lot more heat per volume than cubes full of filthy meatsacks.
That chilled water can be made in many ways, but at normal commercial scale, evaporative (open loop) chillers are pretty economical and usually pretty sustainable too. At datacenter scale, that’s far from true, but sloppy shortcuts means just jumbo-sizing the previous solution and throwing money at it. Which, in places with significant available water, is not even a problem. But, such places either generally have little access to power, or the demand for the land for other purposes drives the prices up too much.
It’s not that datacenters automatically have water problems. After all, the water isn’t lost, it’s just displaced. Places with limited groundwater, or with limited infrastructure to make available water clean enough to not corrode/foul the chillers are a different story, however.
You can substitute evaporating water with power; if power is cheap, use the power (price is often a proxy for efficiency in the local environment). If water is cheap, use the water. If neither is cheap, why are you building a datacenter there in the first place?!
The real problem is when someone throws plenty of money and applies a template solution at a problem without even checking to see if it fits. There’s no such thing as one size fits all, in almost any engineering domain. And that’s more common when someone perceives an early mover advantage rather than a long-term gain: get it done in a hurry and get out with a lot of quick cash. Let the buyer worry about whether it’s actually a good idea.
“job loss, higher power bills, the never-ending tide of low-effort slop” – stays
“and wasting precious freshwater” – goes if they use NVidia and not one of the other foundries
But what about all the other harms the Tensor driven slimemold is causing?
The shortage of RAM
The enshittification of operating systems
The CVE danger advanced models bring
Surveillance and loss of privacy
The number of low quality websites
The Website AI blockers that stop fingerprint secure browsers accessing sites
AI Weaponry
Likely many more I forget, but I will stay a LuddAite thanks.
Heh. The CVE danger is a good thing in a way. That’s just the engineering coming due that was racked up by “move fast and spend time on features not quality”.
When someone complains that the features are being delayed by all of the AI-induced bugfixes, they’re actually admitting that they’d rather be writing cool bugs than taking their time to write solid code.
After all, if an AI could find the vulnerabilities, so could a person.
Water doesn’t disappear…. it comes down as rain, thus less heatwave…
The main problem is in regions where the aquifers are heavily over-consumed, if you draw more water from them, they either run dry, or the rate of water filtering into them goes up. If it goes up too fast, they tend to fill with still-salty water, and transfer through the aquifer faster than it can be filtered. Pretty soon, the groundwater is brackish and kills crops. Regions with these problems generally have to rely on groundwater for any agriculture, so excess consumption can destroy agriculture in a region for decades (or more), and can result in significant salt contamination in the soil, making agriculture much harder in the future.
These tend to be regional issues; places without these problems don’t have a strong reason to avoid open-loop evaporative cooling.
A large number of aquifers in Texas, Florida, Louisiana, California, New Jersey, just as a sampling, demonstrate this in practice.
Two that specifically affect a lot of data center plans: The Texas Gulf Coast aquifer and the Ogalla aquifer, are good contrasts: The Ogalla is just seeing rapid depletion (wells running dry). The Gulf Coast aquifer depletion is causing lateral saltwater migration; the water level isn’t as impacted, but the water that’s available becomes increasingly unusable.
Literally just did this in my office/shed – 6 computers in my nomad cluster, one of which has 2 v100’s in it. 3/4″ PVC to outside, with an F-150 radiator from the salvage yard + box fan. in winter I’ll add a secondary radiator + switch so I can get the heat dumped inside
Anyone else annoyed when people intend to write temperatures in Celsius (or Fahrenheit) but don’t use a degree sign?
Here have some, these are free: °°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°