To the surprise of almost nobody, the unprecedented build-out of datacenters and the equipping of them with servers for so-called ‘AI’ has led to a massive shortage of certain components. With random access memory (RAM) being so far the most heavily affected and with storage in the form of HDDs and SSDs not far behind, this has led many to ask the question of how we will survive the coming months, years, decades, or however-long the current AI bubble will last.
One thing is already certain, and that is that we will have to make our current computer systems last longer, and forego simply tossing in more sticks of RAM in favor of doing more with less. This is easy to imagine for those of us who remember running a full-blown Windows desktop system on a sub-GHz x86 system with less than a GB of RAM, but might require some adjustment for everyone else.
In short, what can us software developers do differently to make a hundred MB of RAM stretch further, and make a GB of storage space look positively spacious again?
Just What Happened?
At the risk of coming across as an ‘in my day’ rant, around the year 2000 I was using an AMD Duron 600 system with probably about 256 MB of SDRAM that succeeded my previous Intel Celeron 400 system with an amazing 64 MB of SDRAM. With Windows 2000 (arguably still the best version of Windows) on its own roomy 1 GB HDD partition there was still plenty of room on the rest of the HDD for applications, documents and some multimedia content like music and videos.
On these systems I could run a browser with many tabs open, alongside an office suite, an IDE, chat applications like IRC and ICQ, an email client, filesharing applications, and much more, without the system breaking a sweat. In the Duron 600 system I would eventually install a Matrox G550 AGP videocard to do some dual-monitor action, like watching videos or consulting documentation while browsing or programming at the same time.
Fast-forward a few decades and you cannot even install Windows on a 1 GB partition, and it requires more RAM than that. A quick check on the Windows 10 system that I’m typing this on shows that currently the Windows folder is nearly 27 GB in size and just the Thunderbird email client is gobbling up over 150 MB of RAM by itself. Compare this to the minimum Windows 2000 system requirements of a Pentium 133 MHz, 32 MB of RAM and 1 GB of free HDD space.
This raises the question of what the reason is for this increase, when that email client in the early 2000s had effectively the same features in a much smaller package, and Windows 2000 is effectively the same as Windows 7, 10 and now 11, at its core when it comes to its feature set.
The same is true for ‘fast and light’ options like Linux, which I had once running on a 486DX2-66 system, a system on which the average Linux distribution today won’t even launch the installer, unless you go for a minimalistic distro like Alpine Linux, which requires a mere 128 MB of RAM. Where does all this demand for extra RAM and disk storage come from? Is it just all lazy waste and bloat that merely fills up the available space like a noxious gas?
Asking The Right Questions
Storage and RAM requirements for software are linked in the sense that much of an application’s code and related resources are loaded into RAM at some point, but there is also the part of RAM that gets filled with data that the application generates while running. This gives us a lens to find out where the higher requirements come from.
In the case of Windows, the increase in minimum storage space requirements from 1 GB to 32 GB for Windows 10 can be explained by something that happened when Windows Vista rolled around along with changes to WinSxS, which is Windows’ implementation of side-by-side assembly.
By putting all core OS files in a single WinSxS folder and hard-linking them to various locations in the file system, all files are kept in a single location, with their own manifest and previous versions kept around for easy rollback. In Windows 2000, WinSxS was not yet used for the whole OS like this, mostly just to prevent ‘DLL Hell’ file duplication issues, but Vista and onwards leaned much more heavily into this approach as they literally dumped every single OS file into this folder.
While that by itself isn’t such an issue, keeping copies of older file versions ensured that with each Windows Update cycle the WinSxS folder grew a little bit more. This was confirmed in a 2008 TechNet blog post, and though really old files are supposed to be culled now, it clearly has ensured that a modern Windows installation grows to far beyond that of pre-Vista OSes.
Thus we have some idea of why disk storage size requirements are increasing, leading us to the next thing, which is the noticeable increase in binary size. This can be put down for a large part on increased levels of abstractions, both in system programming languages, as well as scripting languages and frameworks.
Losing Sight Of The Hardware
Over the past decades we have seen a major shift away from programming languages and language features that work directly with the hardware to ones that increasingly abstract away the hardware. This shift was obvious in the 90s already, with for example Visual Basic continuing the legacy of BASIC with a similar mild level of abstraction before Java arrived on the scene with its own virtual hardware platform that insisted that hardware was just an illusion that software developers ought to not bother with.
Subsequently we saw .NET, JavaScript, Python, and kin surge to the foreground, offering ‘easier programming’ and ‘more portable code’, yet at the same time increasing complexity, abstraction levels, as well as file sizes and memory usage. Most importantly, these languages abandoned the concept of programming the underlying hardware with as few levels of indirection as possible. This is something which has even become part of languages like C and C++, with my own loathing for this complexity and abstraction in C++ being very palpable.
In the case of a language like Python, it’s known to be exceedingly slow due to its architecture, which results in the atrocious CPython runtime as well as better, but far more complex alternatives. This is a software architecture that effectively ignores the hardware’s architecture, which thus results in bringing in a lot of unnecessary complexity. Languages such as JavaScript also make this mistake, with a heavy runtime that requires features such as type-checking and garbage collection that add complexity, while needing more code to enable features like Just-In-Time compilation to keep things still somewhat zippy.
With Java we even saw special JVM processor extensions being added to ARM processor with Jazelle direct bytecode execution (DBX) to make mobile games on cellphones programmed in J2ME not run at less than 1 FPS. Clearly if the software refuses to work with the hardware, the hardware has to adapt to the software.
By the time that you’re a few levels of abstraction, various ‘convenient frameworks’ and multiple layers of indirection down the proverbial rabbit hole, suddenly your application’s codebase has ballooned by a few 100k LoC, the final binary comes in at 100+ MB and dial-up users just whimper as they see the size of the installer. But at least now we know why modern-day Thunderbird uses more RAM than what an average PC would have had installed around 1999.
Not All Hope Is Lost
There’s no need to return to the days of chiseling raw assembly into stone tables like in the days when the 6502 and Z80 still reigned supreme. All we need to do to make the most of the RAM and storage we have, is to ask ourselves at each point whether there isn’t a more direct and less complex way. What this looks like will depend on the application, but the approach that I like to use with my own projects is that of the chronically lazy developer who doesn’t like writing more code than absolutely necessary, hates complexity because it takes effort and whose eyes glaze over at overly verbose documentation.
One could argue that there’s considerable overlap between KISS and laziness, in the sense that a handful of source files accompanied by a brief Makefile is simultaneously less work and less complex than a MB+ codebase that exceeds the capabilities of a single developer with a basic editor like Notepad++ or Vim. This incidentally is why I do not use IDEs but prefer to only rely on outrageously advanced features such as syntax highlighting and auto-indent. Using my brain for human-powered Intellisense makes for a good mental exercise.
I also avoid complex file formats like XML and their beefy parsers, preferring to instead use the INI format that’s both much easier to edit and parse. For embedding scripting languages I use the strongly-typed AngelScript, which is effectively scriptable C++ and doesn’t try any cute alternative architectures like Python or Lua do.
Rather than using bulky, overly bloated C++ frameworks like Boost, I use the much smaller and less complex Poco libraries, or my NPoco fork that targets FreeRTOS and similar embedded platforms. With my remote procedure call (RPC) framework NymphRPC I opted for a low-level, zero copy approach that tries to stick as closely to the CPU and memory system’s capabilities as feasible to do the work with the fewest resources possible.
While I’m not trying to claim that my approach is the One True Approach™, for me half the fun of programming is to do the required task in a very efficient and low-resource manner, which is why I ported for example FFmpeg to the ESP32 so that I could run the same project code on this MCU, rather than deal with the complexity and documentation Hell of Espressif’s ESP-ADF framework.
Sure, I could probably have done something with MicroPython or so, but at the cost of a lot more storage and with far less performance. Which gets us back again to why modern day PCs need so much RAM and storage. It’s not a bug, but a feature of the system many of us opted for, or were told was the Modern Way™.
the coercion of Windows users to switch to Windows 11 (and thus, for most people, brand new hardware, since they either don’t know better or are using 15 year old equipment) couldn’t come at a more obnoxious time
I am still hoping to switch over entirely to Linux soon. There are just a few major applications holding me back. Plus laziness, of course.
And WSL was finally starting to become a decent Linux platform, too. Sigh.
I understand that Microsoft really wants the new security instructions which are the main thing older machines cannot provide. But since 11 demonstrably can run without them, making old hardware unsupported rather than deprecated does still seem gratuitously arrogant.
lack of time is my problem. linux can be a time hog.
Win11 is an absolute time killer. Want a file’s properties? You need to right click and then open the menu, to click a show more button to open another menu to click the Properties option. Win11 makes computing atleast 2x slower lol.
That is indeed an annoyance I do not understand why they put that in. It makes me wonder if the programmeurs who made that silly decision use Windows themselves. It feels so unnessescary.
Learning Linux can be slow. Using it, cI disagree.
Speaking as one who worked on some of the XML Recommendations, the best use of XML is as an interchange and tooling format, sometimes as a development and debuggability/human-facing format, and of course as a document markup format (it did, after all, inherit from SGML), rather than as a primary representation. Ditto JSON. Simply because text is an inefficient way to store non-textual data.
In fact, the best answer may be to map the XML Fata Model over non-xml data as an abstraction layer. One of the improvements in IBM’s XTQHP XQuery implementation that didn’t get contributed to Apache (though the idea was mentioned on Apache’s mailing lists) was my XML Cursor Interface, similar to the cursor abstraction in later versions of the DOM but with navigation operations designed for XPath. I wasn’t given time to pursue it, but adopting this layer would have permitted plugging in proxy implementations of this interface to run XQuery fairly directly over any tree – structured data format (or any that a tree-based mapping could be set up for). Since XQuery introduced non-text data types, this could even have avoided most of the cost of rendering values into text.
I wish I could find cycles to make that change to Apache Xalan. DTM was a specific solution to a specific problem (minimize code and document size on machines and networks where 32 Meg was still considered big), but was intended primarily as implementation rather than interface, and is painful to map over other data structures… including DOM. XCI could also serve as a “lingua franca” common interface between XML and JSON. One can even argue that XCI should be given first-class interface alongside DOM.
Unfortunately, when I can get organized again, there are higher priority things to do to Apache Xalan. (I haven’t even found time to run a release.) But this is one of the things I would still like to see happen.
I don’t think just because something at some point experiences a quick collapse it should be labeled a bubble. One of the criteria of a bubble is that it (relatively quickly) grew to a point of collapse but had no substance (important bit of the definition), I’m not sure we can definitely say that about this so-called AI. I mean it has uses and some of those won’t go away, it’s just over-hyped to a ridiculous level.
Maybe we should call it ‘foam’, bubbles on top of stuff.
Or maybe we can call it ‘current capitalism’, because it’s just everybody trying to ride a gravy train.
AI isn’t a bubble in the sense that it doesn’t have any substance behind it, it absolutely does. In fact it does! It feeds so, so many people. Chip fabs, software people, water purification plant employees, everyone related to the power grid etc.
It’s useful and very cool, but the problem is, it’s not as useful and revolutionary as the marketing claims… A lot of it is speculative. “If we needed 50k GPUs this year, next we’ll need 100k, and 200k the year after that…”, yet they’re buying VP money and earning none of their own
Someone once told me, LLMs and AI rush of the current times is more impressive from a finance perspective than a tech perspective. You would be hard pressed to find another “product” that produces so little value yet is so valuable and has high evaluation based on future growth or groundbreaking results….
Not an alarmist, but the crash is going to be ugly.
None of those things are stuff AI actually does, you could instead have a Hitchhiker’s Guide supercomputer with the same amount of silicon and it would do exatthe same things.
It’s the same as any of the other overcompetitive useless fields like marketing or social media, all we’re doing is shoveling sharp minds and mountains of capital into systems that have minimal positive (and debatable net negative) benefits for society. We could (yes this is an oversimplification) instead choose, as a society, to redirect this to actually useful things, like maintaining infrastructure, reinforcing education or healthcare, doing more fundamental or applied research, or just deciding to let go of some productivity and have 4 day workweeks without any other changes to quality of life.
AI is just the latest shiny thing to distract us from the things that actually matter to us and our communities as humans. I won’t argue that there are zero benefits to it, but even working in a field with “bleeding edge AI” where the C suite is very strongly insisting that AI is changing every way we do work, it has limited useful applications, and definitely not on the scale that the tech sector is building out for.
A-effing-men
I don’t need to survive no apocalypse. In 2019 I built myself a PC with 64 GB of DDR4 RAM so I’m indifferent to what goes on now. Windows works, games work, VirtualBox works. If you were too stingy to upgrade, too bad.
I mean same, but that last sentence is really gross my dude
Note that one of the trade-offs that has been deliberately made by the computer industry as machines have gotten larger and more powerful is to accept code inefficiency in exchange for lower development cost. “Software expands to consume the resources available to it.” It was less expensive for both company and customer to throw hardware at the problems rather than man hours. Optimizers can only do so much; getting really tight code involves selection of algorithms and architectures.
Microcontrollers pushed us back in the direction of more finally tuned code… But they too have gotten big enough and fast enough that it is often less expensive to go with quick but sloppy code. I built a set of networked interface cards around Pi Zero W cards simply because that was the easiest way to prototype them, despite the fact that running Linux just to handle device communications can’t be called efficient by any means. Even Arduino would have been overkill, but would have had a learning curve and delayed implementation, and while I need to spend that time at some point this was not the right time to make that investment.
It’s all trade-offs, picking where you want to spend your resources.
Quality! Service! Price! (Pick any two.)
I think we needed this reset
Software engineering is so bloated, it’s unbelievable. And make no mistake, the majority of that bloat is abstraction. Programmers of the past had to write 100% of a programs’ code themselves, this let them write MVP code which was by nature, light. I wouldn’t be surprised if it’s 5% now, the rest being libraries, auto generated code, boilerplate etc.
SURE you may only need 1% of the features the library provided but you need to pack it in its entirety. Even worse there are apps which ship entire browsers to run their code! Wasteful! But nothing in comparison to applications which ship entire ubuntu images that you must run in a VM to use the software…. Ludicrous
COVID was a wake-up call for hardware guys. Using jellybean parts is always better than that fancy ST or TI or AD part. This one is for software guys, especial high level software, webdev etc
That part made me smile lots, thanks for the article!
As far as I am concerned, I can totally live with a system with a single 4 GiB RAM stick IF it does not have to run web browsers or Electron apps. Those are the main offenders; micro-optimizations, like changing an XML parser to an INI parser, are unlikely to make significant changes, when a single browser tab is eating away all the resources that the machine can offer.
(Okay, I definitely screwed up a terminology here – I meant “small optimizations”, not “micro-optimizations”)
100% agreed! Although, unless you’re talking about a server (or a gaming system), how useful is a system that doesn’t run a web browser these days?
Speaking of servers, I still run a few Linux servers (VMs) with as little as 1 GB of RAM, which is still enough to run several services (even through Docker) without breaking a sweat. I would say even down to 512 MB is doable for simple things, although 1 GB + is definitely better.
Add in a GUI, and all of a sudden 2 GB seems to be about the minimum, with 4 GB + being noticeable better.
Add in a web browser, and now even 16 GB feels tight, with 4 GB being the absolute minimum and can only handle a couple of modern tabs/pages.
In addition to layers of abstractions, I think part of the exploding RAM problem with web browsers is the security model. In the early days, all web pages ran in a single process, sharing memory space. This used less RAM, but caused potential stability and security problems. So, they introduced sandboxing and split each tab/page out into its own process, which improved stability and security but caused a RAM usage explosion.
Additionally, back in the day, web browsers were simple front-end display engines and most of the processing happened on the back-end. A web page was mostly independent of previously viewed pages (from the server) and just rendered what the server sent it. That all changed a decade or so ago. Now web browsers have to run single page apps, where the server may do very little other than return static assets and respond to data requests, and the browser does all the ‘processing’ and has to maintain state between virtual ‘pages’.
This is made worse by all the different high abstraction level web frameworks, like React that are used on modern web apps. Every different web page / tab may be using a different web framework and/or version. So, each one brings its own bloat along with it. This is different than typical GUI apps on a desktop where likely there are fewer frameworks being used, and usually far fewer active versions of said frameworks in play.
I don’t know the solution, but as the article ponders… I have also long wondered why we now require 32 GB + systems but don’t seem to be all that much more productive than a few decades ago where we got along just fine with 100’s of MB of RAM or even a GB or two. In addition to the switchover to web 2.0, we also have much higher resolution displays, and faster internet connections. I’ve often wondered how much those advancements have contributed to the bloat (in addition to higher level languages and frameworks).
Oh gawd. I still must have a floppy of RamDoubler somewhere…
Back when a gig of RAM was a lot (a simpler time) a friend and I tried to make a “RAM drive” adapter that would allow multiple stacks of 128 or 256 k sticks plug into a single RAM slot on the mother board.
Needless to say, it did not work. This also about the time I gave up on becoming a computer hardware engineer. Juice was not worth the squeeze.