When you first learn to program, memory seems pretty straightforward. You have a bunch of bytes, each with an address. If you want to read or write something, you specify the address, and the computer goes to that place and does your bidding. Sure, some addresses might be reserved for hardware or the operating system, but the general idea is simple: memory is a big array of bytes.
Just like we pretend that a wire is a perfect conductor, the big array of memory model used to be a slight twisting of the truth. On a modern computer, however, it is almost entirely a lie. The address you use probably isn’t a physical address. The data you read might not come from RAM at all. Even if it does, it might be sitting in one of several caches. And if you follow the rabbit hole far enough, you discover that your SSD is arguably just another level in the memory hierarchy.
The Need For Speed
The fundamental problem is that processors have gotten much faster than memory. A modern processor can execute several instructions per clock cycle, but accessing ordinary DRAM can take hundreds of cycles. If the processor had to wait for RAM every time it needed data, much of that impressive computing power would go to waste.
The solution is caching. Keep copies of frequently used data in smaller, faster memories closer to the processor. Most modern desktop processors have at least three levels of cache. The L1 cache is tiny but extremely fast, and often split between instructions and data. L2 is larger and somewhat slower, while L3 is even larger and usually slower as well.
Things get interesting with multiple cores, as is often the case these days. Typically, each core has its own L1 and L2 caches, while several cores share an L3 cache. But that’s not universal. Some processors share L2 caches between cores. Others have multiple L3 caches, each serving a group of cores. The exact arrangement depends on the processor architecture.
One important detail: caches don’t generally work byte by byte. They transfer and track blocks called cache lines, commonly 64 bytes on modern desktop processors. If you read one byte that isn’t cached, the processor generally fetches the entire line containing that byte.
This is why sequential memory access is often much faster than random access. If you read the next byte, there’s a good chance it’s already in the cache. Hardware prefetchers may even anticipate what you’ll need next.
But there can be problems. Imagine two cores modifying completely different variables. If those variables happen to occupy the same cache line, the cores may repeatedly invalidate each other’s copies, wasting time even though they aren’t logically sharing data.
Protocols such as MESI keep the caches coherent, coordinating which core owns a line and which copies are valid. MESI is an acronym for cache line states: modified, exclusive, shared, or invalid. Memory systems must figure out how to keep caches in sync. But coherence doesn’t eliminate the need for synchronization or memory-ordering rules in multithreaded programs. The point is that even our simple byte-addressed memory is secretly operating in larger chunks.
The Address Isn’t Real, Either
So far, we’ve assumed that the address you supply identifies a location in RAM. Usually, it doesn’t. Modern operating systems use virtual memory. Your program sees a virtual address space that the operating system and processor cooperate to map onto physical memory.
The memory management unit, or MMU, translates virtual addresses into physical addresses, typically using page tables. This lets different programs use identical virtual addresses without interfering with one another. It also provides memory protection and allows physical pages to be shared.
Of course, translating addresses takes time. So we add another cache.
The translation lookaside buffer, or TLB, remembers recently used address translations. A TLB hit avoids a potentially expensive page-table walk. Some processors even have multiple levels of TLBs. Unlike the ordinary CPU cache, which generally operates on cache lines, virtual memory usually works with pages, often 4 KB, although larger pages are also supported.
Notice what’s happening. We have caches for data, caches for instructions, and caches for the information needed to locate the data and instructions.
The simple act of reading a variable has become surprisingly complicated.
Not All RAM Is Created Equal
Even if we get through the caches and finally reach RAM, things aren’t necessarily simple. On systems with multiple processors or certain multicore arrangements, memory access can be non-uniform. This is called NUMA, or Non-Uniform Memory Access.
A processor may have relatively fast access to memory attached to its own NUMA node, while accessing memory associated with another node requires crossing an interconnect. Both locations are ordinary RAM, and both may appear in the same address space, but the access times and available bandwidth can differ.
The operating system tries to place memory near the processors using it. Moving a thread to another CPU can therefore change its memory performance, even though the program’s addresses haven’t changed.
And NUMA isn’t limited to traditional multiprocessor servers. Modern systems with chiplets, heterogeneous memory, and technologies such as CXL are making memory topology increasingly interesting.
What If It Isn’t In Memory?
Virtual memory has another trick. A virtual page doesn’t necessarily have to be backed by physical RAM at the moment you access it.
When a program accesses a page that isn’t currently mapped in a usable way, the processor generates a page fault. The operating system decides what to do. Sometimes it simply allocates a fresh page. Sometimes it retrieves data from an executable or memory-mapped file. Other times, the page was moved out of RAM to swap space and must be retrieved. Note that things like executable data are generally immutable, so most operating systems won’t swap them out. They simply trash them and know they can reload the same page later when needed.
This means your program can address data that isn’t presently in physical memory. Naturally, accessing an SSD is much slower than accessing RAM, so excessive swapping can bring a computer to its knees.
Virtual memory doesn’t actually require swapping, though. Address translation and protection are useful even on systems that never move pages to disk.
Your SSD Is Memory, Too
Suppose your program reads a file. You might imagine that the operating system sends a request to the SSD and waits for the data. But if the file was recently accessed, the data may already be in RAM.
Operating systems maintain a filesystem page cache, keeping frequently or recently accessed file data in memory. Writes may also be buffered in RAM before eventually reaching persistent storage.
This makes RAM act like a cache for secondary storage, much as L3 acts like a cache for RAM.
The analogy isn’t perfect. RAM is also the primary working storage for running programs, and anonymous memory isn’t necessarily a copy of anything on disk. But the general hierarchy is clear: registers, CPU caches, RAM, and persistent storage provide progressively different tradeoffs between speed, capacity, and cost.
Even SSDs have their own caches and address translations. Their flash translation layers map logical block addresses onto physical flash locations, hiding wear leveling and garbage collection from the host. Apparently, nobody wants to tell the truth about addresses.
Compressing The Middle
But what if we could make RAM effectively larger without adding more? Compression offers another way to complicate our memory hierarchy. Instead of immediately moving an infrequently used page to disk, why not compress it and keep it in RAM?
Linux has two well-known mechanisms for doing this: zram and zswap. Zram creates a compressed block device in RAM. It’s commonly used as a swap device, allowing pages to be stored in compressed form rather than written to physical storage.
Zswap, on the other hand, is a compressed cache in front of a swap device. When Linux decides to swap out a page, zswap attempts to compress and retain it in RAM. If necessary, pages can eventually be evicted to the backing swap device.
Both trade processor time for memory capacity and reduced storage I/O. Decompressing a page costs CPU cycles, but that can be considerably cheaper than retrieving it from an SSD.
There’s a catch, though. These mechanisms operate through the swap subsystem. When a compressed page is needed again, it must be decompressed into an ordinary page before the program can access it. That means page faults, memory management overhead, and moving data around. But what if compressed memory could behave more like ordinary RAM?
Meta Wants To Change The Rules
That’s the idea behind CRAM, or Compressed RAM, a project presented by Meta’s Gregory Price at the October 2026 Linux Plumbers Conference.
Rather than treating compressed memory as a conventional swap device, CRAM proposes exposing it through a special NUMA node. Linux can then use existing memory-management mechanisms to move pages between ordinary DRAM and compressed memory. The most interesting part is the hardware-assisted compression and decompression, which allows access at much finer granularity than conventional page-oriented compressed swap.
The developers report impressive benchmark results, including read performance that approaches ordinary DRAM in some tests and dramatic improvements over zram for selected workloads.
That doesn’t mean compression has suddenly become free. Writes, compression ratios, hardware requirements, and managing incompressible data all matter. CRAM is still developmental work, not ready for everyday use.
But it illustrates where the memory hierarchy is heading. Instead of simply deciding whether a page is in RAM or on disk, the operating system may choose among ordinary RAM, compressed memory, slower memory tiers, and secondary storage.
The Illusion Is The Feature
Most programmers don’t need to think about any of this most of the time. That’s the point of an abstraction. But when performance matters, the illusion starts to leak. Sequential access can beat random access by enormous margins. Two threads modifying unrelated variables can fight over cache lines. Moving a process between CPUs can change its memory latency. A page that looks like ordinary memory might actually require decompression or a trip to an SSD.
On smaller embedded systems, of course, the picture may be very different. Without an MMU or elaborate caches, memory can look much more like the simple physical address space we learned about in the first place.
We started with an array of bytes. We ended up with a hierarchy of caches, translation tables, NUMA nodes, compressed memory pools, and persistent storage devices, all cooperating to maintain that simple illusion. Amazing how complicated something so simple can be.
This reminds me of a thought I had.
With the success of AMD’s 3D Vcache, I feel like the next step is for them to introduce a RAM chiplet onto their CPUs. It would as another lower latency option before going to system memory. They if it’s large enough, it could also allow people to run RAMless mini machines.
You’ve reinvented L4 cache, also sometimes called eDRAM (embedded DRAM)
Cool, now get them to do it
Done, about a decade ago
https://www.intel.com/content/www/us/en/products/sku/88040/intel-core-i75775c-processor-6m-cache-up-to-3-70-ghz/specifications.html
Not AMD and not with modern chiplet design.
BTW, part of the point of my thought on the design is that the RAM wouldn’t be on the chip like L3 cache (or L4 eDRAM), it would be a separate chiplet on the board and connected to the Infinity Fabric like another die.
Since it’d be an entire other chip it could be huge with special memory management hardware and much better latency than standard RAM.
Lol those are things you learn during the first semester of CS course.
If people without education want to write code they should not be allowed becuase they cause more harm than they develop.
Good thing people can do things without being “allowed” to, then!
Exactly, they vibe-code so everyone can enjoy their incompetence :)
Like dentists, doctors and surgeons?
Right now there is $3 trillion riding on the premise that nobody should ever understand code they write again.
We have noted your superiority, unfortunately it is negated by reading the article after the clickbait headline.
I don’t need to read. I just click and comment ( ͡° ͜ʖ ͡°)
This statement is ridiculous. All of these features are there to abstract programming problems from the software developer through hardware. This was done to let programmers focus on the fundamental task of writing code while ignoring the details of memory location and coding for performance. Hardware designerd could save themselves a lot of heartache leaving this out. Processors could be faster, cost less, and use less power if they did.
The problem with CS majors is they know programming and not necessarily the specific topic/application the program is for. This is why we end up with people programming without CS degrees. Some times if you want something done right you have to do it yourself.
That last paragraph is a huge bingo. If you have to wait for the IT guys to configure things you never get exactly what you want, because of the blindingly obvious condition that they don’t understand your job.
Me personally, I think they should shut down all computing journalism, discussion forums and educational materials since if you got a proper education at an expensive prestigious university (like I did) you wouldn’t need those things; those resources do more harm than good because they let people other than me make an attempt at coding. Me personally I just read man pages and language specs all day.
Pfft, allowed to? This is a hacking site. The whole premise is doing things we’re not “allowed” to do. If we only wanted to read manuals and follow instructions, we wouldn’t be here.
:p
This has been the case for most high-level languages and libraries where programmers are not supposed to know how the computer works to get results. That also matches your statement where high-level languages are as fast to write as they are slow to execute.
Clickbait title. Why HA goes so low?
Bc HA dig always deeper to keep headroom in the -Z direction
Ya, also a different pacing to the writing than I’d expect. Almost feels a tad AI with the paragraph blocks and vauge slightly clickbaity headers.
Maybe it’s that AI writing is changing how humans write…. Which might be even scarier than AI itself being used to write this?
Oh no you were correct the first time; it’s definitely LLM-“assisted”.
…
‘Most programmers don’t need to think about any of this most of the time. That’s the point of an abstraction. But when performance matters, the illusion starts to leak. Sequential access can beat random access by enormous margins. Two threads modifying unrelated variables can fight over cache lines. Moving a process between CPUs can change its memory latency. A page that looks like ordinary memory might actually require decompression or a trip to an SSD.’
…is a classic example of LLM-generated output. The style is unmistakable.
I saw that “let me explain what I’m saying as I say it with a list of things to say”-style of writing long before LLMs were a thing. Must less common, sure, but it existed.
Well, I didn’t really believe you, so I cut’n’paste that paragraph into several different AI text detectors. Three said 100% AI, one said 92% chance it is AI… I also tried large chunks (since accuracy improves with more data) of the article, still 100% AI (with 92% one going to 96%).
For reference I used grammarly.com, humalingo.com, app.gptzero.me and pangram.com. Humalingo also called out a 91% plagiarism risk.
I’m not saying they did or didn’t use AI to write this. But your remarks appear to be backed by the data.
Well, Al IS the editor in chief. Ignoring the fact that he’s the author of this article, they use him to write every article. You don’t need grammarly.com to figure that out. You just have to check the staff page.
If someone is interested in reading about memory hierarchies and memory consistency, imho the canonical book to read is “Computer Architecture: A Quantitative Approach” by Patterson and Hennessy.
If anyone can suggest some other quality papers, please share the titles.
It’s come quiet a way from using a RAM disk in DOS to avoid waiting for the floppy drive.
it’s a fine survey article as it goes, but i really did not like the headline, this ‘lie’ framing. no one is being deceived. the people who understand memory enough well enough to imagine the counterfactual (one bus transaction for each load/store instruction) also understand memory well enough to know all this stuff already. there are people who still have yet to learn about cache and coherence and all the details, but for the most part they don’t have false ideas, they just don’t know yet.
an address is just a handle. no need to make it spooky.
Spooky addressing at a distance.
Bravo for the Einstein reference!
@Greg A this article is almost entirely LLM-generated, that’s why it has this nauseating clickbaity style. As soon as you start noticing it, you can tell whether the YouTube video you’re watching is LLM-generated slop or mis/disinformation…
Ah so my spidey senses served me well again by telling me to shitpost in comments without reading the “article” 😂
That’s a grade A meatship there ♪
Sorry, but the article is written in a style “They’re lying to us! Everything is a hoax!”.
We’re completely aware that our computers have virtual memory. The personal ones (aka PC) have it from ~1985, aka 41y ago. The reason it was adopted was to allow the OS and multiple applications running at the same time without breaking/interfering with each other. If you don’t want virtual memory and protected mode (which is more than simply memory protection), you can turn it off and run whatever software is available for such configuration (DOS maybe?).
Regarding the caches, everyone is completely aware of their existence, and basically no one cares to the extent that their applications are running fine (and engineers design their memory hierarchies in a way that makes customers’ applications run a little bit faster with each CPU model release).
These topics are extensively studied in university CS programs, and there’s a ton of technical literature that covers them. All these technologies are just the natural evolution of solutions and compromises that we’re wiling to make in order to advance the performance of our computers.
Can we build architectures where the fundamental optimization target is information density rather than instruction throughput? Basically more bang for the buck.
Well, you don’t need a whole architecture for this, just a density-optimized storage.
The current NAND FLASH chips seems to fit well – they have excellent areal, volumetric and bit-level density. Also we already have filesystems capable of different levels of compression, including open-source ones.
Do you have some exotic application which is not satisfied by the current technologies?
Who cares whether an SSD has sectors and cylinders or not. As long as it serves the right bytes I’m OK with it.
IBM had MXT back in the 2000s. I haven’t read the CRAM paper, but real-time chipset based memory compression has been around a while.
Virtual memory is a look up table wired to some of the upper bits of your address. Logically everything except the bottom 12 bits (4K), but really it’s tiers of lookup to make a sparse data structure so you can have lots of unused (and unmapped) areas, or collapse some levels together to make larger pages (huge pages).
Virtual memory isn’t conceptually very difficult to understand. And even in the physical world we use one table to look up something in another. (or at least we did back when people knew how to use a card catalog or a office filing system)
Oh, yes, everything is so simple when someone else will implement it :D.
If it is there, and you can see it, it’s real
If it’s there, and you can’t see it, it’s virtual
If it’s not there, and you can see, it’s imaginary
If it’s not there, and you can’t see it, it’s been stolen.
I actually liked this article.
I did too.
It’s not all new, but it leads into the recent push by Meta at the compression thing. Most articles are ignoring the whole part where it requires new hardware that our computers don’t have now, and make it sound like we can just download more RAM. Al avoided that.
You say lie, I say technique. “Lie” is such a strong word..
I liked this article.