Your Computer Memory Is A Lie

When you first learn to program, memory seems pretty straightforward. You have a bunch of bytes, each with an address. If you want to read or write something, you specify the address, and the computer goes to that place and does your bidding. Sure, some addresses might be reserved for hardware or the operating system, but the general idea is simple: memory is a big array of bytes.

Just like we pretend that a wire is a perfect conductor, the big array of memory model used to be a slight twisting of the truth. On a modern computer, however, it is almost entirely a lie. The address you use probably isn’t a physical address. The data you read might not come from RAM at all. Even if it does, it might be sitting in one of several caches. And if you follow the rabbit hole far enough, you discover that your SSD is arguably just another level in the memory hierarchy.

The Need For Speed

The fundamental problem is that processors have gotten much faster than memory. A modern processor can execute several instructions per clock cycle, but accessing ordinary DRAM can take hundreds of cycles. If the processor had to wait for RAM every time it needed data, much of that impressive computing power would go to waste.

The solution is caching. Keep copies of frequently used data in smaller, faster memories closer to the processor. Most modern desktop processors have at least three levels of cache. The L1 cache is tiny but extremely fast, and often split between instructions and data. L2 is larger and somewhat slower, while L3 is even larger and usually slower as well.

Continue reading “Your Computer Memory Is A Lie” →

Lead A Discord Exodus With A Matrix Server

The Internet, and large groups of people in general, tend to have a remarkably short collective memory. Whether it’s the nature of algorithms or the people themselves to always be chasing the next novelty, maintaining long-term engagement for even relatively small changes can be challenging. Distractions abound, and motivation tends to fade over time. This past spring, Discord’s unpopular decision to force age verification resulted in a brief flash-in-the-pan of outrage that dissipated when they promised to delay its roll-out. Rather than backtrack on unpopular policies, they’ve been slowly boiling the frog with incremental changes towards this same goal in the meantime. One of the options to deal with this is to migrate over to something like a Matrix server, which has a number of benefits over Discord including being open-source and self-hostable.

Unlike Discord, a closed platform subject to the whims of a private company, Matrix is a communications protocol and open standard. Its major server and client implementations are open source, so anyone can build tools around it and integrate it with other systems that also use this protocol. This protocol-vs-platform comparison mirrors email, which is another protocol that isn’t locked down by any one company, and which can easily be used to send messages across various providers. The most commonly used server-side software for Matrix is called Synapse and the most prominent general-purpose clients are the various implementations of Element, and my exploration uses these defaults.

Continue reading “Lead A Discord Exodus With A Matrix Server” →

ESP32 As Your Raspberry Pi’s Linux Wireless Co-processor

A laptop with no wireless connectivity feels like a rarity – but when you shop for powerful Linux single-board computers (SBCs), it is pretty common to see one skimp on WiFi. At the same time, current wireless cards can leave much to be desired. A mess of proprietary firmware, ephemeral or even binary blob drivers, weird glitches never resolved, and sometimes entirely source-less blobs with an expiration date. Maybe the SBC doesn’t have a free USB port for a plug-and-play WiFi chip, or you don’t want to design for an obscure hard-to-source castellated module.

What if I told you there’s a unique solution to the problem, as long as the SBC has an SDIO or SPI to spare? There’s a wireless card we can all try to get behind, and it’s certainly not flawless but gives us way more room to grow – it’s got somewhat open-source firmware, it comes from a respectable company, and it’s nigh-guaranteed to be easy to source. I am, of course, talking about esp-hosted – using ESP32 modules as your WiFi/BT card, over SDIO or SPI.

I’ve recently added an ESP32-C6 into my project, and setting up esp-hosted wasn’t ultra straightforward – so here’s a guide, and a discussion on what makes esp-hosted so cool and so incredibly promising. Spoilers: it works with privacy switches that cut out power, it could be hacked into a coprocessor for mesh protocols, and it could potentially help us build a fully open-source SDIO WiFi card.

Continue reading “ESP32 As Your Raspberry Pi’s Linux Wireless Co-processor” →

How A British Engineer Homebrewed The Quietest Carbine

Hollywood has a lot to answer for when it comes to “silencers” on firearms. On screen, a suppressed pistol makes a pinging thud, and the bad guy in the next room flops to the floor. Real life is rather less cinematic. A suppressor takes the edge off the report, but it’s by no means quiet or silent. Hence why the term “silencer” is more of a colloquialism, and why “suppressor” is more appropriate.

Every so often, though, someone gets remarkably close to the movie version. The best-known example is probably the De Lisle carbine. A British commando weapon from the Second World War, it was cobbled together from bits and pieces and ended up being a stellar performer in the world of clandestine operations.

Continue reading “How A British Engineer Homebrewed The Quietest Carbine” →

Fixing A Power Grid That Loses Over Half Its Power To Inefficiency And Theft

[Mini Shaji Thomas] has a deeply interesting article on IEEE Spectrum detailing how the power distribution system for a city of over twenty million people went from failing to first-class. In the early 2000s, over half of Delhi’s generated power was lost to inefficiencies of all kinds, and theft. Losses were technical as well as administrative, and the rampant theft was perpetuated by “both the powerful and the powerless”. Spoiler alert: it’s a big job.

That level of loss is staggering, and caused by many issues rather than just one or two. Many of the problems tended to compound one another. For example, an effective power grid relies on balancing active and reactive power, but if that is not well managed then inefficiencies cause losses and outages that tend to lead to even more losses and outages as the system is never able to catch up to demand. On the scale of tens of millions of inhabitants, such losses are far from trivial.

There wasn’t a silver bullet solution. Modernizing and stabilizing the power distribution required technical resources, modernized equipment, political will, expert planning, a willingness to follow through, and authority.

The starting point was grim: a failing network that was constantly overloaded, poor or nonexistent controls and safeguards, inaccurate records and inefficient administration, no real means of sensing faults or inefficiencies or theft, weak to nonexistent enforcement, and an untrained workforce. Still, it could be done, and it was. While there’s still improvement to be made, revamping the system was ultimately successful.

Individually, some of the solutions were quite simple. For example, replacing bare overhead distribution wires with a system of insulated cables containing multiple conductors was not just safer and more resilient, it made it much harder for people to tap into the lines. Other initiatives like making it easier and more convenient to pay accurate bills helped, too. Turns out it’s easier to have legit customers when the product and service is reliable.

Give it a read if you have a few minutes, because besides providing a peek into the technical end of how effective power grids work, it’s a good analysis of what can be accomplished with some co-operation and a willingness to change.

Roman Telescope Saves Fuel, Doubles Mission

Contemplate the events that might end a space mission, and you might picture something dramatic: detonation on the pad, a dead guidance system, or micrometeoroids shattering delicate solar panels. More often, though, the ending is far more mundane. Plenty of perfectly healthy spacecraft have been retired simply because the fuel tanks ran dry. That’s why mission planners guard every kilogram of go juice so jealously, and why careful preparation in a mission is critical to long-term success.

Which brings us to NASA’s Nancy Grace Roman Space Telescope. Barely two weeks after its August 30 launch on a Falcon Heavy, the mission team announced that Roman now has enough fuel for at least 22 years of science operations. That’s well over double its original 10-year fuel budget. It’s a huge gain, so let’s explore how NASA pulled it off.

Continue reading “Roman Telescope Saves Fuel, Doubles Mission” →

Geothermal Cooling For New York’s Subways

You might think that if you dug some tunnels down into the ground where you could be out of the sunshine, that you might find them to be relatively cool and comfortable inside. Yet, fill those tunnels with trains and supporting equipment and millions of commuters, and you’ll instead find yourself sweltering in the heat.

New York’s subways are too hot for comfort, and have been for some time. Unfortunately, unlike any old regular building, you can’t just air condition a subway and call it done. Instead, authorities are looking for other solutions to beat the heat on the famous underground transit network.

Continue reading “Geothermal Cooling For New York’s Subways” →