The 2003 Northeast Blackout And The Harsh Lessons Of Grid Failures

August 29, 2023 by Maya Posch 87 Comments

The grid failure in 2003 which reverted much of the eastern US and Canada back to a pre-electrification era may be rather memorable, yet it was not the first time that a national, or even international power grid failed. Nor is it likely that it will be the last. In August of 2023 we mark the 20th anniversary of this blackout which left many people without electricity for up to three days, while costing dozens of people their lives. This raises the question of what lessons we learned from this event since then.

Although damage to transmission lines and related infrastructure is a big cause of power outages – especially in countries where overhead wiring is the norm – the most serious blackouts involve the large-scale desynchronization of the grid, to the point where generators shutdown to protect themselves. Bringing the grid back from such a complete blackout can take hours to days, as sections of the grid are reconnected after a cascade scenario as seen with the 2003 blackout, or the rather similar 1965 blackout which affected nearly the same region.

With how much more modern society relies today on constant access to electrical power than it did twenty, let alone fifty-eight years ago, exactly how afraid should we be of another, possibly worse blackout?

Continue reading “The 2003 Northeast Blackout And The Harsh Lessons Of Grid Failures” →

Blame It On The Sockets: Forensic Analysis Of The Arecibo Collapse

August 29, 2023 by Dan Maloney 58 Comments

Nearly three years after the rapid unplanned disassembly of the Arecibo radio telescope, we finally have a culprit in the collapse: bad sockets.

In case you somehow missed it, back in 2020 we started getting ominous reports that the cables supporting the 900-ton instrument platform above the 300-meter primary reflector of what was at the time the world’s largest radio telescope were slowly coming undone. From the first sign of problems in August, when the first broken cable smashed a hole in the reflector, to the failure of a second cable in November, it surely seemed like Arecibo’s days were numbered, and that it would fall victim to all the other bad luck we seemed to be rapidly accruing in that fateful year. The inevitable finally happened on December 1, when over-stressed cables on support tower four finally gave way, sending the platform on a graceful swing into the side of the natural depression that cradled the reflector, damaging the telescope beyond all hope of repair.

The long run-up to the telescope’s final act had a silver lining in that it provided engineers and scientists with a chance to carefully observe the failure in real-time. So there was no real mystery as to what happened, at least from a big-picture perspective. But one always wants to know the fine-scale details of such failures, a task which fell to forensic investigation firm Thornton Tomasetti. They enlisted the help of the Columbia University Strength of Materials lab, which sent pieces of the failed cable to the Oak Ridge National Laboratory’s High Flux Isotope reactor for neutron imaging, which is like an X-ray study but uses streams of neutrons that interact with the material’s nuclei rather than their electrons.

The full report (PDF) reveals five proximate causes for the collapse, chief of which is “[T]he manual and inconsistent splay of the wires during cable socketing,” which we take to mean that the individual strands of the cables were not spread out correctly before the molten zinc “spelter socket” was molded around them. The resulting shear stress caused the zinc to slowly flow around the cable strands, letting them slip out of the surrounding steel socket and — well, you can watch the rest below for yourself.

As is usually the case with such failures, there are multiple causes, all of which are covered in the 300+ page report. But being able to pin the bulk of the failure on a single, easily understood — and easily addressed — defect is comforting, in a way. It’s cold comfort to astronomers and Arecibo staff, perhaps, but at least it’s a lesson that might prevent future failures of cable-supported structures.

Continue reading “Blame It On The Sockets: Forensic Analysis Of The Arecibo Collapse” →

Making Electricity At The South Pole

August 24, 2023 by Chris Lott 78 Comments

In case you’ve ever wondered how the South Pole research stations are powered, then a recent blog post, South Pole Electrical Infrastructure by anonymous IT engineer [brr] is for you. Among the many issues covered, let’s look at how the electricity is made and, spoiler alert, how the specially formulated AN8 fuel blend is transported to the generators.

The main source of power is a trio of Caterpillar 3512B diesel generator sets, de-rated to 750 kW each due to the high altitude and the special fuel mixture. Unsurprisingly, all the fuel must be imported to Antarctica, a horribly inefficient endeavor. Fuel arrives initially at McMurdo Station harbor by tanker ship. From there, it can be sent to the Amundsen-Scott South Pole Station in one of two ways. The Lockheed LC-130 is a modified C-130 Hercules cargo plane developed in the 1950s specifically to support polar operations. It is the least efficient method, consuming 1.33 kg to transport 1 kg of fuel. Alternatively, fuel can be dragged by tractors via the South Pole Overland Traverse (SPoT), a 1600 km highway over compacted snow and ice. The trek takes about 40 days and only consumes 0.56 kg of fuel for every 1 kg, which is much better than air.

Continue reading “Making Electricity At The South Pole” →

Giving Solar Power’s Mortal Enemies A Dusting Without Wasting Water

August 16, 2023 by Maya Posch 28 Comments

A prerequisite for photovoltaic (PV) and concentrated solar power (CSP) technologies to work efficiently is as direct an exposure to the electromagnetic radiation from the sun as possible. Since dust and similar particulates are excellent at blocking the parts of the EM spectrum that determine their efficiency, keeping the panels and mirrors free from the build-up of dust, lichen, bird droppings and other perks of planetary life is a daily task for solar farm operators. Generally cleaning the panels and mirrors involves having trucks drive around with a large water tank to pressure wash the dirt off, but the use of so much water is problematic in many regions.

Keeping PV panels clean is also a consideration on other planets than Earth. So far multiple Mars rovers and landers have found their demise at the hands of Martian dust after a layer covered their PV panels, and Moon dust (lunar regolith) is little better. Despite repeated suggestions by the peanut gallery to install wipers, blowers or similar dust removal techniques, keeping particulates from sticking to a surface is not as easy an engineering challenge as it may seem, even before considering details such as the scaling issues between a singular robot on Mars versus millions of panels and mirrors on Earth.

There has been research into the use of the electrostatic effect to repel dust, but is there a method that can keep both solar-powered robots on Mars and solar farms on Earth clean and sparkling, rather than soiled and dark?

Continue reading “Giving Solar Power’s Mortal Enemies A Dusting Without Wasting Water” →

Under The Sea: Optical Repeaters For Submarine Cables

August 8, 2023 by Dan Maloney 25 Comments

Once a month or so, I have the privilege of sitting down with Editor-in-Chief Elliot Williams to record the Hackaday Podcast. It’s a lot of fun spending a couple of hours geeking out together, and we invariably go off on ridiculous tangents with no chance of making the final cut, except perhaps as fodder for the intro and outro. It’s a lot of work, especially for Elliot, who has to edit the raw recordings, but it’s also a lot of fun.

Of course, we do the whole thing virtually, and we have a little ritual that we do at the start: the clapping. We take turns clapping our hands into our microphones three times, with the person on the other end of the line doing a clap of his own synchronized with the final clap. That gives Elliot an idea of how much lag there is on the line, which allows him to synchronize the two recordings. With him being in Germany and me in Idaho, the lag is pretty noticeable, at least a second or two.

Every time we perform this ritual, I can’t help but wonder about all the gear that makes it possible, including the fiber optic cables running beneath the Atlantic Ocean. Undersea communications cable stitch the world together, carrying more than 99% of transcontinental internet traffic. They’re full of fascinating engineering, but for my money, the inline optical repeaters that boost the signals along the way are the most interesting bits, even though — or perhaps especially because — they’re hidden away at the bottom of the sea.

Continue reading “Under The Sea: Optical Repeaters For Submarine Cables” →

DisplayPort: Taming The Altmode

August 3, 2023 by Arya Voronova 7 Comments

The DisplayPort altmode is semi-proprietary, but it can absolutely be picked apart if we try. Last time, we found a cool appnote describing the DisplayPort altmode in detail, switched the FUSB302 into packet sniffing mode and got packet captures, learned about PD VDMs (vendor-defined messages), and successfully replayed the captured messages to switch a USB-C port into the DisplayPort altmode. Today, we will go through the seven messages that summon the DisplayPort altmode, implement them, and tie them all into a library – then, figure out the hardware we need to have DisplayPort work in the wild.

For a start, as you might have seen from the diagram, a single command can be either a request or a response. For instance, if you get a Discover Identity REQ (request), you reply to it with a Discover Identity ACK (response), adding your identity data to your response along the way. With some commands, the DP source will add some data for you to use; for most commands, your DP sink will have to provide information instead – and we’ll do just that, armed with the PDF provided and the packet captures.

We have seven commands we need to handle in order to get DisplayPort out of a compatible USB-C port – if you need a refresher on these commands, page 13 of the ST’s PDF on the DP altmode will show you the message sequence. These commands are: Discover Identity, Discover SVIDs, Discover Modes, Enter Mode, DP Status Update, DP Configure, and Attention. Out of these, the first four are already partially described in the base USB PD standard, the two DP commands afterwards are DisplayPort-altmode-specific but sufficiently described in the PDF we have, and the Attention command is from the base standard as well, mostly helpful for reporting state of the HPD pin. Let’s start with the first two! Continue reading “DisplayPort: Taming The Altmode” →

Clipper Windpower: Solutions In Search Of Problems

August 2, 2023 by Bryan Cockfield 67 Comments

The first modern wind turbines designed for bulk electricity generation came online gradually throughout the 80s and early 90s. By today’s standards these turbines are barely recognizable. They were small, had low power ratings often in the range of tens to hundreds of kilowatts, and had tiny blades that had to rotate extremely quickly.

When comparing one of these tiny machines next to a modern turbine with a power rating of 10 or more megawatts with blades with lengths on the order of a hundred meters, one might wonder if there is anything in common at all. In fact, plenty of turbines across the decades share fundamental similarities including a three-blade design, a fairly simple gearbox, and a single electric generator. While more modern turbines are increasingly using direct-drive systems that eliminate the need for a gearbox and the maintenance associated with them, in the early 2000s an American wind turbine manufacturer named Clipper Windpower went in the opposite direction, manufacturing wind turbines with an elaborate, expensive, and heavy gearbox that supported four generators in each turbine. This ended up sealing the company’s fate only a few years after the turbines were delivered to wind farms.

Some history: the largest terrestrial wind turbines were approaching the neighborhood of 2 megawatts, but some manufacturers were getting to these milestones essentially by slapping on larger blades and generators to existing designs rather than re-designing their turbines from the ground up to host these larger components. This was leading to diminishing returns, as well as an increased amount of mechanical issues in the turbines themselves, and it was only a matter of time before the existing designs wouldn’t support this trend further. Besides increased weight and other mechanical stresses on the structure itself, another major concern was finding (and paying for) cranes with enough capacity to hoist these larger components to ever-increasing heights, especially in the remote locations that wind farms are typically located. And cranes aren’t needed just for construction; they are also used whenever a large component like a generator or blade needs to be repaired or replaced. Continue reading “Clipper Windpower: Solutions In Search Of Problems” →