“Please say it wasn’t a regex, please say it wasn’t a regex; aww, crap, it was a regex!” That seems to be the conclusion now that Crowdstrike has released a full root-cause analysis of its now-infamous Windows outage that took down 8 million machines with knock-on effects that reverberated through everything from healthcare to airlines. We’ve got to be honest and say that the twelve-page RCA was a little hard to get through, stuffed as it was with enough obfuscatory jargon to turn off even jargon lovers such as us. The gist, though, is that there was a “lack of a specific test for non-wildcard matching criteria,” which pretty much means someone screwed up a regular expression. Outside observers in the developer community have latched onto something more dire, though, as it appears the change that brought down so many machines was never tested on a single machine. That’s a little — OK, a lot — hard to believe, but it seems to be what Crowdstrike is saying. So go ahead and blame the regex, but it sure seems like there were deeper, darker forces at work here.
root cause2 Articles
Fail Of The Week: Thermostat Almost Causes A House Fire
Fair warning: any homeowners who have thermostats similar to the one that nearly burned down [Kerry Wong]’s house might be in store for a sleepless night or two, at least until they inspect and perhaps replace any units that are even remotely as sketchy as what he found when he did the postmortem analysis in the brief video below.
The story begins back in the 1980s, when the Southern New England area where [Kerry] lives enjoyed a housing boom. Contractors rushed to turn rural farmland into subdivisions, and new suburbs crawled across the landscape. Corners were inevitably cut during construction, and one common place to save money was the home’s heating system. Rather than engage an HVAC subcontractor to install a complicated heating system, many builders opted instead to have the electricians install electric baseboards. They were already on the job anyway, and at the time, both copper and electricity were cheap.
Fast forward 40 years or so, and [Kerry] finds himself living in one such house. The other night, upon catching the acrid scent of burning insulation, he followed his nose to the source: a wall-mounted thermostat for his electric baseboard. His teardown revealed burned insulation, bare conductors, and scorched plastic on the not-so-old unit; bearing a 2008 date code, the thermostat must have replaced one of the originals. [Kerry] poked at the nearly combusted unit and found the root cause: the spot welds holding the wires to the thermostat terminal had become loose, increasing the resistance of the connection. As [Kerry] points out, even a tenth of an ohm increase in resistance in a 15 amp circuit would dissipate 20 watts of heat, and from the toasty look of the thermostat it had been a lot more than that.
The corner-cutting of the 1980s was nothing new, of course – remember the aluminum wiring debacle? Electrical fires are no joke, and we’re glad [Kerry] was quick to locate the problem and prevent it from spreading.
Continue reading “Fail Of The Week: Thermostat Almost Causes A House Fire”