“Please say it wasn’t a regex, please say it wasn’t a regex; aww, crap, it was a regex!” That seems to be the conclusion now that Crowdstrike has released a full root-cause analysis of its now-infamous Windows outage that took down 8 million machines with knock-on effects that reverberated through everything from healthcare to airlines. We’ve got to be honest and say that the twelve-page RCA was a little hard to get through, stuffed as it was with enough obfuscatory jargon to turn off even jargon lovers such as us. The gist, though, is that there was a “lack of a specific test for non-wildcard matching criteria,” which pretty much means someone screwed up a regular expression. Outside observers in the developer community have latched onto something more dire, though, as it appears the change that brought down so many machines was never tested on a single machine. That’s a little — OK, a lot — hard to believe, but it seems to be what Crowdstrike is saying. So go ahead and blame the regex, but it sure seems like there were deeper, darker forces at work here.
CrowdStrike4 Articles
A History Of Internet Outages
We heard a story that after the recent hurricane, a man noted that while the house was sweltering hot because the power was still out, his kids were more anxious for the internet to come back online. The Internet is practically a basic necessity for most people, but as you may have noticed with the recent CrowdStrike debacle, the Internet isn’t always reliable. Granted, the problem in that case wasn’t the Internet per se, but a problem with many critical hosts that provide services. [Thomas Germain] from the BBC took the opportunity to recall some of the more bizarre reasons we’ve had massive Internet outages in the past.
While teens after a hurricane might miss social media, global outages can be serious business. With 8.5 million computers dead, 911 services went down, medical surgeries were canceled, and — of course — around 46,000 flights were canceled in a single day. We have short memories for these outages, but as [Thomas] points out, this was far from the first massive outage, and many of them have very strange backstories.
This Week In Security: EvilVideo, Crowdstrike, And InSecure Boot
First up this week is the story of EvilVideo, a clever telegram exploit that disguises an APK as a video file. The earliest record we have of this exploit is on June 6th when it was advertised on a hacking forum.
Researchers at ESET discovered a demo of the exploit, and were able to disclose it to Telegram on June 26th. It was finally patched on July 11. While it was advertised as a “one-click” exploit, that’s being a bit generous, as the ESET demo video shows. But it was a clever exploit. The central trick is that an APK file can be sent in a Telegram chat, and it displays what looks like a video preview. Tap the “video” file to watch it, and Telegram prompts you to play it with an external player. But it turns out the external player in this case is Android itself, which prompts the target to install the APK. Sneaky.
Continue reading “This Week In Security: EvilVideo, Crowdstrike, And InSecure Boot”
Hackaday Links: July 21, 2024
When monitors around the world display a “Blue Screen of Death” and you know it’s probably your fault, it’s got to be a terrible, horrible, no good, very bad day at work. That’s likely the situation inside CrowdStrike this weekend, as engineers at the cybersecurity provider struggle to recover from an update rollout that went very, very badly indeed. The rollout, which affected enterprise-level Windows 10 and 11 hosts running their flagship Falcon Sensor product, resulted in machines going into a boot loop or just dropping into restore mode, leaving hapless millions to stare at the dreaded BSOD screen on everything from POS terminals to transit ticketing systems.