Hackaday Links: August 11, 2024

August 11, 2024 by Dan Maloney 8 Comments

“Please say it wasn’t a regex, please say it wasn’t a regex; aww, crap, it was a regex!” That seems to be the conclusion now that Crowdstrike has released a full root-cause analysis of its now-infamous Windows outage that took down 8 million machines with knock-on effects that reverberated through everything from healthcare to airlines. We’ve got to be honest and say that the twelve-page RCA was a little hard to get through, stuffed as it was with enough obfuscatory jargon to turn off even jargon lovers such as us. The gist, though, is that there was a “lack of a specific test for non-wildcard matching criteria,” which pretty much means someone screwed up a regular expression. Outside observers in the developer community have latched onto something more dire, though, as it appears the change that brought down so many machines was never tested on a single machine. That’s a little — OK, a lot — hard to believe, but it seems to be what Crowdstrike is saying. So go ahead and blame the regex, but it sure seems like there were deeper, darker forces at work here.

Continue reading “Hackaday Links: August 11, 2024” →

Regular Expressions Finally Come To Microsoft Excel

May 24, 2024 by Dan Maloney 29 Comments

There are two types of people in the world: those who have no idea what a regular expression is, and those who not only know what they are but can compose them on the fly and tend to use them in situations where they’re clearly not called for. And it’s that latter camp, of which we consider ourself a proud member, that is rejoicing with the announcement that Microsoft is adding regular expression support to Excel.

Or perhaps not rejoicing so much as wondering what took so long. Yes, regular expressions have been part of VBA for a while now, but the new functions allow you to use regexes right in the spreadsheet grid. There are plenty of caveats, of course. The big one is that this is still in beta at this time, so you have to do some gymnastics to enable it, if you’re even allowed to in the first place. Second, support appears limited to three functions at the moment: REGEXTEST, which provides a logical test of pattern matching; REGEXEXTRACT, which returns a substring that matches a pattern; and REGEXREPLACE, which substitutes a string for a pattern. The video below walks through how to use these functions within spreadsheets.

What’s also unclear now is what flavor of regular expressions is supported. There are a bewildering number of entities in the regex bestiary — character classes, positional indicators, quantifiers, subexpressions, lazy and greedy matches, and a range of grouping constructs that perplex even regex pros. One hopes these new functions will support one of the existing regex standards, but Microsoft is famous for “extending and enhancing.” Then again, regex support has been in the .NET Framework for years and is pretty close to the Perl standard, so our guess is that it’ll be close to that.

If you fall into the “What’s a regex?” camp but want to change that, why not get your grep on?

Continue reading “Regular Expressions Finally Come To Microsoft Excel” →

Grep By Example is also available as a PDF Minibook, and a Grep playground helps you learn quickly.

Galvanize Your Grip On Grep With This Great Grep Guide

March 26, 2024 by Ryan Flowers 12 Comments

These days, you can’t throw a USB stick without hitting something that’s running Linux. It might be a phone, an embedded device, or your TV. Either way, it’s running Linux, and somewhere along the line of the development of whatever your USB stick smacked into, somebody used the Global Regular Expression Print utility- better known as Grep. But what is Grep, and why do you need it? [Anton Zhiyanov] not only answers those questions but provides Grep by example: Interactive Guide to help you along.

To understand Linux, one must understand its commercial predecessor, Unix. One of the things that made Unix (and then Linux) unique was its philosophy: Write programs that work together, do one thing well, and handle text streams. This philosophy describes a huge number of programs, and one of these programs is Grep. It’s installed everywhere there’s a *nix installed, and once one becomes familiar with it, their command-line-fu reaches an all new level.

At its core, Grep is simply a bloodhound. It’s scent? A magical incantation called Regular Expressions. Regular Expressions (aka Regex) are simply a way of describing what a stream of text should look like. So when you feed Grep a bit of Regular Expression, it Prints only the text that matches that expression. Neat, right?

The trouble is that Regex can be kind of hard, and Grep has various versions and capabilities that need to be learned. And this is where the article shines- it covers both in an excellent interactive tutorial that’ll help you become a Grep Guru in no time. And if you want to do a deeper dive, check out what it takes to make your own Regex Engine from scratch!

Hackaday Links: October 30, 2022

October 30, 2022 by Dan Maloney 7 Comments

Sad news for kids and adults alike as Lego announces the end of the Mindstorms line. The much-wish-listed line of robotics construction toys will be discontinued by the end of this year, nearly a quarter-century after its 1998 introduction, while support for the mobile apps will continue for another couple of years. It’s probably fair to say that Mindstorms launched an entire generation of engineering careers, as it provided a way to quickly prototype ideas that would have been difficult to realize without the snap-fit parts and easily programmed controllers. For our money, that ability to rapidly move from idea to working model was perhaps the strongest argument for using Mindstorms, since it prevented that loss of momentum that so often kills projects. That was before the maker movement, though, and now that servos and microcontrollers are only an Amazon order away and custom plastic structural elements can pop off a 3D printer in a couple of hours, we can see how Mindstorms might no longer be profitable. So maybe it’s a good day to drag out the Mindstorms, or even just that big box of Lego parts, and just sit on the carpet and make something.

Continue reading “Hackaday Links: October 30, 2022” →

13,000 Regular Expressions Make An Editor’s Life Easier

February 1, 2021 by Jenny List 41 Comments

Being an editor is a job that seems deceptively easy until you are hauled over the coals for letting a textual howler go to print (or website). Most publications have style guides to ensure that their individual voice is preserved, but even the most eagle-eyed will sometimes slip up in their application. At the Guardian newspaper in the UK they have been struggling with this against an ever-evolving style guide that must adapt to fast-moving world events, to the extent that they had a set of regular expressions to deal with commonly-occurring problems. A lot of regular expressions, in fact around 13,000 of them.

Clearly some form of management was required, and a team of developers set about taming this monster. The result is Typerighter, their server-side document-checker, which can be found in a GitHub repository. Surprisingly for rule management they started with a Google Sheet, a choice which proved unexpectedly robust when working with such a long list even though they later replaced it. The back end doing the job of text matching was written in Scala, and for the front end a plugin was created for their Prosemirror text editor.

For a publication of course this is extremely interesting, but where’s the interest for hackers? The answer lies in any text-processing engine that uses a lot of regular expressions; those of you who have dabbled in this space will know how unwieldy this work can become. Any user of computational linguistic techniques in the pursuit of language processing could probably find much of interest here.

If you’re a bit hazy on regular expressions, how about the episode on them from our long-running Linux-fu series?

This Week In Security: IOS Wifi Incantations, Ghosts, And Bad Regex

December 4, 2020 by Jonathan Bennett 8 Comments

I hope everyone had a wonderful Thanksgiving last week. My household celebrated by welcoming a 4th member to the family. My daughter was born on Wednesday morning, November 25th. And thus explains what I did last week instead of writing the normal Hackaday column. Never fear, we shall catch up today, and cover the news that’s fit to be noticed.

iOS Zero-click Wifi Attack

[Ian Beer] of Google’s Project Zero brings us the fruit of his lockdown-induced labors, a spectacular iOS attack. The target of this attack is the kernel code that handles AWDL, an Apple WiFi protocol for adhoc mesh networks between devices. The most notable feature that makes use of AWDL is AirDrop, Apple’s device-to-device file sharing system. Because AWDL is a proprietary protocol, the WiFi hardware can’t do any accelerated processing of packets. A few years back, there was an attack against Broadcom firmware that required a second vulnerability to jump from the WiFi chip to the device CPU. Here, because the protocol is all implemented in Apple’s code, no such pivot is necessary.

And as you’ve likely deduced, there was a vulnerability found. AWDL uses Type-Length-Value (TLV) messages for sending management data. For a security researcher, TLVs are particularly interesting because each data type represents a different code path to attack. One of those data types is a list of MAC addresses, with a maximum of 10. The code that handles it allocates a 60 byte buffer, based on that maximum. The problem is that there isn’t a code path to drop incoming TLVs of that type when they exceed 60 bytes. The remainder is written right past the end of the allocated buffer.

There is more fun to be had, getting to a full exploit, but the details are a bit too much to fully dive in to here. It interesting to note that [Ian] ran into a particular problem: His poking at the target code was triggering unexpected kernel panics. He discovered two separate vulnerabilities, both distinct from the vuln he was trying to exploit.

Finally, this exploit requires the target device to have AWDL enabled, and many won’t. But you can use Bluetooth Low Energy advertisements to trick the target device into believing an Airdrop is coming in from a trusted contact. Once the device enables AWDL to verify the request, the attack can proceed. [Ian] reported his findings to Apple way back in 2019, and this vulnerability was patched in March of 2020.

Via Ars Technica.
Continue reading “This Week In Security: IOS Wifi Incantations, Ghosts, And Bad Regex” →

DIY Regular Expressions

October 29, 2020 by Al Williams 12 Comments

In the Star Wars universe, not everyone uses a lightsaber, and those who do wield them had to build them themselves. There’s something to be said about that strategy. Building a car or a radio is a great way to learn how those things work. That’s what [Low Level JavaScript] points out about regular expressions. Sure, a lot of people think they are scary. So why not write your own regular expression parser and engine? Get that under your belt and you’ll probably never fear another regular expression.

Of course, most of us probably won’t do it ourselves, but you can still watch the process in the video below. The code is surprisingly short, but don’t expect all the bells and whistles you might find in Python or even Perl.

Continue reading “DIY Regular Expressions” →