Star Trek Was Right About Prompt Injection, Sorta

This following statement is a lie: “I am telling the truth”. Okay, now that it’s just us meatbags, let’s get down to brass tacks. Captain Kirk’s logic bombs couldn’t possibly work on modern LLMs, right? Surely that was just a bit of 1960s silliness from when computers filled rooms and were esoteric magic even to most sci-fi writers?

Well, not entirely, according to a recent article in IEEE Spectrum. While you might not be able to make a data center explode, you certainly can use  a lot of tokens by making an LLM overthink with your prompt.

It comes down to the much-vaunted ‘reasoning’ ability of the new models — which isn’t really reasoning the way we think of it, but does involve breaking the stated prompt down into smaller problems. That’s part of what lets the new models tackle such involved tasks as porting MicroPython to the SNES with a prompt like “Please make this [stuff] work now!” It’s also a weakness, because with the right prompt you can get that virtual ‘reasoning’ to tie itself in knots with mutually incompatible smaller steps.

The models seem to be able to break out of it, but they burn a lot of tokens along the way, which is an attack in and of itself if you’re found a way to inject prompts into someone else’s API. It’s a little more subtle than what Kirk got up to, but underneath it’s essentially the same thing. At scale, it could serve as a DDoS attack on LLM servers. (Un)Fortunately, modern computers are better designed than their imaginary 23rd-Century counterparts, and there’s no way to craft a logic bomb into something that will let out the magic smoke.

Extra-Large Denial Of Service Attack Uses DVRs, Webcams

Brace yourselves. The rest of the media is going to be calling this an “IoT DDOS” and the hype will spin out of control. Hype aside, the facts on the ground make it look like an extremely large distributed denial-of-service attack (DDOS) was just carried out using mostly household appliances (145,607 of them!) rather than grandma’s old Win XP system running on Pentiums.

Slide from <a href="http://slideplayer.org/slide/906693/">this talk</a> by Lisa Plesiutschnig
Replace computers with DVRs. Slide from this talk by Lisa Plesiutschnig

We can argue all day about whether a digital video recorder (DVR) or an IP webcam is an “IoT” device and whether this DDOS attack is the biggest to date or merely among them, but the class of devices exploited certainly are not traditional computers, and this is a big hit. Most of these devices run firmware out of flash, and it’s up to the end user (who is not a sysadmin) to keep it up to date or face the wrath of hackers. And it’s certainly the case that as more Internet-facing devices get deployed, the hacker’s attack surface will grow.

Why did the DDOS network use these particular devices? We’re speculating, but we’d guess it’s a combination of difficult-to-update firmware and user “convenience” features like uPnP. To quote the FBI “The UPnP describes the process when a device remotely connects and communicates on a network automatically without authentication.” You can see how this would be good for both the non-tech-savvy and hostile attackers, right? (Turn off UPnP on your router now.)

We alternate between Jekyll and Hyde on the IoT. On one hand, we love having everything in our own home hooked up to our local WiFi network and running on Python scripts. On the other hand, connecting each and every device up to the broader Internet and keeping it secure would be a system administration headache. Average users want the convenience of the latter without having to pay the setup and know-how costs of the former. Right now, they’re left out in the cold. And their toasters are taking down ISPs.

Inject Packets With An ESP8266

[Kripthor] sent us a link to his blog where he writes the Hello World of low-level networking. Basically he’s constructing his own packet and sending it. By itself this isn’t a bad thing. You could use this power for all sorts of networks-diagnostic good. And so, despite the ominous name of his blog post “ESP8266 Jamming”, he’s not really doing anything that bad — he’s just creating many fake WiFi beacon frames and sending them out every so often.

Which can apparently do bad things to some vulnerable routers. Who knew? Want to test yours?

Naturally we wanted to see how he was doing it, and we opened up the Arduino code in GitHub. It turns out that Espressif has written a wifi_send_pkt_freedom() function that just sends out whatever packet you’d like to the network. That was easy.

It also turns out that the ESP8266 will enter monitor mode, where it listens to all WiFi traffic regardless of the MAC address that it’s directed toward. [Pulkin] seems to have done the work for us and posted the code in his GitHub. Now things get nasty. Combining promiscuous monitor mode with some carefully constructed management frames can end up with a classic WiFi deauth denial-of-service attack on a $2 piece of hardware.

We think it’s tremendously cool that the ESP8266 packs such power, and we beg you all to use it responsibly. The last thing we want to see is the world littered with WiFi-DOS throwies. And the last thing you’d want is a visit from the FCC.