This following statement is a lie: “I am telling the truth”. Okay, now that it’s just us meatbags, let’s get down to brass tacks. Captain Kirk’s logic bombs couldn’t possibly work on modern LLMs, right? Surely that was just a bit of 1960s silliness from when computers filled rooms and were esoteric magic even to most sci-fi writers?
Well, not entirely, according to a recent article in IEEE Spectrum. While you might not be able to make a data center explode, you certainly can use a lot of tokens by making an LLM overthink with your prompt.
It comes down to the much-vaunted ‘reasoning’ ability of the new models — which isn’t really reasoning the way we think of it, but does involve breaking the stated prompt down into smaller problems. That’s part of what lets the new models tackle such involved tasks as porting MicroPython to the SNES with a prompt like “Please make this [stuff] work now!” It’s also a weakness, because with the right prompt you can get that virtual ‘reasoning’ to tie itself in knots with mutually incompatible smaller steps.
The models seem to be able to break out of it, but they burn a lot of tokens along the way, which is an attack in and of itself if you’re found a way to inject prompts into someone else’s API. It’s a little more subtle than what Kirk got up to, but underneath it’s essentially the same thing. At scale, it could serve as a DDoS attack on LLM servers. (Un)Fortunately, modern computers are better designed than their imaginary 23rd-Century counterparts, and there’s no way to craft a logic bomb into something that will let out the magic smoke.
