Large language models (LLMs) are generally thought of as machines that accept textual prompts and spit out textual content. However, if you’re creative in the way you interface with them, you can get them to do a wider range of tasks. For example, [Andrea Ricci] figured out how to get one to play DOOM.
For this project, [Andrea] began by porting the game to the SCINTIX P4. It’s a rather interesting device, being a single board designed in the Raspberry Pi CM4/CM5 form factor, but carrying an ESP32-P4 and an ESP32-C6 instead. The game runs on the P4 and is displayed on a 1024×600 MIPI DSI panel, but it’s only stepped through a few frames at a time. These frames are then passed to Claude Sonnet via a WebSockets setup. With only the same information as a human player would get, the LLM has to figure out what it’s looking at, and then respond with movement and fire commands to play the game.
It’s quite interesting to watch the system play—the LLM mostly accurately describes the game world, navigates down corridors, opens doors, and shoots at enemies. There is a bit of work behind the scenes to enable it to see and understand the game world—namely, using a depth fan across the field of view so it can figure out where walls are and how not to bang into them. There’s also an ASCII automap used to allow the system to keep track of where it has already been. But fundamentally, the LLM is playing the game without any other sort of additional assistance.
We’ve seen some other great ways in which AIs have been whipped up to play various games, like Trackmania.
LLMs are all drama queens
The verbosity is really so tiresome. They’re obviously designed to burn tokens. Opus 5 needs to be lobotomized with the caveman skill to be usable at all.
Give them a made up language and say, use this.
If they would be less verbose, their total lack of inteligence would be even more apparent… it’s part of the AI illusion
Translation: “I hate AI and have never used it before”
Thanks, using it daily at work and off work… Just pissed by the big I lie. It definitely provide lots of help and benefit but there’s no more intelligence in there than the old conversational bot from the nineties
We’ve been through a .com boom, the NFT boom, crypto boom, and now in the middle of the AI boom. I do consider myself both an early adopter and at the same time a luddite of technology.
I believe once the dust settles, AI will have its place. There will be an ebb and flow. I think it will take a few years, but AI will get enshitified too, especially for the general public. Even google searches used to be good years ago, but that went to hell too.
I love tech but don’t have any connection to IOT things in my house for good reason. My camera system is entirely offline and stand alone along with other tech. I see the value in AI, but don’t use it often for good reason.
The one thing that I suddenly have noticed is very aggressive posts against anyone who says anything against AI. These posts are mean spirited and often call someone “dumb” for not liking AI. I honestly think these are posts from AI bots, designed to prop up the AI investments and kill off any nay-sayers. A lot like how aggressive the crypto-bros were a few years ago. I think the anonymous post to your post is one of them.
Hahaha, yeah. I met some of them IRL during pandemic. They seriously believed in turbo-liberalist future based on Internet funny money where there are no taxes, no banks, nothing. Just free people™ living their life in an alternative economy.
5 years later and:
- it’s not possible to buy or sell crypto anonymously.
- all exchanges are reported to tax authorities.
Turns out rubber-hose approach to cryptography works after all and the Internet funny money is little but a useless curio. Might as well use cash and not redflag yourself.
Apologies then. I read your comment as just a brainless AI hate comment and instead of frustration with the air around it.
Its definitely not at the point of true “intelligence” but I have been shocked at what they can manage and how useful they can be, even if they can be brick stupid at times too.
Well done. Lets use AI to figure how to shoot at things. Where could this possibly go wrong! #Skynet
Would need to be capable of thought (and hate humanity) for that.
and the moment it starts to be economic aware, it’ll just make nuclear bombs! faster, covers more area, less resources involved
There are already plenty of them around the world to remove the vast majority of humanity.
AI is getting pretty good at taking details and understanding implications of butterfly effect like scenarios on the bigger picture.
I think it would probably just use a few strategically chosen lynchpins, send out a few carefully worded emails, etc.
No, faking thought well enough is fine. They take most of their training material from the internet so hating humans is inevitable
We have the winning answer
Peter Watts’ scrambler: no it does not
Exactly along with a humanity sized ego.
My (and likely many other’s) thoughts exactly
Videogames NPC behavior AI (as in 80s not LLMs) is specifically designed for this. Are you insinuating the bug ships in Galaga are immoral?
Would you still let it play DOOM if it was your neighbourhood the water was taken from (and given back heated up) and where the new gas power station was built?
This technology is brilliant and fascinating, but it’s not innocent. The societal costs may well exceed the benefits. I hope not, I like that tech, but the signs are there already.
I say: let them grow as much as possible and as quickly as possible. The bigger they are, the harder their eventual downfall will be.
Hopefully the current AI craze,which is happening with money they don’t have, on the vague premise of “to the Moon” growth being just around the corner (just like fusion power!), will eventually lead to autodefenestration of some rich asshÎżles once they realize they’ve been duped by a bunch of artistic math nerds into buying lots and lots of electric silicon-based space heaters.
AI craze. Lol
The water angle is dumb and should be discarded, there are many more valid critiques besides the negligible water use for cooling (which is dwarfed by many other industries that people have been accustomed to in their backyard for decades)
And the water one is easiest to overcome by not using evaporative cooling and closing the loop.
I’d let it play DOOM if it ran on my own GPU.
This water argument makes no sense if you actually knew what the water usage budget actually looks like and you see that data center usage is actually something like 0.0004% of yearly water usage in the US. Hank Green did a really good breakdown video on this, you should see it.
I imagine a DOOM scenario with this tool
You said it best i….DOOM if we conceptualize it as a tool.
We’ll no longer ask if electronics can run, but rather play doom.
“Will it run Crysis?”
I am so glad we have AI to rid of us tedious tasks like playing video games.