In the original Animal Crossing from 2001, players are able to interact with a huge cast of quirky characters, all with different interests and personalities. But after you’ve played the game for awhile, the scripted interactions can become a bit monotonous. Seeing an opportunity to improve the experience, [josh] decided to put a Large Language Model (LLM) in charge of these interactions. Now when the player chats with other characters in the game, the dialogue is a lot more engaging, relevant, and sometimes just plain funny.

How does one go about hooking a modern LLM into a 24-year-old game built for an entirely offline console? [josh]’s clever approach required a lot of poking about, and did a good job of leveraging some of the game’s built-in features for a seamless result.

The game runs on a GameCube emulator, and the first thing needed is a way to allow the game and an external process  to communicate with each other. To do this, [josh] uses a modding technique called Inter-Process Communication (IPC) via shared memory. This essentially defines a range of otherwise unused memory as a mailbox that both the game state and an external process (like a Python script) can access.

[josh] then nailed down the exact memory locations involved in dialogue. This was a painstaking process that required a lot of memory scanning, but eventually [josh] found where the game stores the active speaker and the active dialogue text when the player speaks to a villager. That wasn’t all, though. The dialogue isn’t just plain ASCII, it contains proprietary control codes that sprinkle things like sounds, colors, and speaker emotes into conversations.

The system therefore watches for dialogue, and when a conversation is detected, the “Writer” LLM — furnished with all necessary details via the shared memory mailbox — is asked to create relevant dialogue for the character in question. A second “Director” LLM takes care of adding colors, facial expressions, and things of that nature via control codes.

[josh] even added a small bit of shared “gossip” memory among all villagers which keeps track of who said what to who, and how they felt about it. This perhaps unsurprisingly results in a lot of villagers grumbling about just how much currency flows directly to Tom Nook, the raccoon proprietor of the local store.

A very clever detail pointed out by [Simon Willison] is how [josh] deals with the problem of the game expecting dialogue to be immediately available at the given memory location. After all, LLMs don’t work instantly. Turns out [josh]’s code makes clever use of a built-in dialogue control code that creates a short pause. Whenever a dialogue screen opens, a few short pauses ensure that the LLM’s work is done in time.

If Animal Crossing isn’t retro enough, or you prefer your LLMs to be a little more excitable, AI commentary for Pong is totally a thing.