Giving machines the ability to communicate nonverbally has real value, and [Drew Smith] clearly thinks your robot deserves better than an emoji. He shared a very interesting approach with his project Kindalive.
Kindalive is a simulated dot-matrix robot face that responds believably to input text, modeling and expressing both short-term and long-term moods. It’s pure Python and modular enough to invite using it elsewhere, but that’s not the really interesting part.
What sets [Drew]’s project apart is the way he models eight key neurochemicals (including dopamine and cortisol) as the foundation from which to derive emotional states. That’s an approach we certainly haven’t seen before.
Conventional sentiment analysis uses a large language model (LLM) to apply discrete labels to communication, but Kindalive doesn’t do that. It even goes so far as to model the decay and interplay between its simulated neurochemicals to derive emotional states on the fly. It’s more fluid and organic, and reflects both short-term and long-term mood changes.
Physical representation of the emotional mix is done by altering twelve key facial movements (brow raise, lip corner pull, mouth open, and others of that nature) known as the Facial Action Coding System (FACS). These twelve elements combine to express emotion nonverbally with facial expressions. It’s what drives the simulated dot-matrix robot face seen in the image above, and could easily be used to drive a real LED matrix, or servos on an animatronic face.
Much of communication is nonverbal. Humans even weigh nonverbal higher when there’s a mismatch between the content of verbal and nonverbal communication. So, there’s clear value in having robots able to express themselves as such.
Importantly, a realistic and human-like face is entirely unnecessary — something every Star Wars fan already knows. Cartoon eyes and basic sounds are enough to make robots easier to relate to and work with, even if blinking is also important but hard to get just right.
This is how I envisioned llm usage long ago, however instead of going Wolframs route and inputting professionally tailored scientific and mathematical data we got models trained on torrented commercial media
Idk… I’m just gonna speak for myself, but I’d rather have a dumb LLM trained on bad Reddit and Stack Exchange posts than an AI made by extremely competent scientists programmed to accurately simulate adrenal response
Just don’t make your bots deploy any biochemistry out of the uncanny valley.
Yep. Early on, I had a human face and it was… uh, freaky.
It seems like the next step would be for the state of the neurochemical model to influence the interpretation of input (since the same words can be taken very differently based on ones state of mind when they’re read or heard) and, potentially, even letting the model neurochemical state drive unsupervised learning.
It strikes me that the simplistic cost function + gradient descent generally used to train neural networks becomes a limiting factor once you grow beyond single purpose modules (otherwise you’d need one fitness metric to rule them all which strikes me as absurd).
This sort of approach could be a first step towards more nuanced autonomous learning, e.g. if the model was “curious” about something it would give itself reward for success filling out its understanding
in that area but the better its model predicted things in some area the less of a curiosity-based reward boost it’d get.
It seems like there’d be a lot needed on top of these neurochemical models but this seems like an excellent building block for autonomous robots and other situations where you want an underpinning for intrinsic motivation (in addition to whatever relatability benefits it may bring).
If I understand your first paragraph correctly, it’s already implemented. The robot should bring its emotional state to each response. So, concern can grow, emotions can pivot. When I was testing it, I found myself trying to persuade more than I do with a typical LLM for this reason.
Thanks for the encouraging words!
I wonder if in China they are using Chinese Five Element Theory(Wu Xing: Wood, fire, earth, metal(gold), and water.) It’s more commonly known outside China in Traditional Chinese Medicine, but it is applied to just about everything; the lunar calendar, seasons, sense organs, martial arts, tai chi and qi gong, massage, alchemy, emotional health, and most apropos to this article; personality. A Quinary system might be a good approach for ‘bots interacting with humans. Maybe it’s appeared in fiction already (Manga in particular?).
If China starts “winning” the AI race, this would be something to look out for.
OTOH “eight key neurochemicals” -that’s fairly arbitrary, even if it’s been assigned “sciency” names from brain chemistry. Let’s not make more of it than it is. Because, a lot of the lay audience is already succumbing to “AI psychosis”. Actually one humorous thing that came to mind is maybe Musk will develop digital ketamine for his AI’s to enjoy K-holes -it’s an NMDA receptor antagonist that stimulates glutamate production! lol
Fascinating. I’d love someone to fork this and approach it from the Chinese angle.
There was a version of this where you could “inject” chemicals but it felt very gross in a way. I used to work in neuroanatomy research, and it felt unscientific.
It’s probably just a matter of time before someone forks it and adds doping through pharmaceuticals… wow.
Most people’s understanding of “neurochemistry” is entirely pop-psychology. You’d be better off honestly talking about your emotions in plain terms like a normal person, but people heard these science words so they use the science words with basically zero understanding of them. “Dopamine” “serotonin” etc, just say pixie farts, it’s the exact same thing to most of you. “You’re spiking my cortisol..” Just say I’m annoying, okay? I can handle it.
Always creeps me out when people are made to learn entirely new vocabularies to discuss their emotional states which we have plenty of words to describe. It’s cult behavior. Got my orgones all in a twist
An Electric Dreams character.
I’ve watched a lot of robot-related films and series over the years but haven’t seen that. Thanks for the tip.
So the llms are interpreting the paragraphs and producing chemical signals that it thinks matches the emotion of the text?
How do you ensure the llm produces output that the neural engine can you? From what I understand of them they don’t always produce accurate info.
Neat project.
The LLM takes its current state and the input text and adjusts some of its chemical levels, then the emotion and face change, and a response is returned. It’s not meant to be a mirror, just a fellow entity that can be affected for better or worse by you.
Does that help? I’m still working on the story.