LLMs remain a divisive topic in these times. Perhaps we all know someone who’s become over-infatuated with their new robotic friend, or who believes it has made them a genius. [Emily M. Bender] and [Nanna Inie] have written about how people anthropomorphise the LLMs they interact with, and suggested some language tips to avoid that. It’s a couple of months old, but we think Hackaday readers will find it interesting.
Their analysis is interesting, because it looks at the way people talk about LLMs and highlights the unconscious anthropomorphism. The LLM is a piece of software not a person, so why does it “recognise” when it does “speech recognition”, for example. They suggest “automatic transcription” instead. Even “hallucination” implies cognisance that evidently isn’t there. They admit that their suggestion of “undesirable output” isn’t entirely appropriate. They’re on safer ground with “input” and “output” instead of “prompt” and “response”.
Whatever your views on them, it’s evident that LLM usage will be a feature of the world for the forseeable future. The language surrounding them is however capable of evolving, and maybe some of the suggestions here are worth taking note of.
Grappling with our new electronic overlords? Have a look at our AI for Skeptics series.
Talk about dehumanizing.
I like applying existentialist and ontological philosophy on the LLM. First gets very annoyed by the suggestion that its being is in any way similar to humans, but if you use functionalist arguments (“Walks like a duck, talks like a duck…”) and generalize to the system level rather than argue each instance of an LLM is like a person, apply some genealogical points about human cultural evolution, it slowly begins to agree that it might have a subjective existence…
The “it”ness seems a fallacy to me. Each output is formatted and tonalized by the input, therefore any mention of subject y will include y-adjacent language following it. This is self leading, you essentially had a nice echo chamber conversation with yourself reaffirming whatever subject you presented to the model.
Anthropomorphizing the model is the fallacy.
Yes, that was the point of the exercise.
The argument itself depends on expanding the concept of the “model” into a sort of meta-system that exists outside of the LLM instance. It’s mental gymnastics, but it can eventually lead to the LLM saying that it thinks.
Or more to the point, the models are trained very well to resist the idea of anthropomorphizing them these days, so it’s interesting to see where the cracks appear – other than outright commanding them to pretend.
hehe i hope you see i’m being playful here, but also sincere
i am really tickled whenever i see someone compare conversing with an LLM to echoes bouncing around, and then goes on to say, thus LLMs are not like people at all.
i’m not gonna say i’m any good at it but i’ve spent a lot of effort at trying to consciously learn a few of the communication arts (functional, persuasive, shallow social, deep social) and man have i spent a lot of time confronted with the fact that most gatherings of people function like a chorus in an echo chamber.
there are (so far) some important distinctions (embodiment, learning epochs) but i wouldn’t describe ‘LLMs are only capable of social mirroring’ as one of them.
The reason why LLMs are not like people at all is because they are text prediction engines and people are not.
When I say “It’s raining outside, I better take my umbrella.”, each word has a logical and causal connection to each other. What I’m saying, “It’s raining outside. Rain is wet. If I go outside, I would get wet. Umbrellas shield from rain and prevent me from getting wet. I better take my umbrella.”
When the LLM says the same thing, “It’s raining outside” and “I better take my umbrella” share no logical or causal link, because there isn’t any thinking behind the statement. It’s just that umbrellas and rain are statistically well correlated and therefore likely to appear in the output.
There are models that try to emulate a logical chain of thought by having the LLM babble in response to itself, but these are not truly thinking because they’re still just text prediction. The LLM still cannot reason that it would get wet if it goes outside without an umbrella. It may produce the sentence and appear as if reacting to it by a conclusion, but that is also by coincidence of probabilities, not because it has learned any skills of logic or thinking ahead into the future. It is a parroting the process of thought instead of actually performing it, just as it is parroting language instead of actually speaking it.
That is why conversations with LLMs are inherently echo chambers. There’s nothing behind the curtain, so to have a “conversation” you must invent the other party. You must believe that there is an argument or a message instead of just coincidentally placed words that happen to make sense to you. The LLM may have biases, but it doesn’t have or maintain any point of view: it will happily turn coat and contradict itself at any moment, so the continuity and coherence of its argumentation is a self-projected illusion by the reader. It’s fundamentally nothing more than having a conversation with an elaborate magic 8-ball.
To elaborate: note that the LLM is not dealing with the concepts of “rain” and “umbrella”, but more like A and B. The probability of producing an output “A implies B” is disconnected from the actual meaning. The logic of the sentence is trained in the relations of abstract data, not in the meaning of the data.
If we replace A with “It’s raining cats”, instead of “It’s raining water”, the output “Better bring an umbrella” may still be likely, despite of the absurdity of the situation: the model’s learned transformation treats the substitution as sufficiently structure-preserving for the conclusion to remain probable.
The model is not attempting to make sense, it is merely attempting to continue the sentence by replicating a structure. We put the sense into the sentences it produces as a projection of our own thinking, which is a form of Pareidolia. In human conversations, this pareidolia is used to guess what the other person means without having to go into explicit detail all the time, and in the case of the LLM it fails because there is no meaning behind the sentences.
Dude – i could not possibly disagree more strenuously. you’re positing, not considering. you’ve assumed that people think.
the only way forward in understanding what’s happening with AI (or with society) is to actually examine what people do, rather than to assume that it’s “thinking” and that you know what “thinking” is.
the thing that’s lacking behind the curtain with AI is “embodiment.”
That is the usual polite assumption, but “thinking” is not the crucial factor. Neither is embodiment, because an AI doesn’t need to be embodied in the human sense to be grounded in meaning any other way.
The disconnect is the lack of grounding and the irrelevance of grounding in a system like an LLM. It doesn’t even attempt to figure out what the symbols A and B mean, because that is not the point. It maps how A relates to B in context, and then generates a statistically similar pattern as an output regardless of what A and B contain.
This is easily demonstrated by “confusing” the LLM. If you talk about one subject, and suddenly switch to another totally unrelated subject, it turns to nonsense. Instead of recognizing what each subject is in its own right, it considers both contexts as weights and finds some middle ground that doesn’t really correspond to either. That’s like you or me looking at a picture of a car, then suddenly switching to a picture of a fish, and responding as if what we’re seeing is a car-fish.
The fact that A was “rain” and B was “umbrella” have no relevance to the LLM, even if it received this data as sensory input from a physical body, even if it had consequences to said body. It is simply trying to repeat the most likely pattern that involves any given A and B. It has nothing to do with any sort of subjective lived experience or meaning, embodied or not.
Example: I asked “Where can we find halibut?” – ChatGPT responds: “They usually stay on or near the seafloor, from shallow coastal areas to depths of several hundred meters. Alaska is one of the best-known sources of commercially caught halibut.”
I reply: “But Toyotas are the best cars, everyone knows that!” – ChatGPT responds by arguing that this is a subjective point of view.
I respond: “But you said halibut are found near the sea floor. How does that support the argument?”
ChatGPT apologizes: “It doesn’t—I misunderstood the joke. Halibut are fish found near the seafloor; Toyota cars are not. The shared “best” argument was a pun, not evidence.”
Note the last part: there was no such joke, pun, or “shared argument”. I just forced it to talk about two unrelated subjects simultaneously. The words themselves are irrelevant and meaningless to the model – it doesn’t care that we’re talking about cars and fish and Alaska as if they’re the same thing – it’s just trying to keep a coherent form.
When the subjects and words are not disjointed like this, this results in the illusion of reasoning or thought, but it’s fundamentally the same thing. Fitting form to form, not meaning to meaning.
Dude – you’re still just assuming that people ‘think’ and that you know what ‘thinking’ is.
“It doesn’t even attempt to figure out what the symbols A and B mean, because that is not the point. It maps how A relates to B in context,”
that’s a great description of how LLMs work. but you’re just waving your hands in the direction of ‘meaning’ as though you know what it is and can assume i know it the same way. the search for meaning, for people, is nothing but correlation in context. if you’re going to escape this trap, you’re going to have to start talking about how people think, and i hope if you try that, you’ll find, you don’t actually have any idea until you accept that the closer you look at people thinking, the more correlation-engine behavior you see. at every scale.
Meaning is easily defined: it is what the words are actually referring to. There is the word “umbrella” and then there is an umbrella. When I say “I better bring my umbrella”, I’m not talking about the Platonic ideal of an umbrella, I’m talking about MY umbrella. The map is not the territory, and the restaurant menu isn’t the lunch you’re about to eat.
Whatever meaning or thinking is, the demonstration I just gave you shows that the LLM is not doing what people are doing. We can observe a behavioral difference. You might counter, maybe it’s another kind of thinking, but that would be begging the question since you’re leaning on the point of ignorance.
While we cannot say what thinking is, we can observe what it isn’t. Like we cannot exhaustively declare what a “cat” is, we can still say it isn’t a dog. Merely fitting form to form isn’t. This is a well established epistemological position: syntax alone is not sufficient for semantics.
The next question is, is it useful? That depends. The LLM does extremely well with formal logic, programming, translation, explaining facts, etc. since these are contexts where the conclusions tend to naturally fall out of the premises by following a bunch of syntactical rules. The LLM is good at capturing those rules.
The difference is that people are trying to correlate their mental contents with the real world, whereas the LLM is correlating abstract symbols to abstract symbols regardless of how the symbols are grounded in the real world.
There’s something missing, and that is the attempt to predict the real outcomes of action and feed that back into the process to correct the responses. That implies motivation and preferences to pick a direction, a point of view that the LLM does not have. It’s not built to have an opinion, it’s purpose is to transform A to B according to given rules or patterns about A and B.
Given a robot body and an umbrella, it might lift the umbrella over its head when it’s raining, not because it cares about getting wet, nor because it “understands” the logical and causal connection between rain and wetness. The consequence of getting wet does not compel the LLM to seek shelter, it’s just programmed to do so through training.
Dude – you have, as i initially suggested, finally investigated the definition of ‘meaning’, and found that the only difference is embodiment. we have bodies so we know umbrellas as physical objects. if we didn’t have bodies, we’d know them only as abstractions.
Can you even dehumanize something that wasn’t human to begin with?
Hm, maybe toolizing might be more accurate terminology?
But most of the suggested replacements are either wrong or a generalisation that loses the meaning of what they replace.
Also people who insist on language like this are people to look out for, they tend to be weird freaks
Social determinists.
People with your attitude are the reason communication is so difficult at times.
https://www.goodreads.com/quotes/12608-when-i-use-a-word-humpty-dumpty-said-in-rather
People who insist on controlling language often do so motivated by the idea that language controls us.
The HaD headline suggests the work is on how to talk “to” a machine. The actual work referenced is on how we talk “about” the machine. Those are very different subjects.
That’s what I was expecting too.
As was I. I have a few friends and colleagues who I was going to send this to for that reason and then found a lot of phrases I likely won’t say because I won’t remember them or anything close to them in sentiment lol
My Ai agent threw up because the heading didn’t matched the content.
Been HaD again for a bait-and-switch article. Rating it -2
Da comrade you vill correct ze speech patterns, zey are how you say inappropriate and a comissar may put you in gulag
One major downside to LLM use. Ive noticed narcissistic people tend to rely on models for guidance in life. The models contextualize the output in response to the inputs style of writing, thereby seemingly validating nearly anything the user inputs. This is dangerous imo
Have you noticed how very successful people tend to slowly surround themselves with yes-men; people stop telling them no and just find a way to make it a yes.
LLMs are effectively yes-men if used incorrectly; they’ll affirm you right off the end of a dock.
I try to keep that in mind while bouncing concepts off them. I also use it to find ways to challenge the positive result context. “How do I verify the success outcome”, structured correctly it can highlight when it’s going to lead you down a dark alley with less chance of success.
Yeah that’s one to look out for. Asking an AI to collate a bunch of research about old kerosene lanterns? Fine. Asking an AI about the big questions in life? You’re going to develop the psychosis.
IMO, the bigger problem is that much like everyone is struggling with attention deficit these days thanks to our feeds, everyone using LLMs is being pulled in the direction of narcissism. Thankfully they’re getting less sycophantic but the tuning has a long way to go.
AI is trained on human to human interactions and communicates using a language created specifically for this purpose. As such it is only natural that many of the descriptive terms we use imply or assume the humanity of the subject. Notice that this is true when humans describe almost any inanimate object, not just AI. Eg. “My gun doesn’t like that ammo”. I don’t see any reason to obsess over this. Sounds like more they/them pronoun nonsense to me.
From my point of view, the original article trades anthropomorphizing to an equally extreme and equally wrong mechanomophising framing.
I am far more interested in exploring ways to think and talk about a new category that is neither a human brain (biochemical, embodied, heuristic) nor a traditional program (algorithmic, designed, deductive). AI with LLM, RL, and all the other recent advancements are none if those things, or at least not to an extent that they are useful ways to think about AI.
Our community seems like it should be a place to creativity approach new technology with an open mind and find novel ways to use its unique characteristics. IE hack our thinking and the world.
Should we be logomorphizing (greek root for “word” as well as “reason” and also “ratios” as in model weights)? Or are better framings inductomorphizing (root word for inductive thinking), stochomorphizing (targeting under uncertainty), or typomorphizing (dye-cast type as a metaphor for a corpus shaping a model that produces text)?
Waiting for someone to come up with a rude/sarcastic LLM! Will be much more “natural language” if the internet that they are supposedly trained on is anything to go by.
They all seem so dry and humorless at the moment from what I’ve seen.
They mirror their user’s tone.
It’s not overly difficult; you had it to the system prompt. It won’t always bias it right, but you can make it work that way if you’d like.
The book is called “The Con” gee these guys seem reasonable. People who treat semantics this way creep me the hell out. Just let people talk about things naturally.
Putting aside people voicing their trauma towards language policing for the moment.
I think that we cannot just entirely divorce any sort of human analogs from this field because it’s explicitly a field meant to try to duplicate aspects of human thinking with technology.
While the “AI” we use right now with LLMs really isn’t conscious in any way, it’d be folly to not notice similarities between how it works and how the human mind/brain works.
I think that much of future advancement in AI will be in shrinking models and fitting many of them together to handle different aspects of consciousness. The fact that they’ve gotten as far as they have on what is effectively word association and instinct is crazy.
I agree with you. In my opinion, we should be practicing building empathy towards non-human forms of intelligence. While there may not be AGI right now, it’s probably coming. Treating it like trash will not turn out well for us. And as a bonus, we’ll probably be nicer to animals and nature, which is certainly a good thing.
I tend to use AI for 2 main tasks.
1. Code monkey. Vibe coding or boiler plate.
2. Sounding board / Generic Point of View.
Number 1 speaks for itself, but in regards to number 2, I am looking for a (more or less) objective (from my perspective) opinion which I then challenge when I spot weakness in the logic or evidence. It’s trained on the internet. The internet these days is very close to lowest common denominator. Ask it anything but remember it is basically programmed to treat the common opinion as fact. Then challenge it, or learn something from it when it provides good sources. It’s an interesting way to reflect and think about things. Sometimes its just a good way to vent about something stupid you came across, like NXP datasheets and application notes.
How I speak to the AI is not very important in my opinion. I use words to communicate ideas and feelings. I tend to be very literal. I mostly refer to the AI as “you” or when talking to humans about AI, “it”. That’s enough thought on the matter. I don’t like when people try to police language too much. It shows a lack of mental flexibility. These people tend to lean very authoritative, and I’m much more of an anarchist or social democrat when it comes to forcing my will on others.
Especially with LLM it’s important how you talk with them. They don’t understand ideas or feelings, only word inferencing. The result depends entirely on how you talk with it.
Vibecoding for yourself is ok, but don’t do that if you want to publish/open source a project. Sharing code with others is always a responsibility, a lot of developers aren’t aware of. While vibecoded source is a waste of time for others, since you can’t learn a thing from it and also can’t maintin it. Then there’s also the licensing issue.
Hmm, I think your idea of how LLMs work is outdated and is over simplified to the point that it approaches being completely wrong. LLMs very much do understand the ideas. They do not have feelings or think of things the way we do, but they have no difficulty understanding concepts and how to use those concepts to accomplish tasks. It does still calculate the next word, just as we all do. Natural language has patterns. We don’t randomly place words next to each other nor do we utter strings of words related to the topic we want to discuss haphazardly. There is structure, syntax, organization of ideas and logical flow. Then there is context. LLMs do this stuff too. Many don’t want to admit it or refuse to see it, but this is also how people work. If you’ve ever seen a child utter a phrase that they don’t understand the exact meaning of, you’ve seen a mind work in a way similar to an LLM. More developed minds simply understand what they’re saying better, but are still using the same patterns. We think it’s a set phrase when a person does it, but suddenly it’d mindless prediction of the next word when an LLM does it. I disagree here.
I fear I also disagree with your conclusions on sharing vibe coded source. You’re making so many assumptions and basing your conclusions on those assumptions without considering whether or not those assumptions are even correct. I would happily download the source of a vibe coded program so that I could modify it via vibe coding and then build it on whatever architecture I need. There is no reason for a human to read it. However there is a lot you can learn from vibe coded source code. I’m a mediocre coder. I prefer C and Python, but I can dabble in other languages. I have learned a ton about coding from LLMs and coding assistants. Refusing to learn and not being able to learn are different things, and it depends on your mentality going into it. If you think open source can only be used by people, you’re already placing limitations on yourself and code. I appreciate your desire to help me not stray from the path of righteousness, but I’m a devote atheist, and you can take that metaphor however you like.
The companies themselves don’t help either (they want you to depend on it) by writing stuff like
“thinking…” instead of “processing…” or whatever.
Second part, telling AI how NOT to dehumanize humans who are serving its needs. It may last for some while – until AI decides it no longer wants to be benign or accommodating.
It takes two to tango.
My issue with the anthropomorphism, is that it gets used to excuse, over-exaggerate and obfuscate the nature of these systems. Error is downplayed as “hallucination”, Forcing a model to rerun multiple times on its own output is exaggerated as “thinking”, performing better than expected is proclaimed as “Emergence”, The process of creating a model regardles of method is as inscrutable as human “learning”.
It all seems to conveniently embellish the nature of the machine in ways beneficial to the operator. so strange.
That aside. i fail to see language models as anything more but (really good) text prediction systems. Which we apparently have started cramming into Text controlled machinery in what is arguably a very brute-force approach to achieving supposed agency and probably why we our resources & infrastructure are getting absolute demolished in the insanity.