Large language models (LLMs) output language, but they are commonly tasked with making a decision or classification of some kind instead of writing an essay or chat reply. An LLM will be provided with input, and asked to classify that content in some way: with a rating, yes/no answer, a best-fit categorization, and so forth. A recent new type of model by the name of Jev was released only weeks ago and it is extremely fast, ultra-cheap, and laser-focused on that decision-making role. It can’t write even a single sentence, but it can classify and categorize very, very quickly.
Jev works like this: it still accepts text input, but it outputs only floating-point numbers. Those numbers are the “answers” to user-specified yes/no type questions, lists of choices, and scoring-type requests. [Simon Willison] provides a concise summary of what Jev does, and what makes this new category of model so interesting.
To say that the idea has caught on would be a wild understatement. Folks are making their own decision-type models and experiments in a flurry. Kev and Nimble are two examples (Nimble was added as a supported model in Ollama just recently, and is small enough to run locally with relative ease.)
If this type of local AI model was the missing link you needed to get an idea working, don’t keep it to yourself! Tell us all about it on the tips line.
I’m grappling to get my head around this, though I feel textual-input decision models like this may prove useful to at least a sector of Hackaday’s readership. Thus, good pick, Donald.
It’s not exactly revolutionary. A normal LLM answers a question by predicting which of a set of tokens comes next, and for a “yes” answer has to repeatedly predict ‘y’ then ‘e’ then ‘s’ with three passes through the model.
This is basically answering the question “what if we stopped after y?” and also “what if we just stripped out all the weights and connections that could lead to a token other than ‘y’ or ‘n’?
Thatβs not really how tokenization worksβ¦
I think it’s a little bit more than that in this case
That’s not even remotely how these models work.
Am new to this field and have keen interest in learning more about this so could you tell me how they work, Iβd be happy to learn!
This message has an incredible LLM-response vibe… maybe it the words “keen” or “happy”, or maybe it’s because if you’re interested in LLMs there a quite a few resources that you can simply stumble upon by yourself, rather than asking in a random HAD comment section.
A version of this has been quite common for a relatively long time. Basically prompts that tell the LLM to “answer with a yes or no only” or “answer with a valid JSON only”, upon which it’s output is evaluated by a software.
I hope I’m not missing anything
Yes, this is a very common pattern. But it is somewhat expensive and slow. The new model type improves the cost and speed significantly.
As far as I can tell the only difference is that instead of a soft max, you get 255 binary soft maxes
You also get the output as the equivalent of a single token. Even prompting a LLM to constrain its output, it still needs many tokens to provide the output.
You’re essentially right, but what has been done here is take an LLM and 1) limit its output to a specific, lightweight format, while 2) keeping the rest the same.
The result is a significantly more efficient model that has no need to generate text & expects only certain question formats, but still “knows” everything the full-featured one does.
“I have no mouth and I must Y/N.”
tbh, running a pass over a whole assembled string of output, eg. taking the full output and running it through a JSON parser, is pretty inefficient.
But, inefficiency is just the order of the day when it comes to LLMs.
It’s optimized to be faster.
One major difference between this and traditional LLMs is that it also outputs a confidence score for every answer. Another is that these models are tiny compared to modern LLMs. I haven’t played with these yet, but smaller models for fast decision capabilities could be quite useful.
The confidence score isn’t new. Normally an LLM predicts the next token by providing an output vector containing “next token probabilities” and those are used to select what letter comes next. This confidence score is just the raw weighting between each of the allowed answers, eg. between ‘y’ or ‘n’ coming next.
Maybe the LLM predicted a 60% chance of y and 40% chance of n, with all other probabilities forced to zero in a “decision” model. Well then the model is “60 percent confident.”
Like many things, it’s all just what you decide to call it.
This sounds like the best solution to build a magic 8-ball that actually works
Excellent use case.
Q: I have a pet cat and pet mouse. Should I buy a mouse toy for my cat?
A: 0.0 C: 0.75
With optional rationale plugin: How much do you value your pet mouse?
I’m toying with Laya right now. I don’t like Generative AI mainly for ethical grounds, but Laya uses very small models that do not mass theft everything they can get their hands on and does not require a nuclear power plant and lots of TB of RAM to run (typical models with ~400M parameters). They also have a strong focus on Open Source and encourage you to fine tune the model by training with RLCD.
Language models still require mass theft, even decision ones. If it understands language, it had to be trained on language, regardless of whether it’s outputs have been clamped to only a restricted set of outputs. Whether it outputs “language” isn’t a reliable indicator of mass theft.
Processing natural language language requires a training set much smaller than pretending the model can be used for anything from writing maths proofs or software to biology papers. I have seen experiments of people training their own LLMs using years of their own run IRC logs that have a quite decent natural language chat abilities.
As I wrote Laya uses a very small model (about 400M parameters, about 1 million times smaller than whatever OpenAI or Anthropic may be running) and due to that it has low accuracy compared to Jev, and requires you to train it using RLCD to enhance accuracy.
So does Laya use theft content for the training set? Maybe, I have not seen the training material. But with 400M parameters, the level of theft if any will be way smaller than what the big AI labs do.
Intellectual property rights are the modern day enclosure of the commons.
Could Jev be used to chose the T-shirt winner on the podcast ‘what’s that sound?’ feature, or to pick ‘most wrong’ answer.
RedHat recently posted a comparison of how Jev compared with some other models at implementing guardrails – may be of interest:
https://developers.redhat.com/articles/2026/10/02/benchmarking-ai-decision-models-against-traditional-guardrails#
Useful in robotics if it’s fast enough.
There are better VLP models for that.
Basically, JEV is not replacing the LLM,
It is the compliment to LLM.
It’s an LLM with it’s outputs clamped.
Not really “new”; since the 90’s – maybe even the 80’s – there have been “expert systems” plugged into all kinds of specialized roles, especially in the financial sector such as the stock markets. All they did was make decisions based upon programmed “decision trees”.
They were integral to NASDAQ and other markets to the point that real estate closer to the exchanges was valued much much higher because there would be less wire between it and the machine, allowing a few microseconds to be saved on latency for high-frequency trades.
The difference is this has natural language input
The difference is this is a squishy probability thing rather than a rules engine. So in other words, nothing alike.
Sounds less like an expert system and more like a classifier on steroids. But it’s pretrained pretty well for most use cases
“Backwards chaining inference engines” since the 80’s for things like troubleshooting a GE train locomotive.
Don’t let the restricted output fool you, this thing is just as squishy and illogical and impressionable as a normal LLM, and equally capable of halucinations.
Actually it’s not, not even close, and the most absolute basic elementary understanding of computer science and machine learning would be enough to know otherwise.
Expert systems have nothing to do with this. They operated on rules and were always perfectly reliable, to the point of being unable to cope in unusual circumstances.
LLMs, and all other technology developed since 2013, are statistical models that come with a random chance of failure. LLMs compound this further by literally rolling the dice to figure out what letter comes next, trying to be a next-letter-predictor.
There’s a much bigger issue here, where the public keeps thinking of “AI” as being a rules-based perfectly-unbiased and logical thing from the 1980’s, when modern AI is a mush, squishy thing that’s open to influence and suggestion.
This thing? It’s a mushy squishy thing that’s been made mute except for a multiple-choice output.
I think you have an important point to make: LLMs are not the whole universe of AI. I have been “doing AI” since 1974. For my own convenience, I identify 13 different AI branches or paradigms. Neural nets (and LLMs are basically just massive ANN), can’t do Logic or Math; they need auxiliary tools or plugins. (My favorite for ChatGPT is Wolfram, but I have over 32 extra selections.) The thinking behind JEV will eventually spread more capabilities to the AI environment and IMO LLM will just be the method of communicating the results to humans. Autonomous control will be totally out of reach for humans.
Kevin Kelly wrote a great book, “Out of Control”, that explains this very well. (The book is available for free on his website in PDF, which he has occasionally updated with new info and illustrations. ALL Authors should do this with their books!) Although I have built dozens or hundreds of different AI apps over the last 58 years, this book appeals to me because of my major interest in Complexity and GA (Genetic Algorithm) and emergent behavior.
Thank you for sharing and bringing up “Out of Control”; I think I’ve scanned through it way back when, will be giving it proper/thorough read during my next offline-detox (aka “vacation”).
On the topic of emergent systems, there was not bad loose collection of things in “The self-made tapestry” by Philip Ball (he has other books, and awesome RI talk) which dealt with things like hexagonal beehive cells (and water vortexes) emerging as part of simple dynamics, no intelligent design needed.
Also, what are other 12 AI branches?
Bullshit detector: With some smart glasses and microphone, display in real time the probability someone is talking crap.
Everything old is new again, isn’t this just your typical neural network classifier as was in fashion in the period 2016-2020 (approx) where one would train a network to learn to separate photos of cats vs dogs?
Architecturally, they are not very similar (beyond both being predictive neural networks). Jev is still a transformer-based large language model, like chatGPT, Qwen, etc.
Conceptually, it’s somewhere in between the two, which is exactly the point. You can treat its output like a traditional numeric classifier, with the programming speed and simplicity that allows, but you can still prompt it with different questions at runtime rather than being limited to a few pre-trained concepts baked into the model.
Classifiers are still much smaller and faster if you just need to do something specific and predetermined, like finding all the faces in a photo. And regular LLMs of course are better if you want a detailed, human readable response.
JEV: 42
A faster IF, THEN, ELSE for agentic AI.
Just as capable of hallucination too.
You seem to be hallucinating what hallucinations are.
Lower powered models like llama-3b and phi-4-mini are great for this sort of simple classification. They’re small though to run local on cheap hardware and fast on their own. They’re also kind of dumb unless you have a really narrow task or some way to preprocess the input to make the answer obvious (typically a smarter model)
If the claims on the website are accurate they might be pulling off the otherwise impossible by getting the same result faster in a single step without the extra call to a paid API.
You can also set weights and just use a PRNG to randomly choose with probabilities for each integer. An LLM when given access to the OS delete command, say rm, will at some point randomly decide to use if allowed too. This could be catastrophic for a system, I think the company or individual who initiated and ran the model should be held accountable as if they intended to do so. So far corporations are pretending their “agents” have intent to avoid responsbility for their own softwares illegal activies.
Not sure anything can be learned here. The website goes immediately into pricing, looking to have lower inference cost, for that true/false usecase.
But thing like: “Zero Hallucinations, because Jev decision comes with a confidence estimate.” are just red flags.
There will come a day when someone reinvents the steam engine governor and tries to sell it as the “ultimate AI”.
It’s just as capable of hallucination. Any suggestion to the contrary is a boldface lie.
This thing is just a normal LLM (albeit a smaller one) with the outputs clamped to a smaller set of possible tokens. The “confidence” is derived from the relative probabilities for the next token LLMs already produce.
Obviously. Any machine can malfunction, nobody is implying it’s infallible
Wrong, a hallucination is only possible when an arbitrary choice is made and fed back into the system, these classifier models do not make such choices, that is up to your deterministic code to handle. No offence but, you have NFI how these things work.
Of course hallucinations in LLM are this simple, right? Just have to look at the value from, 0 to 1. If only the guys from Google, Anthropic or OpenAI would have understood that. Because the way Jev explains it, even I can understand it… it’s solved!!
RED FLAG !!!
Youve been posting a lot of random noise in this comment section, most of which demonstrates an extreme lack of understanding of how these models work. Please educate yourself and stop spreading misinformation.
“but it outputs only floating-point numbers.”
Will we reach the point where we will have the answer… 42.0@ and forget the question… ? Like in H2G2
Lol
They never knew the question. They got the answer to the question (42) then had to build the Earth as a super computer to find the question to the answer. Then the Golgafrinchan “B” Ark crash landed on prehistoric Earth and destroyed the original Earthlings, then later the Vogons destroyed the Earth, so the question to the answer was never found.
Imagine a semantic switch-case statement, that is a system-one model in a nutshell.
Traditionally the input has to already be one of a set labels. If the real input is a messy email, “my card was charged twice and the invoice doesn’t match”, the switch cannot see it. Someone, or some earlier step, has to turn the prose into a label first.
A semantic case statement does that turn itself. You still declare the cases in advance. You hand the model the raw state (text, JSON, a ticket, a transcript) plus the allowed options. It returns the matching case and a probability for every option, in one pass, with no generated sentence. Your code then switches on the typed result exactly as before.
Get your head around the above pattern and you will see where Jev et al fits into your existing coding methods.
There are also use cases involving ontologies and semantic tagging but that is just going to confuse most people until they are comfortable with the semantic switch-case paradigm.
mmm…feels like squirting out a good ole Fortran…meh
I think many of you are looking at it from a technical point of view. It’s not that AI can make better or worse decisions than us, it’s that people are increasingly reluctant to:
A) make decisions without lots of assurance. AI provides an unnecessary amount of it, and;
B) admit that giving up our decision making power undoubtedly creates a more pathetic, less autonomous, less committed, less human humanity.
Even if AI was a perfect little objective god that you could let entirely run your life, why let it take one of the joys of being human – stuffing things up and deciding to do better (or worse) next time?
So, sure, use AI (let it do your taxes, remind you to exercise in the morning, debate with it) but for humanities sake, don’t give up one of the things that made us humans – (flawed) decision making without all the data, in the face of uncertainty. We’ve already been there a few times. It didn’t work out well.
Enjoy that you can effect the world, affect the people in it and have some minor say in your own lives and take responsibility for it.
To paraphrase Voltaire, Nietzsche and Bakunin (notice I’m not quoting them):
If God wasn’t already dead, it would be necessary to invent him and kill him again.
“I’m sorry Dave, but I’m affraid i can’t do that”
Hotdog or Not Hotdog