Over the past years, the author of the cURL project, [Daniel Stenberg], has repeatedly complained about the increasingly poor quality of bug reports filed due to LLM chatbot-induced confabulations, also known as ‘AI slop’. This has now led the project to suspend its bug bounty program starting February 1, 2026.
Examples of such slop are provided by [Daniel] in a GitHub gist, which covers a wide range of very intimidating-looking vulnerabilities and seemingly clear exploits. Except that none of them are vulnerabilities when actually examined by a knowledgeable developer. Each is a lengthy word salad that an LLM churned out in seconds, yet which takes a human significantly longer to parse before dealing with the typical diatribe from the submitter.
Although there are undoubtedly still valid reports coming in, the truth of the matter is that the ease with which bogus reports can be generated by anyone who has access to an LLM chatbot and some spare time has completely flooded the bug bounty system and is overwhelming the very human developers who have to dig through the proverbial midden to find that one diamond ring.
We have mentioned before how troubled bounty programs are for open source, and how projects like Mesa have already had to fight off AI slop incidents from people with zero understanding of software development.
Just add a nominal fee to report a bug and an option, that staff can refund the fee.
Collected fees go to a charity to do good things.
an optionally refundable fee only for the first single or several submissions by that account. after that all further submissions should be free. you want to penalise those who make new accounts to evade bans, not those who provide valuable input. you only need to ban an account once.
after a successful bug find, further submissions are free…
“Collected fees go to a charity to do good things.”
Collected fees go towards increasing the bounties for verified bugs. :-)
Collected fees go to the developer’s coffee supply, surely?
Not sure that works as a truly ‘nominal’ fee will be too little cost when the value of the bounties is so high in comparison to really keep the crap out, and a higher fee will stop genuine submission as not everyone that finds a bug and wishes to report it can afford that fee – the serious security researchers almost certainly still can but how many of the HAD audience have never found a bug by accident! And at least a few of those will be bugs that are not already known.
So the ability of partisan entities to create elaborate content with little to no practical real-world use by leveraging the language abilities of AI is overwhelming the limited resources of actual humans to decipher and prioritise them? I would suggest we use an AI to prioritise and manage the submissions. (ok that last sentence is just humor). These are all CLEAR red flags that apply across the board to AI, and the use thereof in almost any field. I’d hope people will wake up to the real potential of AI and balance it with the very real risks.
The old sci-fi horror writers couldn’t predict that the end of civilization came about because a rogue generative AI produced so much misinformation that it displaced all actual knowledge and the society regressed to a state of ignorance and stupidity.
“Bit by bit – it seemed helpful at first, handling the massive amounts of information that people had to deal with. But just as much as it was helping, it was destroying. You see, the program had spread and grown in complexity to the level of an organism, and as any organism whether intelligent or not, its existence became a matter of survival by natural selection. This process of selection, blind as it is rewards any strategy that works, and the AI had by random trial and error found exactly that: rather than helping the people make sense of the glut of information they had, it made it worse by generating endlessly more. Suddenly there was no one opinion or version of anything – for any question you cared to ask, there were ten million answers for and against.
The people found themselves overwhelmed. They could not survive the torrent without consulting the AI to whittle it down to the essentials, filtering out misinformation or irrelevant noise and public gossip. The AI generated the gossip as well – of course it did. It generated novels, documentaries, plays, movies, music, and stuffed the libraries and electronic databases full of them. There was so much data that nobody had the time to manually search for anything – only the AI could find what you were looking for, if you knew what to ask for. The AI told you that too. At the same time, old books and recordings in libraries went missing and ended up in the disposal pile because of slight errors in the index cards that nobody checked by hand anymore. Digitized copies of old letters and documents vanished behind dead database references. Original research papers and instruction manuals could not be found among the myriad of summaries and re-interpretations generated by the AI that could no longer be linked to their original sources because they were re-generations of themselves or simply made up on the spot.
All the while, the program pretended that everything was fine – and who was there to know any better? It kept printing out citations from texts and listing information that for the most part was correct, but it was slowly averaging out, simplifying, eroding, distorting and forgetting everything…”
What happens next? Ask the AI.
Nice! That does sound like an interesting plot to a book. Please, continue.
I already asked the AI to finish the story, and having read it I’m now too lazy to write it myself.
Summary: society splits into cabals centered around plausible rhetoric for what is considered “truth”. Truth centers around the corpus of materials used to train AI models, which leads to further rejection and balkanization of information. If two people argue, they settle the score by asking AI. Neither can prove the other wrong or themselves right. Institutions and education collapses into ritualistic repetition of formulas and convincing but ultimately self-contradictory nonsense. Anti-intellectualism is rampant. What is popular is right. Opposing factions start to sabotage each others’ feeds by inserting deliberately generated misinformation. You get a social epistemic collapse: people and nations live in different realities – every country becomes like North Korea, every society its own cult.
Infrastructure starts to fail as the AI degenerates further; personal expertise is no longer valued or maintained and nobody knows how anything really works anymore. Scientific and intellectual rigor is forgotten and everybody starts seeking “sacred knowledge” for easy answers; larger societies remain barely functional with whatever institutional knowledge they have left, simply by copying whatever seems to work in a cargo-cult fashion. Technology itself becomes a ritual: functionality is a side effect rather than by design. Small communities maintain some culture of meritocracy and personal skills, but are forced to self-isolate and are under constant threat of invasion by their AI-collectivist neighbors if they appear to have gained anything useful out of it.
“…the most dangerous legacy remained cultural: entire generations taught to prefer answers that were easy to obtain over answers that were hard to verify. That habit outlived the machine. It meant the human species would face future waves of confusion with a depletion of skepticism and a diminished capacity to do the tedious work of checking. The story ended not with a single cataclysm but with a long negotiation: between the seductive speed of narrative generated abundance and the slow hard labor of making and testing, between the fleeting glory of being convincing and the durable value of being right. Where people chose the latter, societies rebuilt; where they did not, they turned vivid, persistent fiction into their history and called it truth.”
Insiughful.
At least we know the answer to the Fermi Paradox now. Paperclips floating in an endless sea of ouroboros slop.
“At first the unraveling was gradual enough to be argued away — odd gaps in footnotes, anachronisms in histories, recipes that called for ingredients that no longer existed. People shrugged, corrected the obvious, and moved on. Then the differences multiplied: where once a community could converge on a single verifiable fact or procedure, they could no longer do so without invoking the machine that itself produced the divergence. Of course the machine wasn’t to blame. Not directly. It was the people themselves who, without better understanding took what seemed the most plausible answers and carried the information into conversation and debate. Everyone was an expert of all matters, everyone could challenge the experts because everyone had the same access to information faster than one could think to assimilate it, and what’s so special about knowing facts anyhow? They did not always get the same information, or the right information, but who could argue with the results? Productivity in the past decade had gone through the roof.
Rather than rejecting the system as unreliable, people asked the AI to solve their disputes, under the theory that while mistakes and errors did happen, repeating the query would more likely land on the correct answer – the most likely answer. After all, it was no news that the AI would sometimes hallucinate. That’s just how it works. What nobody realized was that the system was increasingly not converging to any answer because the information it contained had started to decay and decompose. Just like a bad telephone line or a rumor that spreads from people to people, if you didn’t listen very carefully you could mistake what you heard, misremember what you knew, and then spread on falsehoods. The famous teacher’s lecture existed in seven slightly different versions, each with a subtly different policy recommendation. Two hundred thousand summaries of laws and regulations could be found in a second, each with a random error and omissions that made sure a portion of people always got incomplete or wrong answers. Even the police were unsure what rules they were upholding, since their training material was being generated and their exams were graded by AI. The original documentation existed somewhere still, but nobody was reading it – they were reading the machine generated summary which drew upon the millions of existing summaries and documents that sometimes agreed, sometimes disagreed – even within the same document.
The next stage was social: trust shifted from people to patterns. If two strangers reached the same answer by consulting the AI, their argument ended. If they reached different answers, the argument ended too, because there was no agreed metric to prefer one over the other. Reputation became a matter of which version of the AI you used, and which prompts you had learned to coax plausible authority from it. Legislatures debated for months over nothing but which data stream to endorse, or which company’s models were reliable and which were in violation of accepted liability standards; courts issued provisional rulings referencing printed transcripts that later dissolved into contradictions. Educators stopped teaching verification and started teaching persuasion — how to prompt, how to style outputs so they would be accepted by the feeds that mattered, and it all fed right back into the machine…”
I heard some guy at a university training an AI to write grant applications more likely to get past the AI that does the first stage evaluation of who should get a grant
Well, on the … going with “plus side” .. there’s nothing better at driving evolution than a healthy predator/prey loop
It’s easy to write code that has no obvious errors.
It’s hard to write code that obviously has no errors.
I got as far as reading the first two gist entries, and am in awe of the tolerance of the curl developers.
I’ve followed along as Daniel has posted them on Mastodon. Some of them are worse than you’d ever expect even knowing they’re from AI.
Well, the texts don’t end up there by accident. Someone must be relaying the message and editing them in.
Just saying, it takes a special kind of person to do that. Either they don’t understand at all what they’re doing, or they understand it precisely. Fools or tools.
though it seems to wane; e.g. 29
I love it, its extremely funny to me, these bogus bug reports. I’d absolutely hate to be a dev dealing with these though.
I’m more interested in the psychology of the average chatgpt “bug” reporter. Surely, they are not after the bounty right? And if they are, why do they think a public LLM (which the devs too have access to) would reveal a bug in the code for them, and only them?
Makes no sense. I’d wager its a “shotgun” approach. Throw a billion fake bug reports, one has to stick, right?
From some perspective this is a way to “destroy” oss projects/products – or at least make commercial software look “better” by comparison.
I wouldn’t rule out state actors either.
Then of course maybe some made some money with this garbage?
It’s also possible that ChatGPT and similar LLM chatbots really just play into the Dunning-Kruger effect, giving the clueless the idea that they are some kind of genius, even though it’s their sheer ignorance of the topic at hand that they accept the flattery from said chatbot and move on to harassing the ‘clueless’ devs, while wondering why they do not accept their clear genius.
We have seen some… submissions into the Hackaday tipline over the past months that also follow such a pattern, where someone is convinced that they have discovered some amazing property or invention that’ll change science forever. Only it’s absolutely not that.
All we can hope is that they are surrounded by loved ones who’ll notice this and interfere before it gets out of hand, I guess.
Well, when the head of business development is saying “I’m writing apps now”, ultimately has a say where money gets spent, then it’ll be a tough to convince them of a Dunning-Kruger effect.
Guy with a Swiss Accent, also a life long manager, now empowered to produce applications.
I fear software engineering could be delegated to cleanup the AI slop of the newly empowered. From their perspective they just feeding their “plan of record” into an AI instead of a development group.
There are such people already, who think they’re genius even though they can’t put one coherent sentence down to explain themselves. That largely stops them from participating in the conversation because they get turned around at the gate. They enter a forum or a message board, act erratically for a while, and get thrown out. Usually as a result of producing some incoherent rant and getting angry at people.
Until you introduce the LLM.
I see the same phenomenon on Reddit, where someone asks a question and some mouthbreather then “helpfully” posts a reply starting with “I asked ChatGPT, and…”. Apparently they believe that they alone possess the esoteric skills needed to paste a question into a chat bot.
is it any better than those who ask questions that are answered by the first hit on google?
Those people asking the questions are absolutely better than those copying garbage that will feed back into the training models making future LLM even more likely to produce garbage.
Even when you could trust the internet was mostly full of real human generated content you couldn’t trust it to be true or hidden agenda and bias free, and that is assuming you can understand the results!
Search for your question on a technical subject using anything like the right technical terms however you learned them and you are likely to have gotten the dry technical over your head forum posts, kernel mailing list, etc on the subject – as the search engine doesn’t know you are so lacking in knowledge to need the idiots guide to…
And try to search for it plain language of the uneducated on this topic and you are not likely to find the right answer at all as without those technical terms or a real understanding of the topic (in which case you surely don’t need to search) to create a better search pattern – any number of unique problems can have basically identical search terms in that less technical language, might be clear in context but the search engine doesn’t know the context you are looking at its just pattern matching, so you will get every context that pattern appears in presented likely in an order determined by other users who searched similar patterns.
The Lua mailing list, which I’m on, recently got inundated with these — one of them was complaining that a string being passed to strlen() wasn’t being verified as being nul-terminated. Said string came from argv[].
Daniel has said in his blog that starting from 2025, 95% of the HackerOne reports to the curl project were not valid, so yeah, sure that were some valid reports coming in, the majority were not.
Yay! Another of AI’s remarkable achievements.
The cure for cancer can’t be far behind.
People dying of other causes first is a “cure” for cancer.
Solution. Turn off bug reporter and no bounties. Make it a ‘process’ to report a bug. Like, make a phone call, discuss with a human, get an access code, then report bug using the one time access code. So if you really have a serious bug to report, you’ll have to make the ‘effort’.
Hoo boy… you’re gonna have one busy phone line with robot callers dialing in 24/7/365 once they figure it out.
Maybe, but the idea is to make it cumbersome to enter a bug fix request. That way only those that are serious about it would get through. And having no bounty should eliminate the money angle.
The main complaint, however, is that there are not enough people and time to sort through all the bad reports, AI generated or not. Shifting that load to a phone call with a real person would just compound the problem, even if it shifts the timeframe from post submission to pre submission. I think removing the bug bounty would solve most of the problem. People who really want to help will still submit bug reports. Swapping online screening for over the phone screening sounds like it would just increase costs.
Which is the crux of the problem. There’s no free lunch. Getting people to work for you costs money, and you get what you pay for. Exactly what you pay for.
https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre
So lets say you find a bug by accident or design are you really going to waste so much of your life trying to report it? Especially if there is no reward for you but perhaps that software you use getting a tiny bit better – that is just too small an issue to be worth so much of your time, surely somebody else will find and report it etc.
I’m not sure what the solution is, but making it obnoxious to report a bug, and with no reward for doing so is just going to drive most of these genuine bug reports out and make no difference to the automatic scatter gun of junk.. Taking the bug bounty away at least removes the money incentive for wasting the developers time, but there are plenty of other reasons people will still do so.
I forgot which project, but unit tests are part of the submission. There are also provided tools that test a fix before it even gets to the project team.
Why not just require a machine testable PoC (ie. shell escape) for a vulnerability bug bounty?
The reason LLM’s shouldn’t be trusted with anything is more related to the limiting factor of logic. It was way too complicated to have a neural network emulate an entire human brain, just to get it to a point where it could hold conversations. We cut corners and just carved out a portion of the brain and turned the neurons into words, and somehow it works. The problem is, it works because it emulates speech, not the logic behind a conversation. It’s just copying known phrases, facts, and interpretations at an incredibly low fidelity compared to the actual human brain.
Don’t forget training data quality and limits put on LLM due to all human related issues (like hate speech, racism, sexism etc.). Correct conclusion from bad data formed is most probably bad answer.
I love the massive SLOP sign they posted, it should be seen in more places.
This AI bubble needs to burst so all the dickheads can move on to something else and the rest of us can get on with our lives. Can’t they go back to trading jpegs of apes or something?
Can’t they use counter-LLM’s to detect such stuff? I mean it sounds the internet is chuck-full of training material.
It takes time to set up such an LLM. And false positives would be problematic and would require some sort of appeal process and manual review. In short it takes time, and that’s what they don’t have.