Like so many large and popular open source projects these days, the Godot game engine struggles with an influx of pull requests. The situation has become increasingly dire due to the advent of AI-generated code. More specifically, the issue involves the inverse relationship between PR code quality and the number of PRs, which wastes a lot of time on the side of a limited number of (volunteer) reviewers. This has now forced the project to update its contribution policy.
An interesting point raised in the announcement article is that of the demoralizing effect of AI-generated PRs on reviewers. Often the human behind such a PR isn’t interested in being educated, or may even be an automated agent which isn’t capable of productive discussion on pros and cons of certain coding approaches — never mind in becoming a more permanent maintainer for the project.
This problem has led to new rules being instated, which include a ban on autonomous AI agents and vibe coding, a ban on substantial AI generating of code, and a ban on AI-generated text in human-to-human communication. It also codifies the requirement that all PRs are to be reviewed and approved by a human being before merging.
In many ways this new policy is similar to that of the Mesa project, which demands code comprehension on the side of the submitter, although it doesn’t go as far as NetBSD, which just outright treats LLM-generated code as ‘tainted’ due to potential licensing and other concerns. Other projects like the Linux kernel opt to make the human submitter responsible for any AI tool usage by forcing them to declare it.
Meanwhile there are also indications that such ‘AI tool’ usage is reducing useful interactions with open source projects. What the future will bring here remains to be seen, but at least as far as open source projects go these tools are clearly increasingly being banished.
Been using ChatGPT for a project (that will unscramble a scrambled image) for about the last year… with various results.
My take, echoed by the head of the local makerspace (also a professional programmer), is that you have to be a domain expert in whatever you’re doing. Under those circumstances the AI can be a tremendous time saver, it can do a lot of things quickly, and give you quick progress on your projects.
You need to be the final arbiter of what goes into your code. In my case I self-impose that by not letting the AI touch my source files, I only cut/paste what it shows me, and then only if I like what it wrote.
I also write extensive test routines for everything, and run them, and this discovers a fair number of bugs in its own right.
This thing of just telling AI to do something, without domain knowledge for whether it’s right, is a recipe for disaster.
It’s the unit testing that seems to be missing from submissions. A lot of open source has testing requirements, some quite extensive. A code submission goes through a testing suite before it even gets to pull requests. And tests for new behavior. e.g. Mesa, NetBSD, Rust, Kubernetes, and PostgreSQL, etc.
As a senior engineer and domain expert, I find these tools amazingly powerful. However, I think open source projects/maintainers need to reevaluate existential questions about what they’re doing and how they’re doing it.
Why are the overwhelmed maintainers not building their own AI based review workflows? Honestly, I did that at my own job, and 85% of what I used to check for in PRs can now be done automatically. The world has changed. You can add 500 features to your project this year, literally. The purpose of the maintainer and reviewer now is to decide what makes sense to add, and if the architecture/security/perform jives. Very soon, we are not going to be caring as much, if at all, what’s in the code because advanced agents like codex can do that impeccably well.
i had a similar experience. i’ve developed a complicated system for ESP8266s to poll a server, picking up state info for GPIO while sending sensor data. every time I’d ask ChatGPT about a reliability issue, it would always suggest not using Strings but to do things with cstrs instead. Sometimes it would produce code that looked and functioned great but was impossible to read (because of the arcane cstr processing it was doing). in the early days, i would just replace my code with this. but months later i would find that the replacements weren’t bothering to handle certain rare things that i absolutely needed it to handle. not only that, this hard-to-read code used more memory and was less reliable! so now i am very careful to only use ChatGPT to write functions with a very limited scope of inputs and an easy-to-test set of outputs.
Just remember that your hard earned skills are all “use it or lose it” — too much time with LLMs and you may not be the domain expert you once were.
The AI-First mindset.
https://www.amazon.com/dp/B0H2HKR15H
Your point about domain knowledge is in the sample.
Seems like a really reasonable requirement imo. I personally detest the use of AI for personal communication. If you’re giving the AI some bullet points, ..just send me those bullet points instead. Having the AI bury those points in a mountain of slop serves no purpose, and its infuriating.
I recently started noticing a colleague’s emails becoming incomprehensible, wordy, and lacking the detail necessary to proceed with the project at hand. Ask Copilot why that was happening…
Infuriating. Yeah, that seems about right.
Run them through a AI to simplify and mail them back, if that’s it what they mean. They will stop doing that soon.
I support heavy handed bans for PRs over no bans at all until ai users and their processes can sort themselves out. If you’re going to even think about using ai, or contributing to a project you need to test, you need to ask yourself and the llm questions. You need to apply as much good faith effort as you expect volunteers will for pulling your code. All of the open source community projects need to maintain good faith and respect for each others time and submitting PRs you didn’t test, or didn’t pull into fresh envs, or didn’t think about is not the way any of this is meant to, nor should run. Users with that respect will be smart enough to work around said bans without ethical and problematic issues, because they use the tools properly, in a way that aligns with the status quo. Its the difference between a good substack user and a help vampire.
It seems like a sensible position when a significant amount of the load on these projects is review(for which they have long been generally under-resourced; but there at least used to be some symmetry between the amount of time required to review something and the amount of time required to generate something that looked like it was worth reviewing); when at least some of them explicitly have some sort of community/collaboration objectives in mind (even ‘AI’ optimists don’t seem all that sanguine about the effect on the world’s supply of entry level learners who would ordinarily be picking up experience by doing); and, ultimately, when even the people who try to cheat the rule basically end up following it(for any purpose except pure ideological opposition to bot use).
It’s not like it’s impossible to use a bot and, with some amount of adjustment, end up submitting something that looks like you did it yourself; but if you do that you aren’t really beating the system; you are generating the sort of human-type pull requests that people want you to.
Despite being slop, AI output is also a clogger. Noslop Godot. Bots in the early net were used for muchthe same purpose, DoS attacks via repeating messages
This problem is not limited to open source projects either. At my day job I have to deal with A.I. agent generated merge requests and code review comments. Unfortunately upper management, under pressure from shareholders, has promised to use A.I. tools to double revenue and halve the development time between releases while maintaining net zero hiring for both software and hardware engineers.
One of the most annoying things is that the A.I. agents seem to spend more of their “effort” on being able to mimic the tone, terminology, and style of each team’s code and review process than they spend on “understanding” the code itself let alone the problem at hand. The result is that, while the bot is often wrong, it’s eloquently wrong (and a good mimic) so it’s not uncommon to spend a whole day picking apart a change only to discover in the end that the subtle bug the bot claimed to have found in your code was just an hallucination. Similarly the bots seem to have a tendency to try and fix symptoms rather than root causes (e.g. putting consistency checks at every point of use for a data structure rather than obeying the invariants to avoid corrupting it in the first place).
I feel like it’s going to be a painful and messy awakening for management when they discover that the A.I. tools they forced down our throats (while touting how they will improve productivity and code quality) are actually hurting both. My concern is that management will demand the extra throughput with or without tools (and use that net zero hiring as a threat against anyone who wants to maintain anything approaching work/life balance).
In other words, banning Artificial Idiocy’s hallucinations. Good move and it is rather sad that this move is not copied by many.
What did I say just ~two years ago, AI-induced slop/fad/whatever is The New Bureaucracy, because that’s exactly how bureaucracies work, first by being quite well organized and coordinated, and accomplishing great things, then eventually growing out of manageable proportions (bureacrats tend to hire more bureacrats instead of workers) and inventing their internal hallucinations/raindance-rituals that eventually become more important than the actual task being done and take most, if not all, resources.
Witness how UN is now largely incapable of preventing or stopping wars, which was the main purpose of creating it after the WWII. (second very, very unpleasant part of the UN myopia is even worse than that – dictatorships of the world, seeing how UN is quick to help with feeding starving masses, now literally use UN help to rob their own nations even worse – by dumping all their responsibilities on UN to handle, education, for example, as in public schools kind, UNICEF, pointing finger at you).
Translation – we as specie largely failed to create a good/solid/reliable body/organization/initiative/institutions that would overview and control the NEGATIVE aspects of our inventions. This (lack of control) has been done before many times over, so nothing is new here. It remains to be seen if this jolts anyone into strategic action – as opposed to tactical band-aid patching here and there.
(how do you translate the german “Tja…” to english?) it was given to us about 2000 years ago, but ever since we keep failing because we insist on inventing our own, which we can’t.
“Tja” translates as “well”, as in starting a new sentence that may contradict what has been said before –
“Well, this may not be the case…” and unrelated to the literal/direct translation as “good/gut”.
I’ve already pointed out cast iron production as one well-known example of government losing control over things like weapon making.
thanks for the suggestion, I didn’t recognize the english “well…” spans all the way between the simple sentence starter to the sigh of despair “said it thousands of times, explained it, showed it, tried to force it, begged for it, now you complain it doesn’t work out doing it different, and I’m tired of commenting on that complaint”, in which case the “tja” is the only word replied, usually in combination with a sad face and a shrug, especially when talking about someone else or a group of others