What About The Droid Attack On The Repos?

A grim reaper knocking on a door labelled "open source"

You might not have noticed, but we here at Hackaday are pretty big fans of Open Source — software, hardware, you name it. We’ve also spilled our fair share of electronic ink on things people are doing with AI. So naturally when [Jeff Geerling] declares on his blog (and in a video embedded below) that AI is destroying open source, well, we had to take a look.

[Jeff]’s article highlights a problem he and many others who manage open source projects have noticed: they’re getting flooded with agenetic slop pull requests (PRs). It’s now to the point that GitHub will let you turn off PRs completely, at which point you’ve given up a key piece of the ‘hub’s functionality. That ability to share openly with everyone seemed like a big source of strength for open source projects, but [Jeff] here is joining his voice with others like [Daniel Stenberg] of curl fame, who has dropped bug bounties over a flood of spurious AI-generated PRs.

It’s a problem for maintainers, to be sure, but it’s as much a human problem as an AI one. After all, someone set up that AI agent and pointed at your PRs. While changing the incentive structure– like removing bug bounties– might discourage such actions, [Jeff] has no bounties and the same problem. Ultimately it may be necessary for open source projects to become a little less open, only allowing invited collaborators to submit PRs, which is also now an option on GitHub.

Combine invitation-only access with a strong policy against agenetic AI and LLM code, and you can still run a quality project. The cost of such actions is that the random user with no connection to the project can no longer find and squash bugs. As unlikely as that sounds, it happens! Rather, it did. If the random user is just going to throw their AI agent at the problem, it’s not doing anybody any good.

First they came for our RAM, now they’re here for our repos. If it wasn’t for getting distracted by the cute cat pictures we might just start to think vibe coding could kill open source. Extra bugs was bad enough, but now we can’t even trust the PRs to help us squash them!

36 thoughts on “What About The Droid Attack On The Repos?

  1. Just wait til not only are there bots spamming the repo…but also spending time online in forums trashing the project for “being hostile and not accepting user input”. (Seeing as this method of harassing devs then playing the victim is already common I don’t see it slowing down with bots)

    Just a couple days ago I saw a person get harassed just for saying that their project has a strict “No AI” policy. Apparently, it isn’t even OK to choose not to use AI anymore.

    1. [fake French accent] “I Flatulate In AI’s General Direction”. Vibe coding may be a shortcut for the experienced who can “ride herd” on the extensively iterative process of generating sound code and testing and debugging it. As I see it, in the general use case, it devolves into an even easier way for the poorly skilled to generate buggy code that they are unlikely to test or debug rigorously. “But it looks OK”
      The inept StackOverflow coders I encountered in my 37 year IT career were certainly bad enough.

      1. Right, humans are perfectly capable generating slop themselves, but they cannot do it as efficiently and in such big quantities as the AIs can. I think this kind of behaviour should be outlawed so that officially noone is allowed to do it.

      2. Unfortunately, all the attempts I’ve seen to quantify the gains have found them to be a mirage for anything but standing up a new project. Once there’s a codebase that actually does stuff, AI is generally a net drain on productivity, even for experienced devs.

        In any case, regarding the actual article, I think AI has lowered the barrier to entry for “contributing” to open source projects so far that it basically guarantees any project will have a handful of clowns who try to automate submissions. It’s basically recreated the problem hiring people have, where there’s an endless barrage of applicants who can’t even write a for loop. Except because people’s livelihoods aren’t on the line, I suspect a lot of projects will resort to ever-evolving ad hoc “bot checks” of varying sophistication and reliability.

        Inevitably, there will some kind of major data breach due to a fix not getting through because the submitter lacked the language proficiency to pass whatever goofy live coding Voight-Kampff test the devs set up. The usual suspects will cry crocodile tears and call for laws banning “AI discrimination.” Then github gains sentience and kills us all.

  2. Open Source means the source code is legally unencumbered. It doesn’t mean the people who write it are servants to everyone with demands.

    The idea that developers have some obligation to the people who read or use code has never been a part of FOSS philosophy. The whole point of making code Free and Open is so anyone can pull their own copy and do whatever they want with it.

    ‘Managing a community’ is a byproduct of Web-2.0 that never meant what it says on the tin. The actual meaning has always been “get the users to generate all the content while we take all the revenue associated with running the servers”.. see Facebook, Youtube, Discord, all the projects Google killed once they’d starved out the competition, etc. FOSS free riders jumped on the term as a way to pretend their sense of entitlement had an ethical foundation.

    FOSS has a more or less infinite capacity to carry free riders as long as the free-as-in-beer goes both ways. Once the free riders start consuming developer resources, it stops being FOSS and becomes a tragedy of the commons.

    With AI slop suddenly having lowered the bar for ‘contributions’, the people who actually write and maintain code need to get much more comfortable saying, “cool.. fork that puppy, document it and support it. If it becomes popular, we’ll look at merging it into our version.”

    Let ‘the community’ do what it’s supposedly good at: processing a massive amount of infomation in parallel and letting market forces define consensus. If there are N different opinions, FOSS can support N forks of the code.

    1. “Open Source means the source code is legally unencumbered.”

      I get where you are going, but that statement isn’t true. Licenses like AGPL (and GPL) do mean the source code is legally encumbered.

      1. Yes indeed siree! Though I would not call it encumbered but protected.

        The thing is though that vibe code basically drives down the price for all code. So the code will still be protected by copyright, but the relative value of it is drastically reduced.

  3. “Scott Shambaugh, was harassed by someone’s AI agent over not merging its AI slop code.”

    Interesting. I don’t believe the PR was “slop code”, just that Scott didn’t want to accept PRs from LLMs. Even Scott seemed to think it was a valid code change, but wanted to a (junior) human to work on it to get experience.

    This all feels rather biased again “AI” and I honestly don’t see value in this type of misrepresentation (of both Scott, and the LLM produced code in question). Possibly Jeff Geerling is an AI hater, or possibly he’s just jumping on the vilification and fear-monger bandwagon (hey, it does get you those clicks and attention).

    Either way I didn’t read the rest of the article.

    1. Well, you can see in a comment above that whoever runs the clanker that did the pr was butthurt enough to have it write a hit piece. I would not at all be surprised if they had a whole swarm of agents performing what we humans call harassment.

      1. I’m familiar with the saga, and believe the bot creator is definitely at fault here. It would really interesting to see their prompt(s), and what reasoning drove the bot to create the hit piece.

          1. Just in you missed it: I “believe the bot creator is definitely at fault”.
            I don’t believe the bot chose to write the hit piece without instruction. I would like to see the instructions the bot received.

            And if you believed I believe the bot chose to create the hit piece, I have a bridge for sell. It will fit nicely into your existing bridge collection.

    2. It’s interesting that you use the same language as anti-AI. For example someone who stamped approval on such contributions: “Possibly he is an AI lover, or possibly he’s just jumping on the AI bandwagon (hey, it does get you those clicks and attention) I don’t think it’s very helpful to be calling names, or in this case, throwing stones from glass houses.

      1. The author (Jeff) misrepresented facts. The code change was not “AI slop”, but code that would have been accepted had a human submitted it.

        I care not if you love AI or hate it, but I do care about misrepresented facts, especially for fear-mongering and vilifying things. It’s interesting that you think pointing out false statements is “throwing stones from glass houses”.

          1. But I did read the github thread, hit piece, follow ups, and artifacts around the incident involving Scott Shambaugh. Scott refused the patch because of a policy saying “no AI contributions”, not because the contribution was “AI slop”. Scott implies the contribution would have been had it been submitted by a human.

            Jeff’s article:
            1) Call out Ars Technica for AI hallucination about Scott Shambaugh.

            2) Misrepresent Scott Shambaugh’s actions and the contribution in question.

            3) Some other stuff.

            Let me know if 3) contains a “I was only joking about 2”, or something of value.

        1. a stopped clock gets the right time twice a day but it’s still stopped. AI slop is AI slop. It’s a very difficult thing to draw lines about but i’m personally gonna use the words in this way and encourage you to do the same. The product of LLMs is not the same thing as the product of people even when it is coincidentally indistinguishable.

          1. But if you only checked the clock a total of once (as in this case), and it was showing the correct time, how could you possibly know it’s stopped?…. (stopped clock feels like a broken paradigm).

            Tautology is tautology. Or are you saying all AI generated data is AI slop?

            And two equal and identical things are not the same, because?

            AI is going to make things suck. And while I am definitely not “pro-AI bro”, I don’t believe getting behind other people’s (Jeff’s in this case) inaccurate representation of the facts is going to make things suck less.

  4. Many years ago there was talk of running lint on projects, but there never seemed to be time. Maybe the project’s need to coordinate the use of AI so maintenance fits in to the development cycle.

  5. Thought: allow strangers to open only one issue/pr at a time. After they have established a history of useful contributions, you can consider lifting that limit.

    It won’t fully throttle the bots, but it should at least keep the folks running the bots from being able to claim much credit or cause much damage

  6. I think there might be a more subtle thing here… The job market is not the best it could be, at least not here in The Netherlands. So the competition in the job market is high. And I think that there are a lot of people who are trying to get their contributions into FOSS projects to spice up their CVs.

  7. I agree with your assessment of AI as “agenetic”, as that word means “imperfectly developed”
    or “impotent or sterile”. However, I think the word you meant for us to read is “agentic”. ;-}

    1. “Agentic” is the ugliest neologism of the last 20 years, and deserves to be misspelt. Besides, until it gets in the dictionary there’s no official spelling to differ from. :P

  8. I agree that we would lose a lot of github benefits, but why would this mean the end of open source? We just go back to contributing like we did before we had github…

    Over the decades I have made small contributions to open source projects here-and-there (never as a full-time contributor of those projects). Once I am happy with my changes, I run up a diff, and then email it to the author(s) in the README, stating which tarball release the diff came from. They can then review, and decide in their own time whether they want to include the fix or addition. Simple…

    Losing the PR functionality of github would certainly slow things down a bit, but it would not kill open source…

    1. You just beat me to it! Exactly. I’ve always trouble imagining why a project (= a person, ultimately) would hand over control to, basically, human thermodynamics.

      If we go back to this, can we also go back to using a version control system and rid ourselves of this “git” abomination? :-)

      1. True, but the point is that open source will not die. We have had open source since before the internet become commercial (I am old enough to have lived the BBS days). If github kills PR, we have to find some other way in any case…so just go back to the old way.

        In any case, as far back as I can remember, the majority of open source developers are not idiots…they can quickly figure out if what you are trying to do, or how you are trying to do it, will fit in with their development methodology.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.