How Much Programming Can ChatGPT Really Do?

March 25, 2023 by Bryan Cockfield 50 Comments

By now we’ve all seen articles where the entire copy has been written by ChatGPT. It’s essentially a trope of its own at this point, so we will start out by assuring you that this article is being written by a human. AI tools do seem poised to be extremely disruptive to certain industries, though, but this doesn’t necessarily have to be a bad thing as long as they continue to be viewed as tools, rather than direct replacements. ChatGPT can be used to assist in plenty of tasks, and can help augment processes like programming (rather than becoming the programmer itself), and this article shows a few examples of what it might be used for.

AI comments are better than nothing…probably.

While it can write some programs on its own, in some cases quite capably, for specialized or complex tasks it might not be quite up to the challenge yet. It will often appear extremely confident in its solutions even if it’s providing poor or false information, though, but that doesn’t mean it can’t or shouldn’t be used at all.

The article goes over a few of the ways it can function more as an assistant than a programmer, including generating filler content for something like an SQL database, converting data from one format to another, converting programs from one language to another, and even help with a program’s debugging process.

Some other things that ChatGPT can be used for that we’ve been able to come up with include asking for recommendations for libraries we didn’t know existed, as well as asking for music recommendations to play in the background while working. Tools like these are extremely impressive, and while they likely aren’t taking over anyone’s job right now, that might not always be the case.

Why LLaMa Is A Big Deal

March 22, 2023 by Matthew Carlson 27 Comments

You might have heard about LLaMa or maybe you haven’t. Either way, what’s the big deal? It’s just some AI thing. In a nutshell, LLaMa is important because it allows you to run large language models (LLM) like GPT-3 on commodity hardware. In many ways, this is a bit like Stable Diffusion, which similarly allowed normal folks to run image generation models on their own hardware with access to the underlying source code. We’ve discussed why Stable Diffusion matters and even talked about how it works.

LLaMa is a transformer language model from Facebook/Meta research, which is a collection of large models from 7 billion to 65 billion parameters trained on publicly available datasets. Their research paper showed that the 13B version outperformed GPT-3 in most benchmarks and LLama-65B is right up there with the best of them. LLaMa was unique as inference could be run on a single GPU due to some optimizations made to the transformer itself and the model being about 10x smaller. While Meta recommended that users have at least 10 GB of VRAM to run inference on the larger models, that’s a huge step from the 80 GB A100 cards that often run these models.

While this was an important step forward for the research community, it became a huge one for the hacker community when [Georgi Gerganov] rolled in. He released llama.cpp on GitHub, which runs the inference of a LLaMa model with 4-bit quantization. His code was focused on running LLaMa-7B on your Macbook, but we’ve seen versions running on smartphones and Raspberry Pis. There’s even a version written in Rust! A rough rule of thumb is anything with more than 4 GB of RAM can run LLaMa. Model weights are available through Meta with some rather strict terms, but they’ve been leaked online and can be found even in a pull request on the GitHub repo itself. Continue reading “Why LLaMa Is A Big Deal” →

BitTorrent For Language Models

March 21, 2023 by Al Williams 6 Comments

In the old days of the Internet, FTP was sufficient for downloading the occasional file. But with the widespread use of computer audio and video, it was easy to swamp an FTP server so — eventually — BitTorrent was born. The idea was you would download bits and pieces of a file from different places and, in theory, people would download bits and pieces that you have if they need them. Now Petals wants to use this same method with language models. These AI language models are all the rage, but they take significant computer resources. The idea behind Petals is like BitTorrent. You handle a small part of the model (about 8 gigabytes which is small compared to the 352 gigabytes required), and other people have other parts.

Of course, if you are privacy-minded, that means that some amount of your data is going out to the public, but for your latest chatbot experiments, that might not be a big problem. You can install Petals in an Anaconda environment or run a Docker image if you don’t want to set up anything. If you just want to access the distributed network’s chatbot based on BLOOMZ-176B, you can do that online.

Continue reading “BitTorrent For Language Models” →

ChatGPT, Bing, And The Upcoming Security Apocalypse

March 4, 2023 by Elliot Williams 46 Comments

Most security professionals will tell you that it’s a lot easier to attack code systems than it is to defend them, and that this is especially true for large systems. The white hat’s job is to secure each and every point of contact, while the black hat’s goal is to find just one that’s insecure.

Whether black hat or white hat, it also helps a lot to know how the system works and exactly what it’s doing. When you’ve got the source code, either because it’s open-source, or because you’re working inside the company that makes the software, you’ve got a huge advantage both in finding bugs and in fixing them. In the case of closed-source software, the white hats arguably have the offsetting advantage that they at least can see the source code, and peek inside the black box, while the attackers cannot.

Still, if you look at the number of security issues raised weekly, it’s clear that even in the case of closed-source software, where the defenders should have the largest advantage, that offense is a lot easier than defense.

So now put yourself in the shoes of the poor folks who are going to try to secure large language models like ChatGPT, the new Bing, or Google’s soon-to-be-released Bard. They don’t understand their machines. Of course they know how the work inside, in the sense of cross multiplying tensors and updating weights based on training sets and so on. But because the billions of internal parameters interact in incomprehensible ways, almost all researchers refer to large language models’ inner workings as a black box.

And they haven’t even begun to consider security yet. They’re still worried about how to construct obscure background prompts that prevent their machines from spewing hate speech or pornographic novels. But as soon as the machines start doing something more interesting than just providing you plain text, the black hats will take notice, and someone will have to figure out defense.

Indeed, this week, we saw the first real shot across the bow: a hack to make Bing direct users to arbitrary (bad) webpages. The Bing hack requires the user to already be on a compromised website, so it’s maybe not very threatening, but it points out a possible real security difference between Bing and ChatGPT: Bing gives you links to follow, and that makes it a juicy target.

We’re right on the edge of a new security landscape, because even the white hats are facing a black box in the AI. So far, what ChatGPT and Codex and other large language models are doing is trivially secure – putting out plain text – but Bing is taking the first dangerous steps into doing something more useful, both for users and black hats. Given the ease with which people have undone OpenAI’s attempts to keep ChatGPT in its comfort zone, my guess is that the white hats will have their hands full, and the black-box nature of the model deprives them of their best hope. Buckle your seatbelts.

Hackaday Links: February 12, 2023

February 12, 2023 by Dan Maloney 24 Comments

So, maybe right now isn’t the best time to get into the high-altitude ballooning hobby? At least in the US, which with the downing of another — whatever? — over Alaska, seems to have taken a “Sidewinders first, threat identification later” approach to anything that floats by. The latest incident involved an aircraft of unknown type, described as “the size of a small car” — there’s that units problem again — that was operating over Prudhoe Bay off the northern coast of Alaska. The reason that was given for this one earning a Sidewinder was that it was operating much lower than the balloon from last week, only about 40,000 feet, which is well within the ceiling of commercial aviation. It was also over sea ice at the time of the shootdown, making the chance of bothering anyone besides a polar bear unlikely. We’re not taking any political position on this whole thing, but there certainly are engineering and technical aspects of these shootdowns that are pretty interesting, as well as the aforementioned potential for liability if your HAB goes astray. Nobody ever really benefits from having an international incident on their resume, after all.

Continue reading “Hackaday Links: February 12, 2023” →

Does Programming A Robot With ChatGPT Work At All?

February 9, 2023 by Lewin Day 41 Comments

ChatGPT has been put to all manner of silly uses since it first became available online. [Engineering After Hours] decided to see if its coding skills were any chop, and put it to work programming a circular saw. Pun intended.

The aim was to build a line following robot armed with a circular saw to handle lawn edging tasks. The circular saw itself consists of a motor with a blade on it, and precisely no safety features. It’s mounted on the front of a small RC car with a rack and pinion to control its position. [Engineering After Hours] has some sage advice in this area: don’t try this at home.

ChatGPT was not only able to give advice on what parts to use, it was able to tell [Engineering After Hours] on how to hook everything up to an Arduino and even write the code. The AI language model even recommended a PID loop to control the position of the circular saw. Initial tests were messy, but some refinement got things impressively functional.

As a line following robot, the performance is pretty crummy. However, as a robot programmed by an AI, it does pretty okay. Obviously, it’s hard to say how much help the AI had, and how many corrections [Engineering After Hours] had to make to the code to get everything working. But the fact that this kind of project is even possible shows us just how far AI has really come.

Continue reading “Does Programming A Robot With ChatGPT Work At All?” →

With ChatGPT, Game NPCs Get A Lot More Interesting

February 8, 2023 by Donald Papp 24 Comments

Not only is AI-driven natural language processing a thing now, but you can even select from a number of different offerings, each optimized for different tasks. It took very little time for [Bloc] to mod a computer game to allow the player to converse naturally with non-player characters (NPCs) by hooking it into ChatGPT, a large language model AI optimized for conversational communication.

If you can look past the painfully-long loading times, even buying grain (7:36) gains a new layer of interactivity.

[Bloc] modified the game Mount & Blade II: Bannerlord to reject traditional dialogue trees and instead accept free-form text inputs, using ChatGPT on the back end to create more natural dialogue interactions with NPCs. This is a refinement of an earlier mod [Bloc] made and shared, so what you see in the video below is quite a bit more than a proof of concept. The NPCs communicate as though they are aware of surrounding events and conditions in the game world, are generally less forthcoming when talking to strangers, and the new system can interact with game mechanics and elements such as money, quests, and hirelings.

Starting around 1:08 into the video, [Bloc] talks to a peasant about some bandits harassing the community, and from there demonstrates hiring some locals and haggling over prices before heading out to deal with the bandits.

The downside is that ChatGPT is currently amazingly popular. As a result, [Bloc]’s mod is stuck using an overloaded service which means some painfully-long load times between each exchange. But if you can look past that, it’s a pretty fascinating demonstration of what’s possible by gluing two systems together with a mod and some clever coding.

Take a few minutes to check out the video, embedded below. And if you’re more of a tabletop gamer? Let us remind you that it might be fun to try replacing your DM with ChatGPT.

Continue reading “With ChatGPT, Game NPCs Get A Lot More Interesting” →