A ChatGPT client running on an IBM Portable PC

MS-DOS Client Brings ChatGPT To The IBM PC

AI-powered chatbots are clearly the future of computing, and it’s only a matter of time before you’ll see them appear on every internet-connected gadget. If you thought you were safe from this by sticking to an ancient MS-DOS PC though, think again: [Yeo Kheng Meng] has recently written a ChatGPT client that runs on DOS.

[Yeo Kheng Meng] didn’t cheat by simply running MS-DOS on a modern PC, either: he tested the client on a real 1984 vintage IBM 5155 Portable PC. This semi-portable PC/XT model sports a 4.77 MHz 8088 CPU, 640 kB of RAM and a CGA video card with a built-in monochrome monitor. An NE2000 ISA network card, running in 8-bit mode, enables the Portable to connect to the internet.

Running the client couldn’t be simpler: just run doschgpt.exe and type in your question. [Yeo Kheng Meng] developed this program using the Open Watcom C/C++ compiler, which was the compiler of choice for most DOS game developers back in the day. Networking support was provided by an era-appropriate packet driver together with MTCP, a TCP/IP stack developed by [Michael Brutman] for DOS-based internet applications.

Connecting to the ChatGPT API and parsing the results was pretty straightforward, but implementing the required TLS encryption was not. Even if there was a library available for MS-DOS, the 5155 wouldn’t have enough CPU power to run it in real time, so [Yeo Kheng Meng] decided to run that bit of the networking stack on a modern PC and send an unencrypted HTTP stream to the DOS client.

The end result is a delightful retro-futuristic setup that seems to have come straight out of a 1980s science fiction movie. We can already picture it together with a Commodore 64 reporting the news and an IRC server running on an IBM PC. Continue reading “MS-DOS Client Brings ChatGPT To The IBM PC”

How Much Programming Can ChatGPT Really Do?

By now we’ve all seen articles where the entire copy has been written by ChatGPT. It’s essentially a trope of its own at this point, so we will start out by assuring you that this article is being written by a human. AI tools do seem poised to be extremely disruptive to certain industries, though, but this doesn’t necessarily have to be a bad thing as long as they continue to be viewed as tools, rather than direct replacements. ChatGPT can be used to assist in plenty of tasks, and can help augment processes like programming (rather than becoming the programmer itself), and this article shows a few examples of what it might be used for.

AI comments are better than nothing…probably.

While it can write some programs on its own, in some cases quite capably, for specialized or complex tasks it might not be quite up to the challenge yet. It will often appear extremely confident in its solutions even if it’s providing poor or false information, though, but that doesn’t mean it can’t or shouldn’t be used at all.

The article goes over a few of the ways it can function more as an assistant than a programmer, including generating filler content for something like an SQL database, converting data from one format to another, converting programs from one language to another, and even help with a program’s debugging process.

Some other things that ChatGPT can be used for that we’ve been able to come up with include asking for recommendations for libraries we didn’t know existed, as well as asking for music recommendations to play in the background while working. Tools like these are extremely impressive, and while they likely aren’t taking over anyone’s job right now, that might not always be the case.

Why LLaMa Is A Big Deal

You might have heard about LLaMa or maybe you haven’t. Either way, what’s the big deal? It’s just some AI thing. In a nutshell, LLaMa is important because it allows you to run large language models (LLM) like GPT-3 on commodity hardware. In many ways, this is a bit like Stable Diffusion, which similarly allowed normal folks to run image generation models on their own hardware with access to the underlying source code. We’ve discussed why Stable Diffusion matters and even talked about how it works.

LLaMa is a transformer language model from Facebook/Meta research, which is a collection of large models from 7 billion to 65 billion parameters trained on publicly available datasets. Their research paper showed that the 13B version outperformed GPT-3 in most benchmarks and LLama-65B is right up there with the best of them. LLaMa was unique as inference could be run on a single GPU due to some optimizations made to the transformer itself and the model being about 10x smaller. While Meta recommended that users have at least 10 GB of VRAM to run inference on the larger models, that’s a huge step from the 80 GB A100 cards that often run these models.

While this was an important step forward for the research community, it became a huge one for the hacker community when [Georgi Gerganov] rolled in. He released llama.cpp on GitHub, which runs the inference of a LLaMa model with 4-bit quantization. His code was focused on running LLaMa-7B on your Macbook, but we’ve seen versions running on smartphones and Raspberry Pis. There’s even a version written in Rust! A rough rule of thumb is anything with more than 4 GB of RAM can run LLaMa. Model weights are available through Meta with some rather strict terms, but they’ve been leaked online and can be found even in a pull request on the GitHub repo itself. Continue reading “Why LLaMa Is A Big Deal”

BitTorrent For Language Models

In the old days of the Internet, FTP was sufficient for downloading the occasional file. But with the widespread use of computer audio and video, it was easy to swamp an FTP server so — eventually — BitTorrent was born. The idea was you would download bits and pieces of a file from different places and, in theory, people would download bits and pieces that you have if they need them. Now Petals wants to use this same method with language models. These AI language models are all the rage, but they take significant computer resources. The idea behind Petals is like BitTorrent. You handle a small part of the model (about 8 gigabytes which is small compared to the 352 gigabytes required), and other people have other parts.

Of course, if you are privacy-minded, that means that some amount of your data is going out to the public, but for your latest chatbot experiments, that might not be a big problem. You can install Petals in an Anaconda environment or run a Docker image if you don’t want to set up anything. If you just want to access the distributed network’s chatbot based on BLOOMZ-176B, you can do that online.

Continue reading “BitTorrent For Language Models”

ChatGPT, Bing, And The Upcoming Security Apocalypse

Most security professionals will tell you that it’s a lot easier to attack code systems than it is to defend them, and that this is especially true for large systems. The white hat’s job is to secure each and every point of contact, while the black hat’s goal is to find just one that’s insecure.

Whether black hat or white hat, it also helps a lot to know how the system works and exactly what it’s doing. When you’ve got the source code, either because it’s open-source, or because you’re working inside the company that makes the software, you’ve got a huge advantage both in finding bugs and in fixing them. In the case of closed-source software, the white hats arguably have the offsetting advantage that they at least can see the source code, and peek inside the black box, while the attackers cannot.

Still, if you look at the number of security issues raised weekly, it’s clear that even in the case of closed-source software, where the defenders should have the largest advantage, that offense is a lot easier than defense.

So now put yourself in the shoes of the poor folks who are going to try to secure large language models like ChatGPT, the new Bing, or Google’s soon-to-be-released Bard. They don’t understand their machines. Of course they know how the work inside, in the sense of cross multiplying tensors and updating weights based on training sets and so on. But because the billions of internal parameters interact in incomprehensible ways, almost all researchers refer to large language models’ inner workings as a black box.

And they haven’t even begun to consider security yet. They’re still worried about how to construct obscure background prompts that prevent their machines from spewing hate speech or pornographic novels. But as soon as the machines start doing something more interesting than just providing you plain text, the black hats will take notice, and someone will have to figure out defense.

Indeed, this week, we saw the first real shot across the bow: a hack to make Bing direct users to arbitrary (bad) webpages. The Bing hack requires the user to already be on a compromised website, so it’s maybe not very threatening, but it points out a possible real security difference between Bing and ChatGPT: Bing gives you links to follow, and that makes it a juicy target.

We’re right on the edge of a new security landscape, because even the white hats are facing a black box in the AI. So far, what ChatGPT and Codex and other large language models are doing is trivially secure – putting out plain text – but Bing is taking the first dangerous steps into doing something more useful, both for users and black hats. Given the ease with which people have undone OpenAI’s attempts to keep ChatGPT in its comfort zone, my guess is that the white hats will have their hands full, and the black-box nature of the model deprives them of their best hope. Buckle your seatbelts.

Hackaday Links Column Banner

Hackaday Links: February 12, 2023

So, maybe right now isn’t the best time to get into the high-altitude ballooning hobby? At least in the US, which with the downing of another — whatever? — over Alaska, seems to have taken a “Sidewinders first, threat identification later” approach to anything that floats by. The latest incident involved an aircraft of unknown type, described as “the size of a small car” — there’s that units problem again — that was operating over Prudhoe Bay off the northern coast of Alaska. The reason that was given for this one earning a Sidewinder was that it was operating much lower than the balloon from last week, only about 40,000 feet, which is well within the ceiling of commercial aviation. It was also over sea ice at the time of the shootdown, making the chance of bothering anyone besides a polar bear unlikely. We’re not taking any political position on this whole thing, but there certainly are engineering and technical aspects of these shootdowns that are pretty interesting, as well as the aforementioned potential for liability if your HAB goes astray. Nobody ever really benefits from having an international incident on their resume, after all.

Continue reading “Hackaday Links: February 12, 2023”

Does Programming A Robot With ChatGPT Work At All?

ChatGPT has been put to all manner of silly uses since it first became available online. [Engineering After Hours] decided to see if its coding skills were any chop, and put it to work programming a circular saw. Pun intended.

The aim was to build a line following robot armed with a circular saw to handle lawn edging tasks.  The circular saw itself consists of a motor with a blade on it, and precisely no safety features. It’s mounted on the front of a small RC car with a rack and pinion to control its position. [Engineering After Hours] has some sage advice in this area: don’t try this at home.

ChatGPT was not only able to give advice on what parts to use, it was able to tell [Engineering After Hours] on how to hook everything up to an Arduino and even write the code. The AI language model even recommended a PID loop to control the position of the circular saw. Initial tests were messy, but some refinement got things impressively functional.

As a line following robot, the performance is pretty crummy. However, as a robot programmed by an AI, it does pretty okay. Obviously, it’s hard to say how much help the AI had, and how many corrections [Engineering After Hours] had to make to the code to get everything working. But the fact that this kind of project is even possible shows us just how far AI has really come.

Continue reading “Does Programming A Robot With ChatGPT Work At All?”