large language model

16 Articles

Bridging A Gap Between LLMs And Programming With TypeChat

July 22, 2023 by 33 Comments

By now, large language models (LLMs) like OpenAI’s ChatGPT are old news. While not perfect, they can assist with all kinds of tasks like creating efficient Excel spreadsheets, writing cover letters, asking for music references, and putting together functional computer programs in a variety of languages. One thing these LLMs don’t do yet though is integrate well with existing app interfaces. However, that’s where the TypeChat library comes in, bridging the gap between LLMs and programming.

TypeChat is an experimental MIT-licensed library from Microsoft which sits in between a user and a LLM and formats responses from the AI that are type-safe so that they can easily be plugged back in to the original interface. It does this by generating JSON responses based on user input, making it easier to take the user input directly, run it through the LLM, and then use the output directly in another piece of code. It can be used for things like prototyping prompts, validating responses, and handling errors. It’s also not limited to a single LLM and can be fairly easily modified to work with many of the existing models.

The software is still in its infancy but does hope to make it somewhat easier to work between user inputs within existing pieces of software and LLMs which have quickly become all the rage in the computer science world. We expect to see plenty more tools like this become available as more people take up using these new tools, which have plenty of applications beyond just writing code.

Closeup of an Apple ][ terminal program. The background is blue and the text white. The prompt says, "how are you today?" and the ChatGPT response says, "As an AI language model, I don't have feelings, but I am functioning optimally. Thank you for asking. How may I assist you?"

Apple II – Now With ChatGPT

June 13, 2023 by 10 Comments

Hackers are finding no shortage of new things to teach old retrocomputers, and [Evan Michael] has taught his Apple II how to communicate with ChatGPT.

Written in Python, iiAI lets an Apple II access everyone’s favorite large language model (LLM) through the terminal. The program lives on a more modern computer and is accessed over a serial connection. OpenAI API credentials are stored in a file invoked by iiAI when you launch it by typing python3 openai_apple.py. The program should work on any device that supports TTY serial, but so far testing has only happened on [Michael]’s Apple IIGS.

For a really clean setup, you might try running iiAI internally on an Apple II Pi. ChatGPT has also found its way onto Commodore 64 and MS-DOS, and look here if you’d like some more info on how these AI chat bots work anyway.

Continue reading “Apple II – Now With ChatGPT” →

C64 Gets ChatGPT Access Via BBS

June 2, 2023 by 4 Comments

ChatGPT, powered by GPT 3.5 and GPT 4, has become one of the most popular Large Language Models (LLM), due to its ability to hold passable conversations and generate large tracts of text. Now, that very tool is available on the Commodore 64 via the Internet.

Obviously, a 6502 CPU with just 64 kilobytes of RAM can barely remember a dictionary, let alone the work with something as complicated as a modern large language model. Nor is the world’s best-selling computer well-equipped to connect to modern online APIs. Instead, the C64 can access ChatGPT through the Retrocampus BBS, as demonstrated by [Retro Tech or Die].

Due to security reasons, the ChatGPT area of the BBS is only available to the board’s Patreon members. Once in, though, you’re granted a prompt with ChatGPT displayed in glorious PETSCII on the Commodore 64. It’s all handled via a computer running as a go-between for the BBS clients and OpenAI’s ChatGPT service, set up by board manager [Francesco Sblendorio]. It’s particularly great to see ChatGPT spitting out C64-compatible BASIC.

While this is a fun use of ChatGPT, be wary of using it for certain tasks in wider society. Video after the break.

Continue reading “C64 Gets ChatGPT Access Via BBS” →

ChatGPT V. The Legal System: Why Trusting ChatGPT Gets You Sanctioned

May 29, 2023 by 66 Comments

Recently, an amusing anecdote made the news headlines pertaining to the use of ChatGPT by a lawyer. This all started when a Mr. Mata sued the airline where years prior he claims a metal serving cart struck his knee. When the airline filed a motion to dismiss the case on the basis of the statute of limitations, the plaintiff’s lawyer filed a submission in which he argued that the statute of limitations did not apply here due to circumstances established in prior cases, which he cited in the submission.

Unfortunately for the plaintiff’s lawyer, the defendant’s counsel pointed out that none of these cases could be found, leading to the judge requesting the plaintiff’s counsel to submit copies of these purported cases. Although  the plaintiff’s counsel complied with this request, the response from the judge (full court order PDF) was a curt and rather irate response, pointing out that none of the cited cases were real, and that the purported case texts were bogus.

The defense that the plaintiff’s counsel appears to lean on is that ChatGPT ‘assisted’ in researching these submissions, and had assured the lawyer – Mr. Schwartz – that all of these cases were real. The lawyers trusted ChatGPT enough to allow it to write an affidavit that they submitted to the court. With Mr. Schwartz likely to be sanctioned for this performance, it should also be noted that this is hardly the first time that ChatGPT and kin have been involved in such mishaps.

Continue reading “ChatGPT V. The Legal System: Why Trusting ChatGPT Gets You Sanctioned” →

Leaked Internal Google Document Claims Open Source AI Will Outcompete Google And OpenAI

May 5, 2023 by 37 Comments

In the world of large language models (LLM), the focus has for the longest time been on proprietary technologies from companies such as OpenAI (GPT-3 & 4, ChatGPT, etc.) as well as increasingly everyone from Google to Meta and Microsoft. What’s remained underexposed in this whole discussion about which LLM will do more things better are the efforts by hobbyists, unaffiliated researchers and everyone else you may find in Open Source LLM projects. According to a leaked document from a researcher at Google (anonymous, but apparently verified), Google is very worried that Open Source LLMs will wipe the floor with both Google’s and OpenAI’s efforts.

According to the document, after the open source community got their hands on the leaked LLaMA foundation model, motivated and highly knowledgeable individuals set to work to take a fairly basic model to new levels where it could begin to compete with the offerings by OpenAI and Google. Major innovations are the scaling issues, allowing these LLMs to work on far less powerful systems (like a laptop or even smartphone).

An important factor here is Low-Rank adaptation (LoRa), which massively cuts down the effort and resources required to train a model. Ultimately, as this document phrases it, Google and in extension OpenAI do not have a ‘secret sauce’ that makes their approaches better than anything the wider community can come up with. Noted is also that essentially Meta has won out here by having their LLM leak, as it has meant that the OSS community has been improving on the Meta foundations, allowing Meta to benefit from those improvements in their products.

The dire prediction is thus that in the end the proprietary LLMs by Google, OpenAI and others will cease to be relevant, as the open source community will have steamrolled them into fine, digital dust. Whether this will indeed work out this way remains to be seen, but things are not looking up for proprietary LLMs.

(Thanks to [Mike Szczys] for the tip)

Peering Down Into Talking Ant Hill

April 19, 2023 by 14 Comments

Watching an anthill brings an air of fascination. Thousands of ants are moving about and communicating with other ants as they work towards a goal as a collective whole. For us humans, we project a complex inner world for each of these tiny creatures to drive the narrative. But what if we could peer down into a miniature world and the ants spoke English? (PDF whitepaper)

Researchers at the University of Stanford and Google Research have released a paper about simulating human behavior using multiple Large Language Models (LMM). The simulation has a few dozen agents that can move across the small town, do errands, and communicate with each other. Each agent has a short description to help provide context to the LLM. In addition, they have memories of objects, other agents, and observations that they can retrieve, which allows them to create a plan for their day. The memory is a time-stamped text stream that the agent reflects on, deciding what is important. Additionally, the LLM can replan and figure out what it wants to do.

The question is, does the simulation seem life-like? One fascinating example is the paper’s authors created one agent (Isabella) intending to have a Valentine’s Day party. No other information is included. But several agents arrive at the character’s house later in the day to party. Isabella invited friends, and those agents asked some people.

A demo using recorded data from an earlier demo is web-accessible. However, it doesn’t showcase the powers that a user can exert on the world when running live. Thoughts and suggestions can be issued to an agent to steer their actions. However, you can pause the simulation to view the conversations between agents. Overall, it is incredible how life-like the simulation can be. The language of the conversation is quite formal, and running the simulation burns significant amounts of computing power. Perhaps there can be a subconscious where certain behaviors or observations can be coded in the agent instead of querying the LLM for every little thing (which sort of sounds like what people do).

There’s been an exciting trend of combining LLMs with a form of backing store, like combining Wolfram Alpha with chatGPT. Thanks [Abe] for sending this one in!

BitTorrent For Language Models

March 21, 2023 by 6 Comments

In the old days of the Internet, FTP was sufficient for downloading the occasional file. But with the widespread use of computer audio and video, it was easy to swamp an FTP server so — eventually — BitTorrent was born. The idea was you would download bits and pieces of a file from different places and, in theory, people would download bits and pieces that you have if they need them. Now Petals wants to use this same method with language models. These AI language models are all the rage, but they take significant computer resources. The idea behind Petals is like BitTorrent. You handle a small part of the model (about 8 gigabytes which is small compared to the 352 gigabytes required), and other people have other parts.

Of course, if you are privacy-minded, that means that some amount of your data is going out to the public, but for your latest chatbot experiments, that might not be a big problem. You can install Petals in an Anaconda environment or run a Docker image if you don’t want to set up anything. If you just want to access the distributed network’s chatbot based on BLOOMZ-176B, you can do that online.

Continue reading “BitTorrent For Language Models” →