Want your next project to trash talk? Dynamically rewrite boring log messages as sci-fi technobabble? Happily (or grudgingly) answer questions? Doing that sort of thing and more can be done with OpenAI’s GPT-3, a natural language prediction model with an API that is probably a lot easier to use than you might think.

In fact, if you have basic Python coding skills, or even just the ability to craft a curl statement, you have just about everything you need to add this ability to your next project. It’s not free in the long run, although initial use is free on signup, but for personal projects the costs will be very small.

Basic Concepts

OpenAI has an API that provides access to GPT-3, a machine learning model with the ability to perform just about any task that involves understanding or generating natural-sounding language.

OpenAI provides some excellent documentation as well as a web tool through which one can experiment interactively. First, however, one must create an account and receive an API key. After that is done, the doors are open.

Creating an account also gives one a number of free credits that can be used to experiment with ideas. Once the free trial is used up or expires, using the API will cost money. How much? Not a lot, frankly. Everything sent to (and received from) the API is broken into tokens, and pricing is from $0.0008 to $0.06 per thousand tokens. A thousand tokens is roughly 750 words, so small projects are really not a big financial commitment. My free trial came with 18 USD of credits, of which I have so far barely managed to spend 5%.

Let’s take a closer look at how it works, and what can be done with it!

How It Works

The API accepts requests in a variety of ways, and if you can craft a curl statement, use the command line, or write some simple Python (or node.js) code, good news! You have all you need to start trying ideas!

I will describe using the API in its most basic way, that of completion. That means one presents the API with a prompt, from which it will provide a text completion that attempts to match the prompt. All of this is done entirely in text, and formatted as natural language.

Using the API in this way can be thought of as implementing a kind of highly-capable autocomplete. The engine itself can be thought of as a sort of fantastically-complex state machine, while at the same time it is also not quite like anything else.

What Kind of Stuff is Possible?

Here’s a very simple example:

The prompt is also how one “programs” the model, and designing a good prompt is a big part of getting good results.

Because the model is capable of understanding language structure and intent, a prompt can be worded as if one were giving a writing assignment to a middle schooler.

Here is a slightly more complex example that does exactly that:

There are a few things to point out in the above example. The first line of the prompt describes what is to be done. The next two lines (“Event:” and “Commentary:”) set the model up on how to the statement should be completed. Note that the “Commentary:” line is left blank in the prompt; the model will complete what was started.

The practice of crafting effective prompts is called prompt design, and it is key to getting all sorts of useful and different responses from the API. Let’s take a closer look at how prompt design affects results.

Getting Better Responses

If one is having trouble getting the model to respond in a useful way, it can be helpful to include multiple examples of what is wanted in the prompt itself to give the model a better idea of what is expected.

The API documentation has an excellent demonstration of this idea, and I will briefly restate it below:

This result isn’t really what we want. However, including some examples in the prompt itself yields markedly better results:

Doing this increases costs — recall that one pays per token, both in the prompt as well as in the output — but providing multiple examples in the prompt can be key to getting the best results in some cases, because it gives the model a much clearer idea of what is being requested, and how it should be formatted.

Again, it is helpful to think of the prompt as a writing assignment for a middle-schooler; a middle-schooler who can in turn be thought of as a fantastically-complex and somewhat variable state machine.

Same Prompt, Different Completions

For an identical prompt, the API doesn’t necessarily return the same results. While the nature of the prompt and the data the model has been trained on play a role, diversity of responses can also be affected by the temperature setting in a request.

Temperature is a value between 0 and 1, and is an expression of how deterministic the model should be when making predictions about valid completions to a prompt. A temperature of 0 means that submitting the same prompt will result in the same (or very similar) responses each time. A temperature above zero will yield different completions each time.

Put another way, a lower temperature means the model takes fewer risks, resulting in completions that are more deterministic. This is useful when one wants completions that can be accurately predicted, such as responses that are factual in nature. On the other hand, increasing the temperature — 0.7 is a typical default value — yields more diversity in completions.

Fine Tuning the Model

The natural language model behind the API is pre-trained, but it is still possible to customize the model with a separate dataset tailored for a particular application.

This function, called fine tuning, allows one to efficiently provide the model with many more examples than it would be practical to include in each prompt. In fact, once a fine tuning dataset has been provided, one no longer needs to include examples in the prompt itself. Requests will be processed faster, as well.

This probably won’t be needed except for narrow applications, but if you find that getting solid results for your project is relying on large prompts and you’d like it to be more efficient, fine tuning is where you need to look. OpenAI provides tools to make this process as easy as possible, should you require it.

What Does The Code Look Like?

There is an interactive web tool (the playground, requires an account) in which one can use the model to test ideas without having to code something up, but it also has the handy feature of generating a code snippet upon request, for easy copy and pasting into projects.

Here is the very first example in this article, formatted as a simple curl request:

curl https://api.openai.com/v1/engines/text-davinci-002/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
  "prompt": "Write a witty error message for a low battery condition.",
  "temperature": 0.7,
  "max_tokens": 256,
  "top_p": 1,
  "frequency_penalty": 0,
  "presence_penalty": 0
}'

And the same, this time in Python:

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

response = openai.Completion.create(
engine="text-davinci-002",
prompt="Write a witty error message for a low battery condition.",
temperature=0.7,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)

Installing the python package will also install a utility that can be used directly from the command line for maximum convenience:

$ openai api completions.create -e text-davinci-002 -p "Write a simple poem about daisies." --stream -M 128

(Notes: --stream displays results as they are received, and -M 128 limits the reply to a maximum of 128 tokens.)

The prompt “write a simple poem about daisies” generated the following text for me, which would be different every time:

The daisy is a beautiful flower 
That grows in the meadow and in the pasture 
It has a yellow center and white petals 
That make it look like the sun 
The daisy is a symbol of innocence 
And purity and is loved by all

All of the above examples work the same way: they fire the prompt off to the OpenAI API (using one’s API key for access, which the above examples assume has been set as an environment variable named OPENAI_API_KEY), and receive a reply with the response.

Responsible Use

Worth highlighting is OpenAI’s commitment to responsible use, including guidance on safety best practices for applications. There is a lot of thoughtful information in that link, but the short version is to always keep in mind that this is a tool that is:

1. Capable of making things up in a very believable way, and
2. Capable of interacting with people.

It’s not hard to see that the combination has potential for harm if used irresponsibly. Like most tools, one should be mindful of misuse, but tools can be wonderful things as well.

Are You Getting Ideas Yet?

Using the API isn’t free in the long term, but creating an account will give you a set of free credits that can be used to play around and try a few ideas out, and using even the most expensive engine for personal projects costs a pittance. All of my enthusiastic experimentation has so far used barely two dollars USD worth of my free trial.

Need some inspiration? We’ve covered a few projects already that have waded in this direction. This robotic putting game uses natural language AI to generate trash talk, and the Deep Dreams podcast consists entirely of machine-generated fairytales as a sleep aid, and was created with the OpenAI API.

Now that you know what kinds of things are possible and how easy they are, perhaps you are already getting some ideas? Let us know about them in the comments!