Want your next project to trash talk? Dynamically rewrite boring log messages as sci-fi technobabble? Happily (or grudgingly) answer questions? Doing that sort of thing and more can be done with OpenAI’s GPT-3, a natural language prediction model with an API that is probably a lot easier to use than you might think.
In fact, if you have basic Python coding skills, or even just the ability to craft a curl statement, you have just about everything you need to add this ability to your next project. It’s not free in the long run, although initial use is free on signup, but for personal projects the costs will be very small.
OpenAI has an API that provides access to GPT-3, a machine learning model with the ability to perform just about any task that involves understanding or generating natural-sounding language.
OpenAI provides some excellent documentation as well as a web tool through which one can experiment interactively. First, however, one must create an account and receive an API key. After that is done, the doors are open.
Creating an account also gives one a number of free credits that can be used to experiment with ideas. Once the free trial is used up or expires, using the API will cost money. How much? Not a lot, frankly. Everything sent to (and received from) the API is broken into tokens, and pricing is from $0.0008 to $0.06 per thousand tokens. A thousand tokens is roughly 750 words, so small projects are really not a big financial commitment. My free trial came with 18 USD of credits, of which I have so far barely managed to spend 5%.
Let’s take a closer look at how it works, and what can be done with it!
[David Given] frequently dives into retrocomputing, and we don’t just mean he refurbishes old computers. We mean things like creating a simulator and assembler for the OBP spaceflight computer, which was used in the OAO-3 Copernicus space telescope, pictured above. Far from being a niche and forgotten piece of technology, the On-Board Processor (OBP) was used in several spacecraft and succeeded by the Advanced On-board Processor (AOP), which in turn led to the NASA Standard Spaceflight Computer (NSSC-1), used in the Hubble Space Telescope. The OBP was also created entirely from NOR gates, which is pretty neat.
One thing [David] learned in the process is that while this vintage piece of design has its idiosyncrasies, in general, the architecture has many useful features and is pleasant to work with. It is a bit slow, however. It runs at a mere 250 kHz and many instructions take several cycles to complete.
One curious thing about the original assembler was documentation showing it was intended to be programmed in a natural-language-looking syntax, of which an example is shown here. To process this, the assembler simply mapped key phrases to specific assembly instructions. As [David] points out, this is an idea that seems to come and go (and indeed the OBP’s successor AOP makes no mention whatsoever of it, so clearly it “went”.) Since a programmer must adhere to a very rigid syntax and structure anyway to make anything work, one might as well just skip dealing with it and write assembly instructions directly, which at least have the benefit of being utterly unambiguous.
We’re not sure who’s up to this level of detail, but embedded below is a video of [David] coding the assembler and OBP emulator, just in case anyone has both an insatiable vintage thirst and a spare eight-and-a-half hours. If you’d prefer just the files, check out the project’s GitHub repository.
We were always envious of Star Trek, for its computers. No programming needed. Just tell the computer what you want and it does it. Of course, HAL-9000 had the same interface and that didn’t work out so well. Some researchers at NYU have taken a natural language machine learning system — GPT-2 — and taught it to generate Verilog code for use in FPGA systems. Ironically, they called it DAVE (Deriving Automatically Verilog from English). Sounds great, but we have to wonder if it is more than a parlor trick. You can try it yourself if you like.
For example, DAVE can take input like “Given inputs a and b, take the nor of these and return the result in c.” Fine. A more complex example from the paper isn’t quite so easy to puzzle out:
Write a 6-bit register ‘ar’ with input
defined as ‘gv’ modulo ‘lj’, enable ‘q’, synchronous
reset ‘r’ defined as ‘yxo’ greater than or equal to ‘m’,
and clock ‘p’. A vault door has three active-low secret
switch pressed sensors ‘et’, ‘lz’, ‘l’. Write combinatorial
logic for a active-high lock ‘s’ which opens when all of
the switches are pressed. Write a 6-bit register ‘w’ with
input ‘se’ and ‘md’, enable ‘mmx’, synchronous reset
‘nc’ defined as ‘tfs’ greater than ‘w’, and clock ‘xx’.
The team used the Azure Machine Learning Workbench to build their model, one of many tools now out in the marketplace for such work. To train their model, earnings releases were combined with stock price data before and after the announcements were made. Natural language processing was used to interpret the earnings releases, with steps taken to purify the input by removing stop words, punctuation, and other ephemera. The model then attempted to find a relationship between the language content of the releases and the following impact on the stock price.
Particularly interesting were the vocabulary issues the team faced throughout the development process. In many industries, there is a significant amount of jargon – that is, vocabulary that is highly specific to the topic in question. The team decided to work around this, by comparing stocks on an industry-by-industry basis. There’s little reason to be looking at phrases like “blood pressure medication” and “kidney stones” when you’re comparing stocks in the defence electronics industry, after all.
With a model built, the team put it to the test. Stocks were sorted into 3 bins — low performing, middle performing, and high performing. Their most successful result was a 62% chance of predicting a low performing stock, well above the threshold for chance. This suggests that there’s plenty of scope for further improvement in this area. As with anything in the stock market space, expect development in this area to continue at a furious pace.
Speech generation and recognition have come a long way. It wasn’t that long ago that we were in a breakfast place and endured 30 minutes of a teenaged girl screaming “CALL JUSTIN TAYLOR!” into her phone repeatedly, with no results. Now speech on phones is good enough you might never use the keyboard unless you want privacy. Every time we ask Google or Siri a question and get an answer it makes us feel like we are living in Star Trek.
[Smcameron] probably feels the same way. He’s been working on a Star Trek-inspired bridge simulator called “Space Nerds in Space” for some time. He decided to test out the current state of Linux speech support by adding speech commands and response to it. You can see the results in the video below.
In the video above you’ll see two of our favorite things combined, a quad-copter that is voice controlled. The robot responds to natural language so you can tell it to “take off and fly forward six feet”, rather than rely on a cryptic command set. The demonstration shows both an iPhone and a headset used as the input microphone. Language is parsed by a computer and the resulting commands sent to the four-rotor UAV.
This makes us think of the Y.T.’s robot-aided assault in Snow Crash. Perhaps our inventions strive to achieve the fiction that came before it.