Tired Of Web Scraping? Make The AI Do It

April 9, 2023 by Donald Papp 19 Comments

[James Turk] has a novel approach to the problem of scraping web content in a structured way without needing to write the kind of page-specific code web scrapers usually have to deal with. How? Just enlist the help of a natural language AI. Scrapeghost relies on OpenAI’s GPT API to parse a web page’s content, pull out and classify any salient bits, and format it in a useful way.

What makes Scrapeghost different is how data gets organized. For example, when instantiating scrapeghost one defines the data one wishes to extract. For example:

from scrapeghost import SchemaScraper
scrape_legislators = SchemaScraper(
schema={
"name": "string",
"url": "url",
"district": "string",
"party": "string",
"photo_url": "url",
"offices": [{"name": "string", "address": "string", "phone": "string"}],
}
)

The kicker is that this format is entirely up to you! The GPT models are very, very good at processing natural language, and scrapeghost uses GPT to process the scraped data and find (using the example above) whatever looks like a name, district, party, photo, and office address and format it exactly as requested.

It’s an experimental tool and you’ll need an API key from OpenAI to use it, but it has useful features and is certainly a novel approach. There’s a tutorial and even a command-line interface, so check it out.

Let Machine Learning Code An Infinite Variety Of Pong Games

February 21, 2023 by Dan Maloney 7 Comments

In a very real way, Pong started the video game revolution. You wouldn’t have thought so at the time, with its simple gameplay, rudimentary controls, some very low-end sounds, and a cannibalized TV for a display, but the legendarily stuffed coinboxes tell the tale. Fast forward 50 years or so, and Pong has been largely reduced to a programmer’s exercise to see how few lines of code can stand in for what [Ted Dabney] and [Allan Alcorn] accomplished. But now even that’s too much, as OpenAI Codex can generate a playable Pong from just a few prompts, at least most of the time. Continue reading “Let Machine Learning Code An Infinite Variety Of Pong Games” →

With ChatGPT, Game NPCs Get A Lot More Interesting

February 8, 2023 by Donald Papp 24 Comments

Not only is AI-driven natural language processing a thing now, but you can even select from a number of different offerings, each optimized for different tasks. It took very little time for [Bloc] to mod a computer game to allow the player to converse naturally with non-player characters (NPCs) by hooking it into ChatGPT, a large language model AI optimized for conversational communication.

If you can look past the painfully-long loading times, even buying grain (7:36) gains a new layer of interactivity.

[Bloc] modified the game Mount & Blade II: Bannerlord to reject traditional dialogue trees and instead accept free-form text inputs, using ChatGPT on the back end to create more natural dialogue interactions with NPCs. This is a refinement of an earlier mod [Bloc] made and shared, so what you see in the video below is quite a bit more than a proof of concept. The NPCs communicate as though they are aware of surrounding events and conditions in the game world, are generally less forthcoming when talking to strangers, and the new system can interact with game mechanics and elements such as money, quests, and hirelings.

Starting around 1:08 into the video, [Bloc] talks to a peasant about some bandits harassing the community, and from there demonstrates hiring some locals and haggling over prices before heading out to deal with the bandits.

The downside is that ChatGPT is currently amazingly popular. As a result, [Bloc]’s mod is stuck using an overloaded service which means some painfully-long load times between each exchange. But if you can look past that, it’s a pretty fascinating demonstration of what’s possible by gluing two systems together with a mod and some clever coding.

Take a few minutes to check out the video, embedded below. And if you’re more of a tabletop gamer? Let us remind you that it might be fun to try replacing your DM with ChatGPT.

Continue reading “With ChatGPT, Game NPCs Get A Lot More Interesting” →

Giving An Old Typewriter A Mind Of Its Own With GPT-3

January 15, 2023 by Dan Maloney 27 Comments

There was an all-too-brief period in history where typewriters went from clunky, purely mechanical beasts to streamlined, portable electromechanical devices. But when the 80s came around and the PC revolution started, the typewriting was on the wall for these machines, and by the 90s everyone had a PC, a printer, and Microsoft Word. And thus the little daisy-wheel typewriters began to populate thrift shops all over the world.

That’s fine with us, because it gave [Arvind Sanjeev] a chance to build “Ghostwriter”, an AI-powered automatic typewriter. The donor machine was a clapped-out Brother electronic typewriter, which needed a bit of TLC to bring it back to working condition. From there, [Arvind] worked out the keyboard matrix and programmed an Arduino to drive the typewriter, both read and write. A Raspberry Pi running the OpenAI Python API for GPT-3 talks to the Arduino over serial, which basically means you can enter a GPT writing prompt with the keyboard and have the machine spit out a dead-tree version of the results.

To mix things up a bit, [Arvind] added a pair of pots to control the creativity and length of the response, plus an OLED screen which seems only to provide some cute animations, which we don’t hate. We also don’t hate the new paint job the typewriter got, but the jury is still out on the “poetry” that it typed up. Eye of the beholder, we suppose.

Whatever you think of GPT’s capabilities, this is still a neat build and a nice reuse of otherwise dead-end electronics. Need a bit more help building natural language AI into your next project? Our own [Donald Papp] will get you up to speed on that.

Continue reading “Giving An Old Typewriter A Mind Of Its Own With GPT-3” →

This Week In Security: GitHub Actions, SHA-1 Retirement, And A Self-Worming Vulnerability

December 23, 2022 by Jonathan Bennett 6 Comments

It should be no surprise that running untrusted code in a GitHub Actions workflow can have unintended consequences. It’s a killer feature, to automatically run through a code test suite whenever a pull request is opened. But that pull request is run in some part of the target’s development environment, and there’s been a few clever attacks found over the years that take advantage of that. There’s now another one, what Legit Security calls Github Environment Injection, and there were some big-name organizations vulnerable to it.

The crux of the issue is the $GITHUB_ENV file, which contains environment variables to be set in the Actions environment. Individual variables get added to this file as part of the automated action, and that process needs to include some sanitization of data. Otherwise, an attacker can send an environment variable that includes a newline and completely unintended environment variable. And an unintended, arbitrary environment variable is game over for the security of the workflow. The example uses the NODE_OPTIONS variable to dump the entire environment to an accessible output. Any API keys or other secrets are revealed.

This particular attack was reported to GitHub, but there isn’t a practical way to fix it architecturally. So it’s up to individual projects to be very careful about writing untrusted data into the $GITHUB_ENV file.

Continue reading “This Week In Security: GitHub Actions, SHA-1 Retirement, And A Self-Worming Vulnerability” →

Here’s A Plain C/C++ Implementation Of AI Speech Recognition, So Get Hackin’

November 27, 2022 by Donald Papp 19 Comments

[Georgi Gerganov] recently shared a great resource for running high-quality AI-driven speech recognition in a plain C/C++ implementation on a variety of platforms. The automatic speech recognition (ASR) model is fully implemented using only two source files and requires no dependencies. As a result, the high-quality speech recognition doesn’t involve calling remote APIs, and can run locally on different devices in a fairly straightforward manner. The image above shows it running locally on an iPhone 13, but it can do more than that.

Implementing a robust speech transcription that runs locally on a variety of devices is much easier with [Georgi]’s port of OpenAI’s *Whisper*.

[Georgi]’s work is a port of OpenAI’s Whisper model, a remarkably-robust piece of software that does a truly impressive job of turning human speech into text. Whisper is easy to set up and play with, but this port makes it easier to get the system working in other ways. Having such a lightweight implementation of the model means it can be more easily integrated over a variety of different platforms and projects.

The usual way that OpenAI’s Whisper works is to feed it an audio file, and it spits out a transcription. But [Georgi] shows off something else that might start giving hackers ideas: a simple real-time audio input example.

By using a tool to stream audio and feed it to the system every half-second, one can obtain pretty good (sort of) real-time results! This of course isn’t an ideal method, but the robustness and accuracy of Whisper is such that the results look pretty great nevertheless.

You can watch a quick demo of that in the video just under the page break. If it gives you some ideas, head over to the project’s GitHub repository and get hackin’!

Continue reading “Here’s A Plain C/C++ Implementation Of AI Speech Recognition, So Get Hackin’” →

OpenAI Hears You Whisper

September 22, 2022 by Al Williams 18 Comments

Should you wish to try high-quality voice recognition without buying something, good luck. Sure, you can borrow the speech recognition on your phone or coerce some virtual assistants on a Raspberry Pi to handle the processing for you, but those aren’t good for major work that you don’t want to be tied to some closed-source solution. OpenAI has introduced Whisper, which they claim is an open source neural net that “approaches human level robustness and accuracy on English speech recognition.” It appears to work on at least some other languages, too.

If you try the demonstrations, you’ll see that talking fast or with a lovely accent doesn’t seem to affect the results. The post mentions it was trained on 680,000 hours of supervised data. If you were to talk that much to an AI, it would take you 77 years without sleep!

Continue reading “OpenAI Hears You Whisper” →