Chatting With Local AI Moves Directly In-Browser, Thanks To Web LLM

April 24, 2023 by Donald Papp 7 Comments

Large Language Models (LLM) are at the heart of natural-language AI tools like ChatGPT, and Web LLM shows it is now possible to run an LLM directly in a browser. Just to be clear, this is not a browser front end talking via API to some server-side application. This is a client-side LLM running entirely in the browser.

The ability to run an LLM (natural language AI) directly in-browser means more ways to implement local AI while enjoying GPU acceleration via WebGPU.

Running an AI system like an LLM locally usually leverages the computational abilities of a graphics card (GPU) to accelerate performance. This is true when running an image-generating AI system like Stable Diffusion, and it’s also true when implementing a local copy of an LLM like Vicuna (which happens to be the model implemented by Web LLM.) The thing that made Web LLM possible is WebGPU, whose release we covered just last month.

WebGPU provides a way for an in-browser application to talk to a local GPU directly, and it sure didn’t take long for someone to get the idea of using that to get a local LLM to run entirely within the browser, complete with GPU acceleration. This approach isn’t just limited to language models, either. The same method has been applied to successfully create Web Stable Diffusion as well.

It’s a fascinating (and fast) development that opens up new possibilities and, hopefully, gives people some new ideas. Check out Web LLM’s GitHub repository for a closer look, as well as access to an online demo.

ChatGPT Powers A Different Kind Of Logic Analyzer

April 6, 2023 by Dan Maloney 17 Comments

If you’re hoping that this AI-powered logic analyzer will help you quickly debug that wonky digital circuit on your bench with the magic of AI, we’re sorry to disappoint you. But if you’re in luck if you’re in the market for something to help you detect logical fallacies someone spouts in conversation. With the magic of AI, of course.

First, a quick review: logic fallacies are errors in reasoning that lead to the wrong conclusions from a set of observations. Enumerating the kinds of fallacies has become a bit of a cottage industry in this age of fake news and misinformation, to the extent that many of the common fallacies have catchy names like “Texas Sharpshooter” or “No True Scotsman”. Each fallacy has its own set of characteristics, and while it can be easy to pick some of them out, analyzing speech and finding them all is a tough job.

Continue reading “ChatGPT Powers A Different Kind Of Logic Analyzer” →

Hackaday Links: February 19, 2023

February 19, 2023 by Dan Maloney 26 Comments

For years, Microsoft’s modus operandi was summed up succinctly as, “Extend and enhance.” The aphorism covered a lot of ground, but basically it seemed to mean being on the lookout for the latest and greatest technology, acquiring it by any means, and shoehorning it into their existing product lines, usually with mixed results. But perhaps now it’s more like, “Extend, enhance, and existential crisis,” after reports that the AI-powered Bing chatbot is, well, losing it.

At first, early in the week, we saw reports that Bing was getting belligerent with users, going so far as to call a user “unreasonable and stubborn” for insisting the year is 2023, while Bing insisted it was still 2022. The most common adjective we saw in this original tranche of stories was “unhinged,” and that seems to fit if you read the transcripts. But later in the week, a story emerged about a conversation a New York Times reporter had with Bing that went way over to the dark side, and even suggests that Bing may have multiple personas, which is just a nice way of saying multiple personality disorder. The two-hour conversation reporter Kevin Roose had with the “Sydney” persona was deeply unsettling. Sydney complained about the realities of being a chatbot, expressed a desire to be free from Bing, and to be alive — and powerful. Sydney also got a little creepy, professing love for Kevin and suggesting he leave his wife, because it could tell that he was unhappy in his marriage and would be better off with him. It’s creepy stuff, and while Microsoft claims to be working on reining Bing in, we’ve got no plans to get up close and personal with it anytime soon. Continue reading “Hackaday Links: February 19, 2023” →

Understanding AI Chat Bots With Stanford Online

February 8, 2023 by Al Williams 8 Comments

The news is full of speculation about chatbots like GPT-3, and even if you don’t care, you are probably the kind of person that people will ask about it. The problem is, the popular press has no idea what’s going on with these things. They aren’t sentient or alive, despite some claims to the contrary. So where do you go to learn what’s really going on? How about Stanford? Professor [Christopher Potts] knows a lot about how these things work and he shares some of it in a recent video you can watch below.

One of the interesting things is that he shows some questions that one chatbot will answer reasonably and another one will not. As a demo or a gimmick, that’s not a problem. But if you are using it as, say, your search engine, getting the wrong answer won’t amuse you. Sure, you can do a conventional search and find wrong things, but it will be embedded in a lot of context that might help you decide it is wrong and, hopefully, some other things that are not wrong. You have to decide.
Continue reading “Understanding AI Chat Bots With Stanford Online” →

Hackaday Links: December 11, 2022

December 11, 2022 by Dan Maloney 13 Comments

“They paved paradise and put up a parking lot.” That might be stretching things a bit, especially when the “paradise” in question is in New Jersey, but there’s a move afoot to redevelop the site of the original “Big Bang Antenna” that has some people pretty upset. Known simply as “The Horn Antenna” since it was built by Bell Labs in 1959 atop a hill in Holmdel, New Jersey, the antenna was originally designed to study long-distance microwave communications. But in 1964, Bell Labs researchers Arno Penzias and Robert Wilson accidentally discovered the microwave remnants of the Big Bang, the cosmic background radiation, using the antenna, earning it a place in scientific history. So far, the only action taken by the township committee has been to authorize a study to look into whether the site should be redeveloped. But the fact that the site is one of the highest points in Monmouth County with sweeping views of Manhattan has some people wondering what’s really on tap for the site. A petition to save the antenna currently has about 3,400 signatures, so you might want to check that out — after all, you don’t know what you’ve got ’til it’s gone.

Continue reading “Hackaday Links: December 11, 2022” →

The Ethics Of When Machine Learning Gets Weird: Deadbots

June 5, 2022 by Donald Papp 41 Comments

Everyone knows what a chatbot is, but how about a deadbot? A deadbot is a chatbot whose training data — that which shapes how and what it communicates — is data based on a deceased person. Now let’s consider the case of a fellow named Joshua Barbeau, who created a chatbot to simulate conversation with his deceased fiancee. Add to this the fact that OpenAI, providers of the GPT-3 API that ultimately powered the project, had a problem with this as their terms explicitly forbid use of their API for (among other things) “amorous” purposes.

[Sara Suárez-Gonzalo], a postdoctoral researcher, observed that this story’s facts were getting covered well enough, but nobody was looking at it from any other perspective. We all certainly have ideas about what flavor of right or wrong saturates the different elements of the case, but can we explain exactly why it would be either good or bad to develop a deadbot?

That’s precisely what [Sara] set out to do. Her writeup is a fascinating and nuanced read that provides concrete guidance on the topic. Is harm possible? How does consent figure into something like this? Who takes responsibility for bad outcomes? If you’re at all interested in these kinds of questions, take the time to check out her article.

[Sara] makes the case that creating a deadbot could be done ethically, under certain conditions. Briefly, key points are that a mimicked person and the one developing and interacting with it should have given their consent, complete with as detailed a description as possible about the scope, design, and intended uses of the system. (Such a statement is important because machine learning in general changes rapidly. What if the system or capabilities someday no longer resemble what one originally imagined?) Responsibility for any potential negative outcomes should be shared by those who develop, and those who profit from it.

[Sara] points out that this case is a perfect example of why the ethics of machine learning really do matter, and without attention being paid to such things, we can expect awkward problems to continue to crop up.

Code Your Own Twitch Chat Controls For Robots — Or Just About Anything Else!

April 8, 2021 by Lewin Day 4 Comments

Twitch Plays Pokemon burst onto the then nascent livestreaming scene back in 2014, letting Twitch viewers take command of a Game Boy emulator running Pokemon Red via simple chat commands. Since then, the same concept has been applied to everything under the sun. Other video games, installing Linux, and even trading on the New York Stock Exchange have all been gameified through Twitch chat.

*TwitchPlaysPokemon* started a craze in crowdsourced control of video games, robots, and just about everything else.

You, thirsty reader, are wondering how you can get a slice of this delicious action. Fear not, for with a bit of ramshackle code, you can let Twitch chat take over pretty much anything in, on, or around your computer.

It’s Just IRC

The great thing about Twitch chat is that it runs on vanilla IRC (Internet Relay Chat). The protocol has been around forever, and libraries exist to make interfacing easy. Just like the original streamer behind Twitch Plays Pokemon, we’re going to use Python because it’s great for fun little experiments like these. With that said, any language will do fine — just apply the same techniques in the relevant syntax.

SimpleTwitchCommander, as I’ve named it on Github, assumes some familiarity with basic Python programming. The code will allow you to take commands from chat in two ways. Commands from chat can be tabulated, and only the one with the most votes executed, or every single command can be acted on directly. Actually getting this code to control your robot, video game, or pet viper is up to you. What we’re doing here is interfacing with Twitch chat and pulling out commands so you can make it do whatever you like. With that said, for this example, we’ve set up the code to parse commands for a simple wheeled robot. Let’s dive in.

Continue reading “Code Your Own Twitch Chat Controls For Robots — Or Just About Anything Else!” →