Streaming Video From An ESP32

September 5, 2023 by Bryan Cockfield 11 Comments

The ESP32, while first thought to be little more than a way of adding wireless capabilities to other microcontrollers, has quickly replaced many of them with its ability to be programmed as its own platform rather than simply an accessory. This also paved the way for accessories of its own, such as various sensors and even a camera. This guide goes over taking the input from the camera and streaming it out over the network to multiple browsers.

On the server side of things, the ESP32 and its attached camera are set up with MQTT, a lightweight communications protocol which uses a publish/subscribe model to send information. The ESP32 is configured to publish its images only, but not subscribe to any other nodes. On the client side, the browser runs a JavaScript program which is able to gather these images and stitch them together into a video.

This can be quite a bit of data to send out over the ESP32’s compact hardware, so there are some tips and tricks for getting more out of these little devices, including using an external antenna for better Wi-Fi signal, or omitting it entirely in favor of Ethernet. As far as getting a lot out of a tiny microcontroller, though, leveraging MQTT really helps the ESP32 go a long way. These chips have come along way since they were first introduced; they’re powerful enough to act as 8-bit gaming consoles too.

Thanks to [Surfskidude] for the tip!

Chatting With Local AI Moves Directly In-Browser, Thanks To Web LLM

April 24, 2023 by Donald Papp 7 Comments

Large Language Models (LLM) are at the heart of natural-language AI tools like ChatGPT, and Web LLM shows it is now possible to run an LLM directly in a browser. Just to be clear, this is not a browser front end talking via API to some server-side application. This is a client-side LLM running entirely in the browser.

The ability to run an LLM (natural language AI) directly in-browser means more ways to implement local AI while enjoying GPU acceleration via WebGPU.

Running an AI system like an LLM locally usually leverages the computational abilities of a graphics card (GPU) to accelerate performance. This is true when running an image-generating AI system like Stable Diffusion, and it’s also true when implementing a local copy of an LLM like Vicuna (which happens to be the model implemented by Web LLM.) The thing that made Web LLM possible is WebGPU, whose release we covered just last month.

WebGPU provides a way for an in-browser application to talk to a local GPU directly, and it sure didn’t take long for someone to get the idea of using that to get a local LLM to run entirely within the browser, complete with GPU acceleration. This approach isn’t just limited to language models, either. The same method has been applied to successfully create Web Stable Diffusion as well.

It’s a fascinating (and fast) development that opens up new possibilities and, hopefully, gives people some new ideas. Check out Web LLM’s GitHub repository for a closer look, as well as access to an online demo.

Surfing The Web Like It’s 1978 — Carbonyl

February 6, 2023 by Jonathan Bennett 29 Comments

[Fathy] gets a kick out of doing odd things with Chromium, and Carbonyl is a clever byproduct of that hobby. In this case, it’s what you get when you connect chrome’s renderer to an SVG output module and then convert that SVG to colored characters on a terminal. See, html2svg is an earlier project, taking Chromium’s Skia engine and plugging it into an SVG back-end. And once you have SVG, why not render it to the terminal?

And the results are actually pretty impressive. Imagery is rendered using Unicode 2584, a half-block character. The background and foreground color can be set per character, giving us two controllable pixels per character. Text is handled a bit differently, rendering using the normal text fonts, making for readable pages.

The source is very much a work in progress, but there are some neat ideas already coalescing around the project, like using sixels for better rendering. There’s already decent mouse support and audio output, making for an impressive terminal experience. This might be a project to keep an eye on.

Classic Calculators Emulated In Browser

January 30, 2023 by Bryan Cockfield 24 Comments

The Multiple Arcade Machine Emulator, now known simply as MAME, started off as a project to emulate various arcade games. The project is still adding new games to its library, but the framework around MAME makes it capable of emulating pretty much any older computer. The computer doesn’t even need to be a gaming-specific machine as the latest batch of retro hardware they’ve added support for is a number of calculators from the 90s and early 00s including a few classics from Texas Instruments.

Since no one is likely to build an arcade cabinet version of a TI-89, all of these retro calculators are instead emulated entirely within a browser. This includes working buttons and functions on an overlay of each of these calculators but also pixel-accurate screen outputs for each of these. The graphing calculators have more of what we would consider a standard computer screen, but even the unique LCDs of some of the more esoteric calculators are accurately replicated as well thanks to the MAME artwork system.

There are a number of calculators implemented under this project with a full list found at this page, and the MAME team has plans to implement more in the future. If you’re looking for something fun to do on a more modern calculator, though, take a look at this build which implements ray tracing on a TI-84 Plus CE.

Thanks to [J. Peterson] for the tip!

PyScript: Python In The Web Browser

June 8, 2022 by Al Williams 25 Comments

A chainsaw can make short work of clearing out the back forty. It can also make a good horror movie. So while some people will say we don’t need another tool to allow more malicious scripting in the browser, we also know that, like any tool, you can use it or abuse it. That tool? PyScript, which is, of course, Python in the browser.

The tool is in the early experimental phase, so the project doesn’t suggest using it in a production environment yet. However, if it works well, the promise is not just that you can write browser-based applications in Python — you’ll have a handy way to reuse existing Python code and even be able to run the same code on the browser that currently runs on the server. This has a lot of implications for improved client/server applications, or cases where you want to be able to run against a local backend when disconnected and a remote backend when you do have a connection. Of course, you can interoperate with JavaScript, too.

Continue reading “PyScript: Python In The Web Browser” →

Linux And C In The Browser

May 28, 2022 by Al Williams 45 Comments

There was a time when trying to learn to write low-level driver or kernel code was hard. You really needed two machines: one to work with, and one to screw up over and over again until you got it right. These days you can just spin up a virtual machine and roll it back every time you totally screw up. Much easier! We don’t think it is all that practical, but [nsommer] has an interesting post about loading up a C compiler and compiling Linux for a virtual machine. What’s different? Oh, the virtual machine is in your browser.

The v86 CPU emulator runs in the browser and looks like a Pentium III computer with the usual hardware. You might think it is slow and it certainly isn’t going to be fast as a rocket, but it does translate machine code into WebAssembly, so performance isn’t as bad as you might think.

The post goes into detail about how to build and create a simple machine web page that hosts v86. Once you cross-compile the kernel you can boot the machine up virtually. The other interesting part is the addition of tcc which is a pretty capable C compiler and much smaller and faster than the very traditional gcc.

The tcc build is tricky because the normal build process compiles the compiler and then uses the same compiler to build the default libraries. When cross-compiling, this doesn’t work well because the library you want for the host compile is different from the library you want to target for the second pass. You’ll see how to work around that in the post. The post continues to show how to do remote debugging and even gets QEMU into the mix. Debugging inside v86 doesn’t seem to work so far. There are more posts on this topic promised.

Honestly, this is one of those things like teaching a chicken to play checkers. It can be done, there’s little practical value, but it is still something to see. On the other hand, if you spend the weekend working through this, your next Linux porting project ought to seem easy by comparison.

Amazing what you can pull off with WebAssembly. If you need a quick introduction, check this one out from [Ben James].

Flashing TI Chips With An ESP

April 3, 2022 by Bryan Cockfield 29 Comments

Texas Instruments is best known to the general public for building obsolete calculators and selling them at extraordinary prices to students, but they also build some interesting (and reasonably-priced) microcontrollers as well. While not as ubiquitous as Atmel and the Arduino platform, they can still be found in plenty of consumer electronics and reprogrammed, and [Aaron] aka [atc1441] demonstrates how to modify them with an ESP32 as an intermediary.

Specifically, the TI chips in this build revolve around the 8051-core microcontrollers, which [Aaron] has found in small e-paper price tags and other RF hardware. He’s using an ESP32 to reprogram the TI chips, and leveraging a web server on the ESP in order to be able to re-flash them over WiFi. Some of the e-paper displays have built-in header pins which makes connecting them to the ESP fairly easy, and once that’s out of the way [Aaron] also provides an entire software library for interacting with these microcontrollers through the browser interface.

Right now the project supports the CC2430, CC2510 and CC1110 variants, but [Aaron] plans to add support for more in the future. It’s a fairly comprehensive build, and much better than buying the proprietary TI programmer, so if you have some of these e-paper displays laying around the barrier to entry has been dramatically lowered. If you don’t have this specific type of display laying around, we’ve seen similar teardowns and repurposing of other e-paper devices in the past as well.

Continue reading “Flashing TI Chips With An ESP” →