Often, reprogramming a microcontroller involves placing it in reset, flashing the code, and letting it fire back up. It usually involves shutting the chip down entirely. However, [bor0] has built a virtual machine that runs on the ESP32, allowing for dynamic program updates to happen.
The code is inspired by the CHIP-8, a relatively ancient interpreter that had some gaming applications. [bor0] had already created a VM simulating the CHIP-8, and repurposed it here, taking out the gaming-related drawing instructions and replacing them with those that control IO pins. Registers have also been changed to 16 bits for added flexibility and headroom.
It’s probably not something with immediate ground-breaking applications for most people, but it’s a different way of working with and programming the ESP32, and that’s pretty neat.
The ESP32 is a powerful chip, too, as we all know – and it makes a great 8-bit emulator to boot. Sound off in the comments with your thoughts on what would make a killer application for the ESP32 VM!
[Thanks to satancete for the tip!]
Wasn’t it QNX that advertised that they could update any part of the system (even kernel modules) without rebooting?
Some time in the last millennium that was true. Post-RIMM/Blackberry? Dunno.
I remember reading about that, but I thought it was something different than QNX. Apparently they open-sourced some of QNX back in 2007 (see: http://www.qnx.com/news/pr_2471_1.html). Maybe you can look into it and find interesting things to help.
I’ve done some work towards getting Fuzix working on the ESP32, but it’s tough — the CPU is _extremely_ weird, with a bajillion optional modules all of which need their state saving when you do a context switch. Espressif supply a gigantic and vile #include file full of #ifdefs for doing this but it assumes you’re using their SDK. There is a mini multitasking executive supplied in ROM, which I’m hoping I’ll be able to use to do the heavy lifting, but the RAM requires are extremely undocumented. So far I have the userland building and the kernel will boot up to a point where it trashes its own memory and falls over and I don’t know why it’s doing that.
It does have an MMU, but just like the rest of the system it’s weird. It’s tied to a process controller, where you can program a memory mapping for up to eight threads. However switching between threads is hard (you have to mask NMIs!) and certain bits of hardware only work in certain specified threads. In user mode you can’t access the ROM, which has all the gcc helper routines in it, but in supervisor mode you can’t remap memory. Plus only about half the RAM can be remapped anyway so I’m not sure it’s worth it.
@DavidGiven thanks for posting this. Sounds like some pain we’ve had in the past on other controllers where the inner workings were not well documented. Makes me feel so not alone.
It’s actually quite well documented. The biggest problem is just that it’s really complicated, and these CPUs are so customisable that it’s not at all clear which options are actually enabled — and each option changes things like the exception table layout. The documentation, of course, describes _all_ the options, whether they’re relevant or not. The ESP8266 is actually the same processor, but with fewer options, so it doesn’t support things like register windows. But the ESP32 has these, making context switches are incredibly painful.
It sounds like a very borky pieced-together design.
To me it sounds like something that’s tedious for a reason. I mean, you have all of these complicated instructions anyway, in order to change the program while running you would need to do a series of operations to secure the data you want to keep, pause processing at a specific time, etc. It’s a different way of thinking, kinda-like multi-threading. I don’t have any experience with ESP32’s architecture, but it seems like it would come in really handy for some very specific applications.
There is an OTA library for Arduino, so updating programs in the fly is already possible without using a VM ;-)
Hmmm… I think in the case of the mainstream ESP parts Espressif Systems goes to Tensilica and with their assistance configures their Xtensa LX6 or LX7 IP cores with what they want/need to build their ESP32 product. That’s a whole other layer of documentation and visibility you do not have, it’s proprietary. That level of detail “is” for the most part the e.g. ESP32 “product”. I doubt you could get access to it even under NDA. Without that level of detail, you’re so close to the metal your only option may be to reverse engineer the hardware from the outside. Ugh.
That looks somewhat similar to Toit (https://toit.io) although Toit is a bigger (and commercial) project with a cloud infrastructure and a subscription plan.
Could it be a valid alternative to Toit?
Or, is there software like Toit but open?
Andreas Spiess (The Guy With The Swiss Accent) had a video on Toit on YouTube recently.
Meanwhile when I read Toit I can’t help but think about Dr Evil: “You have a toit body. Toit like a tiger”.
Would you like a smoke and a pancake?
A bong and a blitz?
A lot of Toit is open-source: https://github.com/toitlang/toit.
Note that most of Toit is open-source: https://github.com/toitlang/toit.
I have used esp-link to reprogram or control another esp, or arduino, over IP. It is a serial bridge firmware and works great, allowing you to load new firmware remotely, press the reset button remotely, access the serial console, and more, as if the end device were locally connected.
And how do you do it? Do you have any documentation that you want to share with us?
Theoretically, with a bytecode-interpreting VM and the crudest possible multitasking, you could present an editor over serial and allow for on-device code development. Write bytecode (or even source code, with very simple macro compiling!) in the editor, then flip over and run it on the VM.
Very quickly this starts to look like early unix though. :P
There’s also uLisp, which lets you write, run and edit code on the fly on lots of microcontrollers – including even the Arduino UNO! I really enjoy how makes the dev/compile/test cycle instant, allowing for frictionless exploratory programming.
http://www.ulisp.com/
Forth! Forth allows you to do this, even on an AVR!
Forth is annoying to program though. How about some TCL or even C? Python perhaps?
I just checked, and I think MicroPython already allows you to send interpreted commands via serial
Any idea how easy, or otherwise, it would be to implement Forth on something like a Tang Nano 4k FPGA/MCU (cortex M3) combo SOC?
the micropython port for esp32, has it also
The embedded RISC-V cpu already supports this without much trouble, but it’s clearly, really, really limited.
You can also use Wasm3 to run webassembly on the ESP32, so it’s more or less the same without the troubles.
Guess what?⸘‽!
There is even an ESP*-(Micro/Circuit)Python-byte-code-VM!
And don’t CPM8266 and RunCPM count as VMs?
What about all the Lisps, Forths, Luas, …?
Wasn’t here an ESP(8266)-BASIC too?
So what exactly is the point I’m missing about the VM of this article?
Why should I get excited or surprised now?
Smart me up, Scotty!
I believe it’s about overhead – with this VM it is minimal. You are correct that MicroPython is similar, but programs are essentially ASCII and then you have to count the memory requirements from MicroPython itself, when evaluating a program. I think this approach might turn out very useful for smaller devices (not necessarily ESP32 since MicroPython runs well on it).
Sorry, maybe I’m not smart enough to understand all the nuances, but wasn’t the “external”/”secondary”/”relayed” execution already achieved by works of niicoooo and BrianPugh[1]? Agreed, these solutions need a unikernel approach, but given when they were released I would have expected that there would be many more “miniOS” available now and instead people keep creating these lame virual machines, which frankly are more useful as a style exercise than as development environments. what am I missing (besides the ability to make this miniOS myself)?
[1] https://github.com/joltwallet/elfloader / https://github.com/niicoooo/esp32-elfloader
CHIP-8 is a nice way to explore assembly language, and its great that Boro’s system is live. Having an interpreter greatly reduces the turn-around time. On my Macbook Pro, it can take over 30 seconds to compile and download a large Arduino program for the ESP32. Going through that process over and over quickly becomes tedious. An interpreter reduces the turn-around time for program changes to a few seconds. That’s really important, especially for beginners just learning about microcontrollers.
Just to mention another interpreter…
MicroBlocks (https://microblocks.fun) is an interpreter-based, blocks programming system for the ESP32, micro:bit, RP2040, and other microcontrollers that supports live programming. As the user works, their code is incrementally downloaded to the microcontroller and stored in Flash memory. The program is executed by an interpreter and runs whenever the microcontroller is powered up. Although MicroBlocks programs are written in a blocks langugage, they can do low-level things such as generating waveforms, controlling I2C or SPI devices, or doing Arduino-style pin I/O. Interpreter performance is similar to MicroPython.
That’s literally what I am doing at homebots/vm on GitHub, but for esp8266 😀
I’m writing a mínimal VM inspired by CHIP8 and my end goal is to have a Node.JS server holding the state ou my entire house, witha redux-like design, and effects that can dispatch small commands to be interpreted by this VM