Modern microcontrollers like the RP2040 and ESP32 are truly a marvels of engineering. For literal pocket change you can get a chip that’s got a multi-core processor running at hundreds of megahertz, plenty of RAM, and more often than not, some form of wireless connectivity. Their capabilities have been nothing short of revolutionary for the DIY crowd — on any given day, you can see projects on these pages which simply wouldn’t have been possible back when the 8-bit Arduino was all most folks had access to.
Thanks to the increased performance of these MCUs, hackers and makers now even have a choice as to which programming language they want to use. While C is still the language of choice for processor-intensive tasks, for many applications, Python is now a viable option on a wide range of hardware.
This provides a far less intimidating experience for newcomers, not just because the language is more forgiving, but because it does away with the traditional compile-flash-pray workflow. Of course, that doesn’t mean the more experienced MCU wranglers aren’t invited to the party; they might just have to broaden their horizons a bit.
To learn more about this interesting paradigm shift, we invited the fine folks at Adafruit to the Hack Chat so the community could get a chance to ask questions about CircuitPython, their in-house Python variant which today runs on more than 400 devices.
State of the Union
Limor “ladyada” Fried, Phil Torrone, and Scott Shawcroft covered so much ground during the chat that it ran twice as long as originally planned. As usual when Adafruit joins the Hack Chat, the event was simultaneously streamed to their YouTube channel, which brought in even more questions and comments. It was a bit like spending an hour drinking from a fire hose, but we’d say the end result was more than worth it.
The discussion started with some talk about current events, such as the ongoing development of USB Host mode, which would allow things like USB keyboards to be plugged directly into a board running CircuitPython. The team also talked a bit about the upcoming ESP32-based “My Little Hacker”, which was designed to be a drop-in replacement for the sound effect boards used in children’s toys.
We also heard a bit about Adafruit’s work to create a modern replacement for the legendary Bus Pirate. The chips used in the original design from Dangerous Prototypes are apparently no longer being produced, so Adafruit has come up with their own version of the electronics debugging tool using the RP2040. With the benefit of the chip’s native USB and Programmable I/O (PIO) hardware, plus the flexibility of CircuitPython, the new gadget promises to be a more than worthy successor. All that’s left is to figure out what to call it…
A Tale of Two Pythons
After bringing folks up to speed on what’s new in the world of CircuitPython, the team was hit with a fairly prickly question — why does CircuitPython exist in the first place? Those familiar with the situation may recall that CircuitPython is actually forked from MicroPython, and some have argued that Adafruit would have done better to support an existing open source project rather than pour development time and effort into what could be seen as a competitor.
In response, Scott and Limor gave a fascinating look at the origins of CircuitPython and the decision to spin it off into a separate project. The short version of the story is that the respective teams had some disagreements on how to handle certain things, chief among them being hardware APIs.
Specifically Adafruit wanted to be able to use the same libraries they had already ported over to standard Python, but that wouldn’t be possible under MicoPython, which had replaced some core low-level functions with versions that were modified in an effort to reduce how much space they would take up in flash. This also meant that MicroPython code wouldn’t necessarily work the same if it was run under the standard Python interpreter, which Limor found particularly troublesome. As such, it was decided to spin CircuitPython off into a clearly distinct project rather than just trying to get their changes approved.
That said, Adafruit continues to financially support MicroPython and does not internally consider them to be competing projects given their focus on the educational and hobby market. They also still consider CircuitPython to be downstream from MicroPython, that is, the intention is to eventually integrate new features and capabilities added to MicroPython into CircuitPython. Naturally, some in the chat asked if the projects might one day merge back into one. Scott said the idea comes up in discussions with the MicroPython team regularly, and while there are still some fundamental issues that would need to be resolved, the possibility is there.
Making CircuitPython Yours
With the existential debate over with, the conversation then moved on towards bringing new boards into the CircuitPython family. Namely, what an individual needs to do to make sure their own custom creation will work as expected in the environment.
The simple answer is that as long as you’re using one of the core chipsets they support like the ESP32, then getting the little details sorted out is relatively straightforward. There’s even an official guide from Adafruit on how to get your custom board added to the list of supported hardware.
Another topic brought up is the unique feature of CircuitPython that halts the currently running program and reloads it from the file once the USB mass storage file system sees that the source file has been saved. This was added specifically to make development as straightforward as possible, especially in educational settings where the computers in the classroom might not have any software installed more advanced than a simple text editor. To facilitate this feature, it was decided that halting the program should revert the system to a “clean slate” condition that doesn’t retain any state information from when the code was running.
The downside is that the same thing happens even if the program has been manually halted with Ctrl+C, which can frustrate more advanced users who are looking to debug their program by examining the state it was in when it was stopped. After some technical discussion, it was decided that it should be possible to optionally disable this feature for those who want more control over the execution of their code. Scott said he would be willing to lend a hand should somebody in the community wish to implement such a capability and submit it as a patch, and Limor commented that it could be officially merged into CircuitPython so long as it didn’t impact the default functionality.
Onwards and Upwards
As things started to wind down, several more topics were covered such as the adoption of TinyUSB within the industry, the challenges involved in supporting USB Mass Storage, and community-developed patches for new RF protocols such as Zigbee and ESP-NOW. There really is a wealth of technical information contained in the recorded live stream; if you’re interested in the nuts and bolts of CircuitPython, there’s certainly worse ways you could spend hour of your time than settling in and watching the whole thing.
Special thanks to Limor, Phi, and Scott for taking us on this whirlwind tour of where CircuitPython has been and where it’s going. It’s easy to get overwhelmed when there’s so much active development going on, so getting this type of insider perspective is immensely valuable for those who are looking to dip their toes into the fast moving waters of bare-metal Python.
The Hack Chat is a weekly online chat session hosted by leading experts from all corners of the hardware hacking universe. It’s a great way for hackers connect in a fun and informal way, but if you can’t make it live, these overview posts as well as the transcripts posted to Hackaday.io make sure you don’t miss out.
Is circuit python appropriate for real time applications?
No! Python is a great language for use on fully fledged computers, Blender is written in it, much data analysis and machine learning is done with it, but it has no business being on microcontrollers. A compiled language like python only discovers certan crash types when it s running, C/C++ would warn at compile time. This is ok for a program running on a PC but a disaster on a microcontroller which may well be running unattended for weeks. And these python derivates make far slower, far more space hungry code than C/C++ would, fine on a PC or cluster, or a Pi like single board computer with plenty of speed to spare, no real-time requirements, and ample memory, but no good on a microcontroller trying to do something fast, repeatably, reliably, within constrained resources. Stick to C/C++/Arduino or some sort of java like languages for some microcontroller types, when you need real-time, or any other use interacting with GPIO level circuitry. Python is a great tool, but microcontrollers are not the place for it, just like chainsaws and axes can be great tools, but you’d never use one of those on a precision circuit board either.
This is being a little more than harsh as you’re assuming this person’s definition of “real time” is the low latency, guaranteed timing stuff when they could be referring to something with a 100-300ms response time. I’d wager anyone asking about real time application and python likely isn’t well versed with the more advanced topics of real time applications or OSs. Maybe answer the question in a way that doesn’t immediately dismiss reasons a person might use Python: ease of use. Something along the lines of, “If you need guaranteed timing for your use case, Python and it’s variants aren’t the most reliable and you should stick with lower level code, but if you’re ok with not being exact, then it’s a great place to begin to learn how to use some of these microcontrollers.” Finally, Blender isn’t written in Python it includes a python interpreter mostly for plugins, and most data science and machine learning uses Python as, possibly unknowingly, a wrapper to underlying low-level compiled libraries written in c/c++ and Fortran. We don’t need more jerks dismissing technology that makes it easier to get involved in things that could revolutionize the world simply because they were tortured learning the complex stuff first while walking in the snow up hill both ways. Those of us that need to know the low level stuff will always have a job, but that doesn’t mean we can’t make it easier for the next generation of inventors.
Not really making it easy by teaching techniques that don’t scale or perform.
If you want to learn programming μCs and it’s your first touch, why not. But learning yourself serious real time app. programming on CircuitPython is a waste of time. To me it’s like learning french knowing you are travelling to USA.
I could never understand this “ease of use” argument. How Python is “easy” if it’s not any different from the likes of C? Same relatively low level eager imperative language with very little room for implementing higher levels of abstraction.
The only real appeal of Python is abundance of libraries made for all possible use cases. But none of them are available for MicroPython, making it an odd beast – not really “easy to use”, not easier than C, and not having any of the batteries included that actually gave it this “ease of use” feel.
It’s easier to use in that you can access REPL, an IDLE-like interactive interpreter, while tethered to your computer. Without rebooting the board you can connect an i2c device, initialize the GPIO, send some bytes, and see what happens. It goes way beyond the prototyping and debugging you could do with serial alone.
As stated in the article, you do not need to flash the device for minor updates to the code, either. Just drop a new file onto it and it runs. For chips with wifi, like the Pi Pico W or ESP32 family, you can program them to self-update over the air with trivial effort.
Some timing-critical uses could also be addressed with the RP2040’s PIO on those boards.
Yes a compiled language will run faster and take less space. But if your use case can be covered by these embedded Python languages, why not use them?
mark g makes the points I was going to….in addition, try giving someone in a school setting or a casual experimenter with electronics and programming a crash course on string manipulation in the C type languages vs Python….
As always, the applicability of a solution depends on the problem – yes, Python on a micro is going to be “slower”, “fatter” etc than a C type language, but if the speed and space constraints don’t – well, constrain in the particular application, what’s the problem?
Doesn’t the *python compile down to native C to run on the actual PIC? This was always my assumption since there’s more than just Python interpreters available.
I’ve been sitting on the idea of building a Crystal interpreter for a few weeks now and just from my surface level research this should be how it works.
If I’m correct, your argument against Python is moot. Although, I personally can’t stand Python as a language and I’ll support any real argument when not to use it but in this case there’s no viable reason why NOT to use it.
No, that’s the whole point. As soon as you save the file, the MCU reads the Python and interprets it. There’s no waiting for anything to compile.
No, that’s wrong. Micropython compiles your file to bytecode and caches it so it’s not done twice. It can even compile to binary code directly on the embedded CPU coprocessors.
A similar point again – from the perspective of the user, you change the code, save the file, it runs. You don’t need to know about board settings, CPU frequencies, flash interface being DIO or QIO and so on. If you do want to know about and control such things, you are not the target audience.
Blender is NOT written in Python, it is written in C. Check the source code.
There’s so many misconceptions here that you better document and review your opinion. Python currently is a mono-task interpreter. It holds a global lock (called the GIL) while it’s running. This means that it’s deterministic for its processing time for its code, which is a very important feature for RT code.
Micropython compiles your python code to bytecode and can even JIT it to binary code. The python code will run very close to C code in timing (but you don’t care, see below)
It also supports raw _thread, unlike Python, where they don’t take the GIL (so you can do the C/C++ tweak you usually want to do without disturbing the main task).
Unlike C/C++ where a crash means game over for your gizmo, µPython will capture an exception and you can deal with it (in fact, it deals with it better that the default ESP32-IDF panic code). You can retry the operation, log it to a file (which C/C++ can’t do usually, since the crash has fucked up the registers, you can’t rely on anything in C/C++, only reboot).
I have µPython devices that have months of uptime in the wild and counting. You’ll suffer from HW failure before you’ll suffer from µPython’s failure.
Both have watchdog so even if fucked up, you’ll have your hardware WD reboot your system in a reliable manner.
In µPython, you have C/C++ modules built-in for low level peripheral, so most of the time critical code is actually running the SAME CODE as your C/C++ application.
When you need to modify some code, in µPython, you’ll FTP to your board, upload the changed file and you’re down. It takes ms to perform.
On a C/C++ project, you’d need to rebuild a firmware, then reflash it. This takes mins instead so it’s much much faster to develop, the code is more concise and it doesn’t break with whatever new version of ESP-IDF.
You can debug µPython code over the network, while you can’t do that with C/C++ (not yet, should be possible with a network stub over gdbstub).
Please stop sweat talking this pink python glasses lover blindness.
the GIL is a horrible thing that should never have existed. I have a 6 core 12 thread Ryzen 5600, and this GIL lets 11 of those 12 threads idle, and it’s hard to work around.
GIL also blocks everything and your grandmother, while the idea of real-time programming is to have as much things run concurrently and things not block each other.
Then about the compile / program / run cycle.
Anyone beyond an absolute beginners course has his coding enviroment set up in such a way that compiling and programming cycle is put in a easy acessible way, for example a single keystroke (this is not arduino stuff). Compiling a program for an STM32 takes about half a second, while the programming cycle with an ST-Link is also in that order of magnitude. Older processors such as the AVR’s can be a lot slower to program. ESP32 is an horror on this side. Even a blinking led program tends to suck in megabytes of libraries and are very slow to program, and that is part of why I’ve mostly ignored that platform.
And about the crashes…
What are you trying to prove here?
Crashes are usually caused by faults in software, and I extremely prefer a compilation failure on a typo then python just inventing a new variable for me. I have got C programmed microcontroller nodes running in an RS485 network for 15+ years. A few years ago one of the network nodes failed because the brushes of a DC motor were worn out.
What do you mean with: “A crash is game over for your C/C++gizmo”. First you have the compile and link steps, which catch lots and lots of typing errors. Then you have tools like assert and debuggers like GDB (depending on your platform).
About speed…
Just no, don’t go that way. Sure, python programs can run “nearly as fast” as C / C++ programs… as long as most of the program is linked in as libraries written in C or C++.
Yeah, sure, I don’t like python, and I acknowledge I’ve got these other glasses on, but I thought it appropriate because of your one sided and misleading praise of that language.
And now I want a picture of a hairy feet with curly toes stamping down on that silly snake.
How did that snake even become the icon wile even the name was “inspired” by Monty Python?
Why are you running microPython on a Ryzen 5600?
arduino for esp32 in the normal arduino IDE is horror and super slow. BUT try PlatformIO in VS Code, its fast, its C/C++ it has proper caching and compiles in seconds if at all. also has auto complete and all that fancy stuff. flashing is also just a button press. And the best part, you can define if you want to use the espressive libs or the arduino libs (and define the chip per project so no switching around that takes ages to do).
Also yes (Micro)Python has nothing todo near a µC. its slow, it crashes, it has no multithreading. and the no types thing makes things often alot worse to debug. fast prototyping yes, real applications, god no.
All the cool kids have moved on to Rust.
Rust runs exactly what in the real world? It’s not even close to C in performance, ease of integration or platform coverage.
But yeah, I hear kids use it…
if by “cool kids”, you mean contrarian 20something internet denizens, sure, but nobody else wants to (or has the time to) learn it. What does Rust’s exotic syntax and permanent beta status offer?
A fresh approach
They used video and astroturfing to market Swift and Rust at the kids –all they had were books to market Ada and Pascal (which existed to solve the same problem).
But Swift, Rust and for the non-programmer/dot-com tool scriptures even Ruby and Device(Type)Script are all available for microcontrollers.
I used CircuitPython as a custom board testing framework for a fairly big multi-FPGA board (24 ASICs, 3 FPGAs, and a SAMD21 microcontroller used for overall control) used in a scientific experiment. It worked great – made board-level testing and adjustments easy, and then once things were obviously working we implemented a control packet framework, replaced the CircuitPython core with a simple Arduino framework firmware, and just… copied a ton of the CircuitPython code to Python and kept working.
I *love* using Python for investigative-type testing of a board, or stuff you only need to do once, like board initialization. As someone who works with FPGAs frequently, it reminds me of using soft microcontrollers in an FPGA – when the control flow/logic is complicated and speed is unimportant, readability and speed of development is key.
That’s correct, I too use for investigative testing. For example, when I have get a new sensor, it is easier to interactively read/write registers using Micropython or circuitpython rather than compiling and uploading C code each time for different registers.
I wish they’d have stuck with MicroPython. I do want to use CircuitPython for a project, but the key library I need (MIDI) is so poorly/wrongly/semi/un-documented, I’ve been thoroughly discouraged. On top of that, Adafruit has displayed very bad behaviour on YouTube, deleting/blocking even slightly critical comments, ignoring questions, etc. I’d love to love Adafruit, but my feelings are now quite the opposite.
This is the second comment recently that I’ve seen giving a poor report of Adafruit as a business. That is quite a shame to hear. :/
Honestly, from what I’ve seen, they’re just doing *so much*, with so few people, that a lot of these tasks fall by the wayside. Considering the sheer amount of “hey, wouldn’t it be cool if…” stuff that they implement and then sell for almost cost, I’d be shocked if it was all properly documented.
I’m a much higher level software guy, so working with microcontrollers is….very difficult for me, and requires I think in different ways than I’m used to, and I’d benefit a great deal from better doc.
But again, the shelf full of crap from Adafruit that I can’t use yet because I’m not good enough at it isn’t enough to make me say “no, stop producing rad shit and instead do boring doc instead”. I’ll just suffer while I learn.
There is definitely a bit of land-grab going on here, it very much suited them to fork micropython from a commercial point of view. I’ve noticed how they are evolving their libraries in general to abstract the underlying hardware in a way to make it more difficult to use with non-adafruit supplied gear. I guess that’s entirely their call, but it feels a bit disingenuous/against the spirit of FOSS. Don’t get me wrong, Adafruit’s contribution to the ecosystem has been amazing and transformative, and I’m grateful for that, but they really don’t need to embark on this protectionism – it’s driven me away from their products, which is sad.
p.s., Also, good luck getting their recommended editor/IDE(?) to work. It has failed on 3 of the 4 machines I’ve tried it on, and might even be abandonware…
Adafruit uses Mu (https://codewith.mu) as their recommended (simple) editor. They do not enforce or require Mu; Adafruit needs to have a “baseline” to provide online documentation as screen-prints for novices.
As for others who are faulting Adafruit for branching microPython into CircuitPython, I find no reasonable arguments to not have taken this route to provide support for their hardware purchasers: it is exactly the Arduino paradigm which was the foundation of Adafruit’s business.
Rather than criticism, Adafruit should be applauded for having a vision and keeping their customers firmly in view.
Additionally, since Adafruit provides a customer support forum, it is critical to provide a direct group of processes that the customer can utilize to achieve success. Anyone that has ever worked the Arduino forums understand the need to have straight-forward procedures that both the Op and the guru can follow. Adafruit is simply acting as a caring business by providing hardware and software that provides a good customer experience while providing a free support forum to assist novice users that have a train wreck.
If you do not “like” Adafruit, no one is forcing you down their business methodologies; there are so many other options.
Yeah I really want to like Adafruit but so far nothing I’ve seen has persuaded me to… coming in and forking MicroPython seemed like a shitty move for a very promising and well executed project, and fragmentation like this almost always hurts the overall cause as the support & user base gets split in two.
The big thing I hope they focus on is low power support, especially with Bluetooth. CircuitPython has a lot of potential!
For their own products like the Mega Grand Central, I use CurcuitPython. Libraries are there for easy integration. For the the RPi Pico I use microPython and rshell for loading and debugging programs. So just ‘depends’ on which I use. I found copying to /pyboard just as easy as how CurcuitPython works dropping files. Actually even simpler as I don’t have to perform a sync after dropping the file. I don’t use IDEs for Python development. Just a text editor like Geany.
Just run micropython. Seriously.
My company was an original supporter of MicoPython and we were disappointed to see such an early fork by an influential company. I had seen this many times before in situations like adding floating point to Forth, or strings with an EOF character versus arrays with a length. The forks usually continue then in eccentric avenues. So, we never looked at Circuit Python again. Does it have the same speed-up methods as MicroPython? And Damian George is a physicist, so…. https://www.youtube.com/watch?v=hHec4qL00x0&t=248s
Something that others have touched on here, but that can’t be overstated is that Micropython and CircuitPython both have their entire build systems resident on the target. All you need on the host is a text editor and a USB connection. This means you don’t need to download and configure hundreds of megabytes of compilers and tools to update and debug your code. Even a phone or tablet could be used without any rooting or configuration. Plus the integrated repl means that you can even debug you code live on the target and when things fail, you actually get meaningful (if abbreviated) failure messages. You end up getting a lot of the goodness of an embedded FORTH but with the “batteries included” feel of python and a much smoother feel all around. Would even Arduino-flavored C be more performant? Absolutely. You pay for all that convenience, but for many applications it is a reasonable, or even desirable tradeoff. Plus having the full, plaintext source resident on the target means there’s no question about which build is running when troubleshooting something.
Also, speaking as someone who has done work with both flavors of embedded python, the differences (and thus the fork) make sense of you understand what use-cases they’re targeting. CircuitPython is aimed at the education and entry-level electronics markets, and thus has some key “idiot proofing”, which limits the power somewhat, but makes it a little easier to understand for beginners and limits headaches for those that teach/support them. MicroPython is a little more general purpose, and has more opportunity for additional power or performance, while still being approachable for beginners. There’s room for both models in the marketplace.