There are plenty of small microcontrollers available for all kinds of tasks, each one with its unique set of features and capabilities. However, not all of us want to spend time mucking about in C or assembly to learn the intricacies of each different chip. If you prefer the higher planes of Python instead, it’s not impossible to import Python on even the smallest of microcontrollers thanks to MicroPython, which [Rob] shows us in this project based on the ESP32.
[Rob] has been working on a small robot called Marty which uses an ESP32 as its brain, so the small microcontroller is already tasked with WiFi/Bluetooth communications and driving the motors in the robot. Part of the problem of getting Python to run on a platform like this is that MicroPython is designed to be essentially the only thing running on the device at any one point, but since the ESP32 is more powerful than the minimum requirements for MicroPython he wanted to see if he could run more than just Python code. He eventually settled on a “bottum-up” approach to build a library for the platform, rather than implementing MicroPython directly as a firmware image for the ESP32.
The blog post is an interesting take on running Python code on a small platform, and goes into some details with the shortcomings of MicroPython itself which [Rob] ended up working around for this project. He’s also released the source code for his work on his GitHub page. Of course, for a different approach to running Python and C on the same small processor, there are some libraries that accomplish that as well.
Typo:
“not” should be “now”
‘ it’s not impossible to import Python on even the smallest of microcontrollers ‘
You are not right.
It seems like you have no inkling of what “the smallest of microcontrollers” means. Show me MicroPython on an ATtiny24, and I’ll shut right up.
I think you just jump to unrelated conclusions. It does not make sense to say “it’s now impossible to import Python … “. Geez.
Well, that wasn’t me, but it was also pretty brash to say “it’s not impossible to import Python on even the smallest of microcontrollers.”
Physically small – there are some micropython capable micros that are disturbingly small, not stupidly small memory and rudimentary cpu core.
“smallest of microcontrollers”
“ESP32”
Umm…
even on an esp8266! :)
Oh, just BS. “not all of us want to spend time mucking about in C or assembly to learn the intricacies of each different chip.” Right. That’s why nobody uses Arduino. The Arduino application is just an IDE for C++, with a bunch of presets for Atmel and a bunch of other microcontroller architectures. It’s really nothing more than that. And for that matter, C was designed specifically to make it easy to program without having to care what computer architecure you’re using, so that you don’t have to rewrite each program when you move to a different architecture.
The ONLY advantage MicroPython has (and yes, this is a significant advantage) is that you can flash the interpreter onto the microcontroller, then put your programs on an external device, like an SD card, and not be limited by the size of a microcontroller’s program memory. This is why we see both MicroPython and BASIC used on microcontroller-based small computers. (Which language is chosen seems to depend mainly on the age of the developer…) To the microcontroller, your BASIC or Python program is just data on an I/O device, so as long as you can fit the interpreter into flash memory, your program can be much larger. And with this extra space, you can include a text editor, which then allows putting all of the tools on the micro system, rather than having to plug it into a “real” computer to compile and reflash the micro.
Thanks for explaining that use case. I’ve been wondering for a while now why anyone would actually use circuitpython instead of a compiled language on a mcu. It always struck me the same way as the doom port—something done for the challenge, not for practicality.
Also very good point about C (and compiled languages in general) being originally oriented towards platform-agnostic programming. Of course if you’re writing firmware, you still need to know what registers to use for this or that, and how timers and interrupts work, but arduino does a pretty good job of hiding that (and I use “hiding” here in a little bit of an accusatory sense, because I would bet that a large portion of arduino users don’t even know what a register is—but at least they’re programming and hacking, right?).
I can also think of one more very small use case: home brew calculators. I often use the python shell (or for more complex stuff at work, GNU Octave) on my computer as a calculator, so I could imagine it would be pretty easy to assemble a calculator that way.
There are also three levels of ways to speed up MicroPython, including a built-in assembler. You can write handlers for ports and any of the I/O in assembly. https://www.youtube.com/watch?v=hHec4qL00x0
That’s ann interesting perspective. I like python, and I like programming microcontrollers, but had poor experience with circuipython on a dev board because it didn’t have much memory. I like your concept.
The ONLY advantage MicroPython has(…)
No, MicroPython’s main advantage is that it has REPL so you can write code and debug at the same time.
Let’s not forget: Learning Python for many (myself included) is easier than learning C++. I can do vastly more things with my knowledge with the higher level language. I’m certain I could do exactly the same with C++ had I invested more time, but I’ve got things to do. Yes, your experience may be different. But my experience certainly is not unique.
And that’s because of the features of each language, and here I’m really using C and Python as shorthand for compiled vs. interpreted code. You could easily use a compiled form of Python (I think there are at least two implementations of this) to get compiled Python, which would have performance similar to C, but that’s not really what I’m arguing. The limitation inherent in microcontrollers is that adding ANY executable code, that is, code that is directly executed by the CPU, regardless of the language that code was written in, requires reflashing the ROM.
you can’t compile python the way you can with C, meaning you can’t translate it into machine code. you can “compile” it into a bytecode that’s faster for the interpreter to parse than raw python code (that’s what the *.pyc and __pycache__ files and directories in your projects are!), and you can probably have a runtime that uses a Just-In-Time compiler to translate things to machine code in real time, but python is a very dynamic language and there’s just too many things that you can’t get from the code itself, but need additional context only available at runtime (for example, data types!).
Python is a great language, and it performs really well given its many slow design aspects (very dynamic, regular non-JIT interpreter in most cases, tons of stuff is implemented as Dict entries behind the scenes, it uses garbage collection). Python is one of my favorite programming languages for desktop and server applications, and discovering MicroPython has taken the dread out of embedded projects for me. going forward, I’ll spend the couple extra bucks to put an ARM chip in a project that technically could’ve used an AVR or something, because C is a nightmare. but, outside of extremely contrived scenarios, python absolutely cannot have performance similar to C. The V8 runtime for Javascript, a similarly dynamic language, has been relentlessly optimized over many years by Google, and it’s still much slower than C. The only reason python might seem similar is because microcontrollers are pretty fast these days, most of what people do with them is pretty simple, and writing optimal C code is much harder than writing optimal python code.
Ah. Good point about “compiled” Python not actually compiling to machine code. However, saying that things like dynamic typing can’t be done without an interpreter is just wrong. All you need is a library that hides the underlying class structure in your objects. Which, granted, would eliminate much of the performance advantages of C and C++, but anything that can be done by an interpreter can, pretty much by definition, be done by code that is linked into your compiled code. I would argue that dynamically-typed variables are a good way to generate bad code, but if you really want something like that, in C you always have the option of using void pointers to objects of any type, as long as your functions know how to deal with all of the types you can throw at them.
But going on to say that “writing optimal C code is much harder than writing optimal python code” just puts you in the fanboy category in my eyes, since NON-optimized C code will almost always outperform highly-optimized Python code. Which just leaves me shaking my head.
In my own experience, having decided to give Python a try, due to its supposed virtues, I found that it was every bit as problematic as C, and the performance hit was just ridiculous.
You actually are constrained by the memory of your microcontroller. Whether your program is loaded from SD card, external flash memory or internal flash, it still needs to be parsed and compiled, and that means that not only it has to fit in RAM, but you also need twice as much RAM, to store the data structures used during parsing. So no, you don’t get that particular advantage with MicroPython.
No, it doesn’t. You can have a buffer in RAM that takes one line of code at a time, and interpret that. Which, yeah, is even slower, but no, you’re not constrained by RAM. Really, the only thing that has to be in memory is a symbol directory. And not even that, if you have a file on the SD card that holds that. Yes, you have to give up speed to get capacity, but it’s doable.
You can, of course, a) pre-compile or b) ‘freeze’ your code to reduce memory use. The former allows you to skip the parse/compile stage (but still execute from the filesystem). The latter takes that a step further and embeds the precompiled bytecode into a firmware binary.
They both trade-off convenience for reduced memory use.
Look: here’s what I’m getting at:
Microcontrollers are great – they are “systems on a chip”, and you don’t have to implement RAM, ROM, and basic I/O, because all of that’s already done for you. So it’s natural to think, “hey, I wonder if I could make an 80s-style stand-alone general-purpose home computer based on a microcontroller.” And yes, I’ve thought that. But what you run into very quickly are two solid walls: 1) ROM is ROM, and RAM is RAM, and you can only execute what’s in ROM, and 2) both built-in ROM and RAM are very small.
But not yet beaten, you think, “okay then, I can hook up an unlimited amount of external memory using SPI,” but you’re still stuck with only being able to execute from ROM, which is not, repeat NOT expandable. So maybe you think, “microcontrollers can write to their own flash ROM, so how about I compile programs and store them in a filesystem on external memory, then have a permanent function in ROM that reflashes part of that ROM from this external memory?” Which you have to dismiss out of hand, because of the limitation inherent in flash memory to how many times it can be erased and rewritten. And we’re still limited to the size of the internal ROM, AND that has to also hold the drivers for that external filesystem as well.
“No problem,” you finally, reluctantly accept, “I can put an interpreter in ROM, along with as many time-saving useful optimized functions as possible in ROM, and use that infinitely-expandable external memory to store the high-level code.”
And that’s what I see over and over. Many people have come to this same conclusion, and it’s a sensible one. But let’s be honest: we’d much rather have the option of running compiled code, for the performance advantage. But we’ve just had to come to terms with the fact that with microcontrollers, that just isn’t practical.
So let’s skip over the denial phase. On those 80s home computers, there was ALWAYS a mechanism available for embedding raw object code in your programs, and these were explored by anyone whose programs ran slower than was comfortable because of the inefficiency of interpreters, or needed to access hardware in ways the writer of the interpreter anticipated. Many of the games and other applications for these computers were written in assembler and compiled into byte strings which bypassed the interpreters completely. But this option does not exist on microcontrollers, because what is in ROM (flash memory) is ALL that it can ever directly execute.
should be “in ways the writer of the interpreter had NOT anticipated.”
To be fair, micropython has some huge advantages in development speed. I’m using it professionally to produce a diagnostic imaging medical device, where using micropython and openmv has mean we’re getting to marked in about quarter of the time/budget it would have taken in C/C++.
The live interpreter is not the main advantage here, though it combined with jupyter notebook provides a powerful platform system-test / bring-up tool for elec and systems teams.
We also do not have our code on external flash, the production code is byte-compiled into the main firmware and flashed over jtag/swd, much like any other microcontroller. However, during development the production code can be updated piece-wise with live code updates to enable much faster development.
Also, having a unix/linux port of the same codebase means we can run over 95% of our firmware on the desktop and in CI testing, we’ve got a completely virtual copy of the medical device we can interact with on PC for dev/testing. This was enabled with literally only a handful of lines of code to mock some low level hardware interfaces.
On top of all this, there’s a whole raft of other benefits like memory reliability (no buffer overflows), high levels of portability of python code across multiple hardware platforms and code re-use (structured modules on central repository).
We’re using micropython as the base platform for a growing number of professional contract engineering projects, and we’re not the only company doing so.
Thanks for the correction/clarification
If you believe that’s the *only* advantage of MicroPython I do encourage you to get more involved in the community; there are significant other benefits!
The first is most obvious; writing code in a higher level of abstraction is faster and less prone to errors. Using a GC is not without consequence – in particular you need to keep on top of memory fragmentation – but the upshot is that memory management is simplified. Even as a software engineer with a couple of decades of C under my belt there is no doubt that I can develop solutions in Python with far less effort.
It’s impossible to overstate the benefit of having a live REPL, particularly on an embedded device. Not sure if your peripherals are operating as expected? Just try it out and see. I mean even having i2c.scan() to identify your peripherals is a huge benefit.
Performance is an interesting one – while operating within the interpreter there is of course a performance hit. However, MicroPython is written in C and there are many operations that you’re unlikely to do any better in a pure C application. For example, you won’t send SPI buffers any faster – but it sure is a lot easier to send them in MicroPython! And if you have any performance-sensitive code then it’s relatively easy to write that in C and provide a MicroPython interface. In some applications this is necessary – but it’s very rare to require it in more than, say, 20% of your codebase.
The filesystem is even more powerful than you might think; it’s very easy to mount your local filesystem – from your PC – over a serial port and execute code as if the files were on the device. The C-style compile/deploy iterations are painful once you’ve developed like this.
In short, developer productivity is significantly improved at the cost of higher hardware (particularly memory) requirements. The company I work for can typically quote a solution in MicroPython for 60-80% of the equivalent development effort in C. If you’re not considering developing in MicroPython be aware that your competitors might be – and they’ll likely be able to develop a solution faster and cheaper.
Lua would be a better choice. Smaller footprint than Python, faster, and easy integration with C.
Yes, agreed. While everything else continues to bloat beyond control and comprehesion, lua stays true to original intentions.
Is that true? I certainly hate the fact that python is becoming a kitchen sink language. Yup Lua isn’t. But looking at MicroPython there seems to be more code in all the device drivers and communication protocols than in the core language. How does that work in Lua? When using it on an esp32, does one just have the core language and then nice stubs that call into all the device drivers in esp-idf? Or is there a huge library that implements all the device drivers in lua to talk to the HW?
I took a quick glance at the whitecatboard “Lua-RTOS”… looks to me like “bloat” is a good descriptor… but maybe that’s the wrong “lua on esp32” to look at?
How much Lua is taught in Computer Science departments?
Why is this relevant? The movers and shakers in the tech world only use traditional educational as the most basic of springboards from which they progress, otherwise everyone over a certain age would still be using Pascal, Fortran, and COBOL. Besides, the “only in it for the paper degree/money” crowd could not care less about hackaday and hobby projects involving microcontrollers.
I don’t know much about Lua, I might look into it. One of the reasons that Python for microcontrollers might become quite popular is that a lot of people already use Python on their ‘big’ computers for a number of reasons, including the number of libraries available for it etc. Even if it’s not necessarily the most resource efficient choice for embedded systems, it’s certainly easier to use what you already know if you don’t have a lot of time (or are a bit too lazy) to learn something else.
I would say that Lua is an alternate choice, not necessarily a better choice. There are pros and cons on both sides of the lua vs. python options. MicroPython works very well and has been very reliable for me. I also have lots of gripes about it. May try Lua again just to see how it feels on an esp32…
We need a python microcontroler that can handle 6 servos. audio in /out, discrete digital in/outs, analog in/outs, wifi, bluetooth, radio connections.
and small display screen..
and magnotometer, & inertia acelerometer,GPS?
I haven’t done any major projects (only blink an led and such) with circuit Python (or micro Python), but I like the concept. I plug in my Trinket or Arduino Mega/Metro boards (AdaFruit) via USB , and it comes up as a ‘drive’ on my Linux desktop. Now all I have to do is drop a ‘main.py’ onto the ‘drive folder’ to make changes to the code. The device automatically takes off and runs the new code. Simple. I can use my favorite editor on my desktop to code. Python 3 has become one of the most favorite languages to work in for me. Use it all the time at work for tasks, and of course all the time on the RPIs. Because of processor speeds, Python is fast ‘enough’ for most tasks I deal with. Note ‘C’ and some assembly is what I did most of my programming in my career for realtime applications in the automation world (power plants, substations, communications) and I like it too. Don’t really care for C++ in that world, so stayed away from it. I wrote a lot of Object Pascal (Borland, Delphi) though on the desktop side. Really it all depends on the application at hand to what should be used…. But nothing beats the convenience/ease of coding in Python. That’s been my experience as a programmer by trade.
For now, my goto development platform for the micros I use most (ESP8266) is Arduino, especially when using a more advanced Arduino IDE like VisualMicro (with MS Visual Studio) or sloeber (eclipse). This seems to be the right mix of “easy” but still having C++ ways to tweak and optimise code for resource-limited devices.
I do like Python, and as microcontrollers and small SBCs gain increased resources, I can definitely see that MicroPython will make device programming more accessible.
The huge step up from Arduino to MicroPython is the interactivity. When you’re trying out a new device it’s very easy to type some stuff into the REPL to twiddle IO lines, or to call some functions to see what happens.
Once you have a project more or less working, the REPL is a godsent for troubleshooting. When new errors occur in devices you’ve “deployed” it easy to set something up so you can remotely run interactive commands and inspect what happened and try new things out. This doesn’t have to be remote, even on your bench it makes it easy to get in and inspect what the state of things is. Without having to hook up JTAG, which take pins, run a debugger and all that jazz.
Oh, and when something bad happens it just turns into an exception that can be caught and logged or reported, not some null pointer deference and a reset, which wipes out all troubleshooting info.
The ESP32 isn’t exactly a small embedded controller. It’s quite beefy in terms of cpu processing and memory. It has as much memory as my Amiga and Atari ST back in the day.
There is nothing written about the performance of Python on the 32bit mcu’s so I would wager it is quite slow and not suited to more demanding tasks. If it were the users of it would be bragging but they don’t.
So what is it’s main use on mcu’s?
Education, one-off projects where electronics are an afterthought, art, etc.
Running Python alongside or inside a C program seems like a good idea for an MCU, because heap fragmentation is a serious issue.
On ARM, ESP32, and other 32-bit architectures, you can go ahead and use traditional toolchains to get decent performance, because these generally don’t separate the program memory from data memory. That is, you can execute code from either flash memory or RAM. I think. That’s the case for ARM, anyway. This means that if you prefer programming in Python, you can get a performance boost, at least in theory, by using a compiled version of Python, such as Cython. There’s also another, but I can’t think of the name right off.
It’s really awesome because you can get from zero to something working much faster than with C/C++. The language is just higher level. Besides the interactive REPL which allows you to try things out interactively, the edit/upload/run cycle is also much, much faster than with C/C++.
Once you have something working, troubleshooting issues is also much easier and reliable.
All this time lost making MicroPython buildable with CMake just to… avoid learning C.
Oh, no. MicroPython on the esp32 is so much more enjoyable than C/C++ (I’ve programmed professionally in C/C++ for several decades). I’ve now done stuff where speed matters in MicroPython, such as a driver for an RGB TFT display, and the pure python performance is good enough. For example, updating a 320×240 16-bit color display in 70ms using byte pushing and WR line toggling all in python. According to the display specs it can be done 6-7x faster and I may end up rewriting this function in C to get that speed, but right now that task has dropped to the bottom of the priority list.
Marty is cool anyway. Not really into shoving interpreter into a MCU. Is usable on 32bit MCU with plenty of RAM and storage but just barely. Reminiscent of days when used animal skins and bone knives stuffing BASIC interpreter into tiny resourced processor.
E.g., the Basic Stamp?
Ha. Ya thats still around. Which BasicStamp is being addressed here? (Rhet.) Many TinyBASIC variants over the years on various MCUs too. The distinction should be made that some were and are externally compiled and not native interpreters. Original Stamp(BS-1) was extremely limited, slow, and not well received by MCU fanatics but educational use has allowed the manufacturer endure.
By comparison modern MCU (2020) have more resources than many of my first micro computers. Cavernous space and coronal discharge speed/power compared to grain of sand tumbling/rolling down a soft incline.
Disparity still exists between compiled and interpreted but blurring as time and tech move forward. Then there’s the applications that remove ambiguity.
Then there’s ridiculous
E.g. STM32 for a thermostat control on tankless water heater. Not PID , safeties or array sensors. Economics beats Need. Cheaper than the 8bit of original design.
micropython is a long way away from the bad old days of embedded basic. Especially on stm32f4 or bigger, most code runs barely/imperceptibly slower than C.
If you’ve got some critical sections you can wrap them in @viper decorator and have loops run the same speed as in C (it’s kinda python byte-compiled into asm).
If there’s a larger chunk you just need optimised you can write modules of code in C and import them like a python module, with the remaining bulk of your code in easy-to-write python code.
But for most projects we do at work, there’s no C code needed to be added – and these are registered medical devices, written in micropython.
Just take a look at OpenMV if you want another example of the high level projects you can build with micropython (yes their processing libraries are in C, but you get to use them in python).
micropython makes coding fast and easy, as well as performant, on cheap chips.
Oh no!, it’s a fanboy(girl).
As TinyBasic isn’t BASIC, microPython isn’t Python.
Missing something somewhere as in “certain core libraries.”. Fully functional works how then?
woo a Byte code interchange.
Im still waiting on native mode super fast Java cheap hardware processor which ‘will be everywhere’. My Unicorn says its coming soon but him not sure about Python version. Probaly its hanging out with native ANSI C microprocessor in a seedy bar somewhere in Cali or Tejas. Maybe call this one Sometimesoon compiler where JIT seldom JIT.
Im too well aware of Medical Equipment bureacracy and that a tongue depressor made from one company is medical equipment. Same tongue depressor from company ‘B’ is not. Cotton balls become a fiasco of BS and Electronic thermometers become hellfire expanse of qualifiers.
Even the word ‘certification’ has many definitions.
I cant remember any jobs/assignments that required only one computer language or at the very least one language, one Assembler(processor specific) and OS. There were a few that boiled down to Data Entry or Clerk. Its a paycheck but dont like living there.
Various IDE implied as well. Sure the job requirement may list one but interview/consultation reveals more.
Hey its 2020. Maybe thats a thing now. Not seeing it though.
Not that it really matters. Only delaying the eventuality that Scratch! (more likely Snap! presently) will be THE language. Even has a cool translate button so us creeky programmers can relate in “JavaScript, Lua, Dart, Python, or PHP. It can also be customised to generate code in any textual programming language.”
Damn them Googley bastards.
Yes I am a fan, and a contributing developer, not sure why that warrants such a sarcastic response.
I’m not talking about a “medical toothpick” with a micro crammed in, if you’re actually interested it’s a point-of-care diagnostic imaging platform. It’s designed to provide on the spot diagnosis of a range of illnesses / hormones / proteins (depending on test cartridges used), replacing the need for a lab based blood test in its target applications. If anyone wants to know more, I presented on the platform and the benefits (and challenges) micropython has provided at pycon au: https://www.youtube.com/watch?v=YovngSLXoxw
The point of my original post was that Micropython isn’t just a wild promise for future greatness, nor is it just about being the “in” language. Micropython an efficient functioning platform ready for use now.
Yes, in many cases you will need a slightly larger micro to run it effectively, however they’re often so cheap these days that the savings in development time more than make up for this.
And no, every package available on desktop python doesn’t “just work” on micropython, you do still need to write some code for most applications. However many do just work, and there is a rapidly growing library of micropython-ready packages for use.
In our case we would have used exactly the same micro to develop in C for it to handle the image processing, but we would have needed a much larger dev team to accomplish the same project timeline.
Yes there will still be a place for 20c 8051 processors developed in C for devices manufactured in the millions, just like there will be always be a place for larger applications using embedded linux. But for an ever growing middle ground micropython can provide so many real-world benefits to developers and product designers it deserves to be taken seriously.
Compiled code will always run faster than interpreted code. And, if the same application is always run at power-up, then what is the point of the onboard interpreter? Use all the RAM and ROM for the application and its data; don’t waste it on an interpretr.
Of course, with MicroPython you can *freeze* your code to eliminate the parsing and compilation steps.
Not every application running on an MCU needs to be blazing fast or running at bare metal. Most folks “playing in Arduino land” are building applications that don’t even tax 1/10th of the CPU core, and use hardly any of the peripherals. If your application needs C/C++ use that, if it doesn’t then don’t be so anti exploring other languages. Ease of development and fast iteration times are just a few of the amazing features of using CPY or MPY on a microcontroller.
And for those of you that are thinking “no real world applications would use Python on an MCU over C/C++” – You are totally wrong. It’s being used on cutting edge medical devices, satellites, scientific devices and more.
And when µPy is too fat, use https://sneklang.org ;-)