Scripting languages are for large computers, right? “Real” embedded device work is a hellish, never-ending cycle of code, compile, and re-flash. Well, I used to think so too, but with the proliferation of scripting and other interactive languages to microcontrollers over the last few years, the hurdle to interactive development on the small chips has gotten a lot lower.
On the ESP8266 platform, I’ve tried out NodeMCU’s Lua and ESP8266 BASIC. (For the last half-year, I’ve been using the awesome Mecrisp-Stellaris almost exclusively on the STM32F1xx and F4xx chips, but haven’t dipped into ESP8266 Forth yet.)
NodeMCU is great because it’s got everything you could want built in, and through cloud services it’s easy to get a tailored build made that maximizes free flash memory for your projects. I just don’t dig the asynchronous Lua thing (you might, try it!). ESP BASIC has a different set of libraries, and is missing MQTT for my purposes. Still it’s pretty slick, and worth a look.
So when the MicroPython folks announced that they were releasing the binary builds for the ESP, I thought it was time to give it a spin. I’ve used Python for nearly twelve years now, so it’s like a comfortable shoe for me. Would MicroPython be the same on the ESP8266? The short answer is yes and no.
Installation
The install couldn’t have been easier, thanks to the MicroPython folks releasing binaries a few days ago (a special thanks to the Kickstarter backers on that one!).
Just download the ESP8266 binary image and flash it into the chip. I used the ever-popular esptool.py, which means typing esptool.py -p /dev/ttyUSB0 write_flash 0x0000000 esp8266-*.bin and you’re set. A few minutes later, I reset the ESP and got a serial port. Connecting to it at 115,200 baud brings up a Python prompt.
>>> print("hello world") hello world
That was easy.
Getting Settled In
If you type Python code into something text-based like iPython or similar, you’ll find yourself at home. The MicroPython folks really got the user-experience part right: tab-completion brings up an object’s methods, up-arrow recalls the last command for editing, and so on. A nice touch is that control-e allows you to paste entire blocks of code from a file at once. In short, it’s a nice workable Python environment entirely hosted off of the ESP8266. Kudos!
With the logistics under control, it was time to find a tutorial. The official tutorial on the MicroPython site is fantastic. If you know Python already, you’ll be up to speed with ESP8266 MicroPython in a half hour.
MicroPython and ESP8266 Extensions
Printing “hello world” over the serial terminal is great, but what about blinking LED’s — the “hello world” of the microcontroller set? There’s a machine module which you can import, and it has a Pin object.
import machine p = machine.Pin(2, machine.Pin.OUT) p.low() p.high() # or p.value(not p.value()) # toggles
If you haven’t played around with interactive environments on a microcontroller, there’s something cool about typing a command and getting an instant response. But the productivity increase from being able to test out code on the fly is what will keep you hooked in the long run.
Another standout is the ease of putting the ESP8266 to sleep, which is absolutely essential if you’re trying to run on batteries, for instance. esp.deepsleep(10*1000000) sleeps for ten seconds — in microseconds. And don’t forget to tie the WAKE pin, GPIO 16, to the reset pin.
Networking and the Web
But you’re using the ESP8266 because you want WiFi, right? That’s where the network module comes in. You can go read the docs for more details, but getting set up is as simple as
import network
n = network.WLAN(network.STA_IF)
n.active(True)
n.connect('<your ESSID>', '<your password>')
# n.isconnected()
# n.ifconfig()
Once the network is up, you can do networky things (more on this in a minute). The tutorial has you watching ASCII Star Wars in no time. I had to use the ctrl-e feature and paste all the commands at once to make it work. Who is going to be the first to display this on an LCD screen for a standalone device?
Although it doesn’t mention this in the docs, the settings seem to be stashed in flash somewhere. The ESP8266 comes up on my WiFi network every time I plug it in. Nice.
Disappointment Strikes!
My first real disappointment came when digging around in the os module — with a small-memory ESP8266 like the one I’m using, there isn’t enough space for a filesystem. os.listdir() returns OSError: [Errno 19] ENODEV. Consulting the website, you can’t save and load files internally on the ESP8266 unless the module has 1M of flash or more. For the super-cheap ESP8266 devices like the ESP-01 or even the Wemos D1 that I’m using, that’s a show-stopper. Without the ability to save user code on the device, it’s just a toy. Get a newer module with more flash if you want to save your code to the module.
(Edit: The filesystem error that I was getting seems not to be related to the flash memory size. Adding the --flash_size=32m option to the esptool command got a working filesystem created on a brand-new Wemos D1 Mini, but failed on a couple of similar units that had previously been used. There’s some glitch in the install here, but it will presumably get worked out. Take my complaints about the filesystem with a grain of salt.)
Next, I looked into what HTML-parsing and web server modules were included. None. To the MicroPython team’s credit, the socket module is a good copy of the desktop Python version, and you can get a very simple webserver coded up in a few lines, but that’s a few more lines than I wanted to type. Half the reason I use Python these days is for things like BeautifulSoup, lxml, or requests. There is no MQTT client built in either, and all of my ESP8266 devices need to speak MQTT these days.
Not having these tools at hand, and faced with doing all the low-level HTML stuff by hand, I shed a decently sized tear. Is this the end for ESP8266 MicroPython?
Emotional Rollercoasters
Then I found micropython-lib. This is where people are working on MicroPython libraries to bring exactly those functions over to the MicroPython platform in general, if not the particularly constrained ESP8266 port. So it shouldn’t be too hard to do HTML-heavy work using the libraries at hand. MQTT support is being worked on and has just recently been merged into the main micropython-lib repository.
With an ESP that has more flash than mine, you could easily add a few of these modules to your programs, and you’d be nearly in embedded nirvana. And given that all the modules are written in (a limited dialect of) Python, they’re easy to read, maintain, and expand. I have high hopes for where the community will take this.
And there are a lot of cool modules built in: SPI, I2C, OneWire, NeoPixel, Timers, PWM and Servo libraries, for instance. You can do a lot, very easily, with what’s built in.
Conclusion, and Code
Faced with an ESP8266 module with too little flash space to do anything super-fancy, I decided to try one last trick with what I had. The NeoPixel / WS2812B driver support was built-in, and it had socket support, which suggests: networked color blinkies. If you’ve never set up TCP socket connections before, it might surprise you just how simple it can be.
First, some server-side code:
import socket
def server_init():
s = socket.socket()
s.bind( ('', 31337) )
s.listen(1)
c, a = s.accept()
print "Client connected"
return c
c = server_init()
c.send("\x10\x2A\x0F")
The server_init() creates a socket, binds it to any IP address at a given port, listens for one connection, and then sits and waits at the s.accept() state for a client to connect, at which point it returns a client object that can send or receive arbitrary data to the other side. The data sent uses Python’s standard string encoding of hex bytes, prefixing with \x. You can alternatively use chr(16) + chr(42) + chr(15) if you want to type the same number out in decimal.
On the ESP8266, because it’s all in Python, the code is similar. In fact, the same basic code would work between two computers running vanilla Python just as well as the ESP8266 running MicroPython.
import socket
import neopixel
import machine
def connect():
s = socket.socket()
s.connect( ('192.168.178.25', 31337) )
return(s)
def remoteRGB(s):
n = neopixel.NeoPixel(machine.Pin(2), 1)
n[0] = s.recv(3)
n.write()
s = connect()
while True:
remoteRGB(s)
Connecting to a socket connection is simpler than creating a server. All the client has to do is connect. The remoteRGB() function then listens for three bytes, and sets the LED’s color accordingly. As long as neither side closes the connection, any three bytes coming across the WiFi will get interpreted numerically and fed out to the WS2812. If you’ve got the parts lying around, try it out.
(Some) Batteries Included
In all, MicroPython on the ESP8266 is a mixed bag. It’s awesome to be able to type code in real time and watch the LEDs light up. When you’re doing something more elaborate, like talking to peripherals over SPI, the ease of interactive debugging can get addictive. With a big community behind it, and given how simple it is to develop and port modules in Python, I can see this project getting big fast, and not just on the ESP8266.
But by ESP8266 standards, MicroPython is a memory hog. Because I used an el-cheapo module, I didn’t even have enough flash to test out the filesystem and make full use of the available libraries. Even with a bigger part, it’s clear that the developers of the ESP8266 MicroPython are pushing the limits of what’s possible, and they are certain to run up against more memory (RAM and flash) constraints in the future.
One of the joys of Python on a big computer is the “batteries included” philosophy of having all the modules you could want at your fingertips (or easily installable) at all times, and this just won’t work on the ESP8266. There’s always going to be a tradeoff between space for user code and space for libraries and modules.
On bigger chips, or course, these tradeoffs would be less painful, but because of WiFi the ESP8266 is impossible to ignore. Personally, if MQTT support were in the mainline binary distribution, I’d switch over from NodeMCU/Lua to MicroPython for ESP8266 development right now. As it stands, I’m going to keep my eyes on the project, and buy some ESP8266’s with more flash.
4mb of Wemos D1 is not enough ?
Great write up -thanks, off to get a ESP with more flash.
It’d be handy if that hackaday store page actually listed how much flash is on that esp8266.
>unless the module has 1M of flash or more. For the super-cheap ESP8266 devices like the ESP-01 or even the Wemos D1 that I’m using, that’s a show-stopper.
I’m confused. The D1 Mini Technical Specs says “Flash 4M bytes”. The ESP8266 Arduino core’s ESP.getFlashChipRealSize() confirms this on my WeMos D1 Mini.
I’ll have to go double-check! I might be a big dummy, and you might be my hero. I have no idea why I just assumed (and never verified) that they had 4M bits rather than bytes…
Pretty sure the D1 Mini is 32Mbits/4MBytes
@phlogiston: You’re 100% correct. The chip on mine is _also_ 32 Mbits = 4M. Verified with `esptool.py flash_id`.
To enable the remaining storage, you have to pass a flag to esptool.py. I will update the article ASAP to reflect this.
Thank you very much! This doubles the coolness.
I upload my custom build to my WeMos D1 Mini with the command:
esptool.py –port /dev/tty.wchusbserial1420 write_flash -fm dio -fs 32m 0 build/firmware-combined.bin
I did the same, multiple times. What ended up working for me was running the esptool.py erase_flash command a few times before flashing.
I noticed that my old WiFi settings were still present, even after flashing MicroPython in, which suggested incomplete erasing of the memory. So I hammered it, and that seems to have worked. (Spooky!)
That’s great. I was worried that the specs and test output had lied and I was going to have to reconsider using the D1 Mini as my go to ESP8266 board. I’m very glad I was able to, in a small way, help you out. Your excellent articles have taught me a lot. Thanks!
ok, I’ll give this a try. I assume that the huzzah style module has enough flash to ‘run’ the filesystem and save/load files?
what’s the best currently buyable (easily) platform for testing MP? I like the esp modules but if there are better ones that have lots of support and won’t be EOL soon, I’m willing to cross over ;)
I was googling yesterday and it seemed hard to find info about memory sizes with which the modules are shipped. For example, there’s that official ESP wiki page describing various models yet there’s no indication about flash sizes with which those modules might be shipped. I’ve found info on how to check flash size of an existing module, so I’ll check my modules I bought recently, but it’s not that easy to do with ESP-12 I have since they’re SMD and I don’t have any sockets for them – so it’d involve soldering pins to a module just to see its flash size.
If you own the module, esptool.py flash_id will tell you the ID number, which you can then lookup http://code.coreboot.org/p/flashrom/source/tree/HEAD/trunk/flashchips.h
If you’ve already got MicroPython on the chip, for instance: import port_diag ; port_diag.main() will tell you all about the flash:
FlashROM:
Flash ID: 1640e0 (Vendor: e0 Device: 4016)
Flash bootloader data:
Byte @2: 00
Byte @3: 40 (Flash size: 4MB Flash freq: 40MHZ)
“””There is no MQTT client built in either, and all of my ESP8266 devices need to speak MQTT these days.”””
“micropython/esp8266/README.md”:
Supported features include:
– REPL (Python prompt) over UART0.
– Garbage collector, exceptions.
– Unicode support.
– Builtin modules: gc, array, collections, io, struct, sys, esp, network,
many more.
– Arbitrary-precision long integers and 30-bit precision floats.
– WiFi support.
– Sockets using modlwip.
– GPIO and bit-banging I2C, SPI support.
– 1-Wire and WS2812 (aka Neopixel) protocols support.
– Internal filesystem using the flash.
– WebREPL over WiFi from a browser (clients at https://github.com/micropython/webrepl).
– Modules for HTTP, MQTT, many other formats and protocols via
https://github.com/micropython/micropython-lib .
MQTT is one of the Kickstarter stretch goals. It’s being worked on.
https://github.com/micropython/micropython/issues/2055
umqtt has been released. Code, initial docs and examples at:
https://github.com/micropython/micropython-lib/tree/master/umqtt.simple
I have absolutely no doubt that the community will hammer these last little bits into place.
The brilliant thing about opening up the ESP8266 binaries is that there’s going to be a lot more eyes looking at the bugs, and a lot more Python hackers working on extending the system out. I’m stoked to see this grow.
and this is why arduino has so much popularity and traction. the sheer number of modules to control anything without writing drivers or code is enough to make even experts use it.
I know of at least 2 commercial products that use Arduino inside because it is the best Embedded Rapid Design choice out there right now.
Here is hoping they get the footprint down so that smallish programs can be saved on the cheaper and plentiful older units.
Just curious, when you say those commercial products use an Arduino, are they using an Arduino capable AVR chip, or wasting the money on a full, legit Uno (hopefully a Micro)?
Dosen´t the WeMos mini d1 have 4MB of flash? The specs say so: http://www.wemos.cc/Products/d1_mini.html
Ack! Yes, fixing. Hold the comments…
There is a filesystem – “If your devices has 1Mbyte or more of storage then it will be set up (upon first boot) to contain a filesystem. This filesystem uses the FAT format and is stored in the flash after the MicroPython firmware.”
http://docs.micropython.org/en/latest/esp8266/esp8266/tutorial/filesystem.html
>>> import os
>>> os.listdir()
[‘boot.py’]
And basic operating system services
eg. uos.listdir()
http://docs.micropython.org/en/latest/esp8266/library/uos.html
If you compile your own firmware, you can add your own scripts with the build, including customising boot.py, which lets you run whatever you like on power on/reboot.
You can even access the filesystem over the WebREPL cli tool.
https://github.com/micropython/webrepl
Doesn’t the Wemo D1 have……. just kidding. I’m glad you covered this because I have been working on your other tutorial with the Wemo and MQTT. I was excited to see the announcement from MicroPython, but couldn’t find that MQTT was available, glad to see that I can switch to this, lua is alright but I already know python.
Nice!. Just to mention there is an ESP8266 Micropython contest on hackaday.io: (https://hackaday.io/contest/12641-esp8266-micropython-contest)
I’m surprised you’ve tried NodeMCU and ESP8266 Basic but not tried Espruino, which runs really nicely on ESP8266 AND has a name that matches the chip entirely by accident!
http://www.espruino.com/EspruinoESP8266
Last I checked, esptool.py wouldn’t work with Python 3. Does anyone know if this is still the case?
I’ve been meaning to go through it carefully and make it compatible with Python 3.x, but haven’t had the time+motivation work out simultaneously yet.
Seems silly if it’s still Python 2 only–even MicroPython is Python 3. It’s not hard to write code that works in both.
Oh, *Micro* Python. When I saw the link at the bottom of the badminton robot article I read that as MontyPython.
“Would you like an ESP8266? It’s waffer thin!”
I’ve done some fairly important stuff with NodeMCU for my employer and some nice stuff for myself. Simple stuff, but important and nice.
The main limit with scripting languages and the ESP isn’t the flash, but the RAM, which limits the heap. Scripting languages with file systems and other modules need soem of that RAM to take care of housekeeping and they have to reserve RAM for all their features, even the ones you don’t need.
This is why you won’t find too many generic full-function webservers in any ESP scripting language; doing it all takes up all the RAM. So you have to code what you need and throw away what you don’t. The person doing an all-purpose module can’t know what you need and don’t need so you are unlikely to find a module tailored to your specs, but with a few lines of script you can throw up a web server that will do exactly what you need with minimal resource waste.
There are also some gotchas in ESP land that relate more to the firmware than to the scripting layer. You have limited buffers for sending and receiving data, and if you want to serve a webpage longer than about 1.5K you need to do it in chunks and ignore other traffic while you’re farming it out. Some of this is still very poorly documented for scripting language users; the docs assume you will go to the SDK, which would be nearly incomprehensible if you are actually a Lua or Python programmer even if a lot of it wasn’t poorly translated from Chinese.
Snakes. Why’d it have to be snakes?
Micropython on the ESP8266 is awesome! But I’ll wait for the ESP32!
I’m having problems with the build dated July 22 2016. I get verification errors on both a nodemcu and a d1 mini. I tried an earlier version of esptool, but it behaves the same way, Anyone else seeing this?
I was getting errors like yours too, like when I tried to list the files on the board it gave me an exception, when the device booted, it showed a message about corruption on the filesystem, etc..
Some forums suggested that the error could be caused by insufficient current being fed to the board, so I decided to use a powered usb hub as a proxy to flash it. Worked flawlessly every time. :)