Compiling Your Own Programs For The ESP8266

When the ESP8266 was first announced to the world, we were shocked that someone was able to make a cheap, accessible UART to WiFi bridge. Until we get some spectrum opened up and better hardware, this is the part you need to build an Internet of Things thing.

It didn’t stop there, though. Some extremely clever people figured out the ESP8266 had a reasonably high-power microcontroller on board, a lot of Flash, and a good amount of RAM. It looked like you could just use the ESP8266 as a controller unto itself; with this chip, all you need to do is write some code for the ESP, and you have a complete solution for your Internet connected blinking lights or WiFi enabled toaster. Whatever the hip things the cool kids are doing these days, I guess.

But how do you set up your toolchain for the ESP8266? How do you build projects? How do you even upload the thing? Yes, it’s complicated, but never fear; [CNLohr] is here to make things easy for you. He’s put together a video that goes through all the steps to getting the toolchain running, setting up the build environment, and putting some code on the ESP8266. It’s all in a git, with some video annotations.

The tutorial covers setting up the Xtensa toolchain and a patched version of GCC, GDB, and binutils. This will take a long, long time to build, but once it’s done you have a build environment for the ESP8266.

With the build environment put together, [CNLohr] then grabs the Espressif SDK from the official site, and puts together the example image. Uploading to the module requires pulling some of the pins high and some low, plugging in a USB to serial module to send the code to the module, standing well back, and pressing upload.

For his example image, [CNLohr] has a few WS2812 RGB LEDs connected to the ESP8266 WiFi module. Uploading the image turns the LEDs into something controllable with UDP packets on port 7777. It’s exactly what you want in a programmable, WiFi chip, and just the beginning of what can be done with this very cool module.

If you’re looking around for some sort of dev board with an ESP8266 on it, [Mathieu] has been playing around with some cool boards, and we’ve been looking into making a Hackaday version to sell in the store. The Hackaday version probably won’t happen because FCC.

Edit: [CNLohr] points out there’s a more preferred toolchain that most people have switched to.

73 thoughts on “Compiling Your Own Programs For The ESP8266

  1. I have never used or imported an unauthorised low power transmitter into the USA.

    Be interesting to see how many WS281x’s you can run directly off the esp8266, i figured it’d only be a few but i should NOT TRY IT AT ALL

    1. There are versions now that you can get that would pass the FCC tests (i.e. shielded, within specs, etc.) I’ll give it a shot on mine tonight to see how many I can drive. I personally would expect it to be 490.

      1. This is the problem we’re facing with an ‘official Hackaday ESP dev board’. Even though the module itself would pass FCC specs, we would have to certify the board again.

        TBH, this is the sort of thing someone should do a crowdfunding campaign for. I’d be down for throwing $10 at someone who would certify a board with the FCC.

        1. Someone that is doing FCC or CE test/measurements should have the layout. Thus, community test/measurements should be based on opensource layout.

          An assembley kit that requires soldering, ‘expert knowledge’ and parts from different sellers (e.g. flash chip, LDO) in order to make it work at all, avoids CE. Maybe this is also true for FCC?

          So this GPIO-approch has no problems with interrupts?

      1. That is only half the truth.
        Indeed you certify the CE yourself – but that doesn’t mean you can just say “yeah version 1.5 is basically like version 1.0, so we have lift-off”. Well you CAN, but you also have to provide any party requesting them with your test results within 48 hours or so. And if that party happens to be a competitor, they can sue your ass off.
        The most fun part of that situation: YOU have to know WHAT to test… EMV, insulation, whatsoever – finding out what applies to your design requires knowledge of the according laws and standards – most standards cost in the region of some hundred € upwards to even peep in there. So you hire someone to do the whole CE thing. In China they are cheap, but if they screw up, you are liable.

        With WiFi testing you might run into thousands of € before even selling a single board.
        I don’t even think the FCC testing is more expensive. Does anyone know details?

        This applies to ALL goods you import and sell to end-users in the EU – the importing party is liable for the CE conformity of your goods.

      1. Any way you can get the post updated to include a blirb about the new one (I am rather embarrased I didn’t check with the community earlier before releasing the video)? … Though the method I outline does work a-okay.

      1. The toolchain is the same. What’s different is the mechanism that builds it. Scripts used by Charles Lohr are extremely simple, and that’s all what’s good about them. Crosstool-NG OTOH is well known toolchain building framework. It is more automated (no need for manual downloads), easier configurable (standard Kconfig interface for toolchain options customization) and more powerful (e.g. allows building g++ and libc).

  2. The Raspberry Pi is an ideal environment for something like this to, because it is 3.3 v compatible, and has a bunch of GPIO’s to trigger the board into “program” plus is reasonably capable of compiling images for the esp8266.I saw CNLohr’s video a few days back and trying to get it to work on the Pi.

    1. I thought the same, though the power consumption of the ESP8266 (up to 300mA) is more than the 3.3v rail on the Pi can provide safely, no?

      With the beefed up B+ it seems more doable, but I wasn’t able to find a good citation that this much current is available, so I haven’t taken the plunge yet

      1. how about:

        – publish a link to oshpark and let people buy their own boards
        – publish a spreadsheet of parts, import into mouser or digikey

        this way people can get a “kit” even though nobody is selling a “kit”

      1. I believe that people have the right to choose how they want to share their work, so I’m not too fussed.

        What I am fussed about is that Espressif must be selling so many of these now, you’d think they’d invest in a proper translation of the datasheet…

        1. @Fennec agreed! Better documentation would really help the adoption of the SOC internationally.

          @eas, good point ! didn’t realize that it was closed source. Open source would’ve been better no doubt. Having said that, I still really like the lua approach for fast prototyping

      2. While open source would be preferable – is the default firmware it comes with open source? If not, the objection to a replacement that’s a blob isn’t super compelling, in that we’re no worse off, and are in fact better off because we at least have a choice. (it rankles more, perhaps, because lua itself is open source, so you’d hope the author would return the favor, but … better something than nothing. He could be selling it.)

        1. original software is as open as raspee is, you get SDK + binary blobs

          that is still better than some random chinese dude providing you those binary blobs wrapped in his own binary blob, espiecially because like you said he is using Open source Lua project, but decided not to publish sources to his port.

  3. Someone would earn a lot of attention if you can get one of these programmed using an arduino or USBTiny or one that can be used with the Arduino IDE(Integrated Development Environment)

    1. I tried a few things. The ESP has a low-power mode in which it is said to sip only 40uA or so of power, but apart from the RTC, nothing’s running. Seems the way to get the device out of low-power-mode is to basically reset it, meaning it’ll go all the way through a scan – associate – dhcp cycle, which means 5 seconds of using full power (70-100mA). In theory, this can be sped up, but with the lower level libraries being closed source, only Espressif themselves can do this.

      1. I believe the datasheet was claiming 10u and was updated to 60u for the EX chip. It appears that you need to be in sleep mode(0.31mA) to get woken by external interrupt. I hope they fix this and allow external IO wake up within the 60u.

        I think 60uA is not that bad, gets about 3-4 years of standby on a pair of AA(assuming no other transmission).

      2. Sprite_tm, I am just beginning with “c” language and especially IoT_Rtos_SDK. I would like to know ,whether there is an in-built RTC with esp8266? If not, then how to run a clock in esp8266 without using RTC chips.

        Thanks in advance!!

  4. How about something eminently more useful for the IoT? Two Words:

    Mesh Networking

    Espressif, open source community at large, OpenWRT gurus, CozyBit developers: Where is the 802.11s mesh networking support for the module?

    1. If software is not plug-and-play then it needs feedback from people like you who are trying to install it.

      If you want to mooch from free software you should at least make an effort to give back.

  5. Quote: cheap, accessible UART to WiFi bridge

    Actually the one thing I’ve not yet seen anyone do with this without requiring first to send AT commands to it to initialise.
    A simple TTL to wifi bridge would internet enable a whole bunch of stuff that already is spitting out data from a TLL port already built in.
    Either I’m the only one that things this would be useful over than perhaps all the companies selling such devices for upwards for $90, or it’s not possible.

    Yes, I’m too dumb to figure out how to do this myself and want spoon feeding.

    1. Those devices are expensive because of the niche they are used in.
      You can make a cheap WiFi to uart bridge with even the cheapest routers that support openwrt or the like, I have done it quite a few times.
      This is not really that attractive to hobby stuff, as you don’t get that much by cutting the UART cable. But when you put a web server on it and it has its own interface and is independent on the other side, that is much more useful.

      1. Yes. You can. But it’s a bit like using a nuclear bomb to build a harbour.

        A router is ridiculously over powered for the job.
        Hey this is what I’m doing now. I have a bunch of wires going to places on serial to USB converters running into a box reporting stuff.
        Problem is that this means wires everywhere. Also not possible to wire everything up. So I’d have multiple routers.
        Actually I’m using a couple of serial to network devices (with ghetto PoE) for this task too.
        Suddenly the power consumption is going up for 5 or so routers / ethserial devices around the house.
        The 8266 is a much better device for the job.

        Webserver on the 8266 isn’t needed. Just dumb bridge.
        Now yeah another application would be a webserver on the 8266 to control a device which takes serial commands, but I can’t see a need for that. If I wanted to control a device, I would connect to one IP and it will do the backend stuff. In Home auto, you don’t connect to each device to change it’s settings, you have a master that does all the heavy lifting.

        BTW 8266 to 1wire interface would be somewhat awesome.
        Again beats running cables and you can segment the 1wire network to cut down on one fault taking down everything.

        1. “A router is ridiculously over powered for the job.” I have seen one of these adapters a while ago using some sort of chipset normally found in a router. It can make sense cost wise. If you want it as simple as possible, maybe even a ENC28J60+micro can do it.

          I tried making that serial thingy you are talking about with NRF24 chips, it was no problem to have a central serial place. I did not get too far…no time for it.

          I am sure some people will come out with a serial/1wire/i2c/spi bridge for the ESP. I could use the serial bridge myself.

          1. Since I cannot edit..
            BTW, reading that thread you posted also helps to understand what the challenge is in making this into a simple Serial to WIFI bridge. Having thought that the serial port was dedicated to the task, that thread mentions it’s actually GPIO pins configured in serial mode and they can be configured for other things too. But it also sounds like folks are working on this.
            Something for the new year I hope :)

      1. Somewhat.

        One use is for offering the serial output (debug or information) on a TCP port to anyone that connects.
        that would be used for things that log. Like power consumption meters, weather stations.
        To take their output on demand to a box which is using it for XYZ.

        The second use if are a device which has a serial port for control.
        Again however the 8266 needs to listen for the remote device to connect to it and then forward data received to the TTL port. Example for this might be TV’s with serial input to change channels or other AV kit.

        Third use, two way comms. Both sending data to the device and listening for a response.
        But again the 8266 needs to wait for a connection from something remote.
        This might be say a washing machine, to start the program and report back when it’s finished (or more likely you’d do this with I/O)

      1. While the ESP8266 is like 10 times cheaper than the CC3200 Launchpad, I wouldn’t discount the CC3200 yet. At $30 It is still quite cost effective compared to some of the Arduino WiFi Shields or even the original LPC1768 mbed board. The CC3200 board (based on a Cortex-M4 core BTW) is quite a bit more sophisticated that the ESP8266. It also comes with excellent TI documentation, a professional level SDK and arduino support in the form of the energia environment. I’ve yet to see this ease of use associated with the ESP8266. The Lua firmware while promising is buggy and has serious limitations. The ESP8266 C/C++ toolchain approach is a tricky and time consuming proposition. Its poor documentation also does not help. The AT serial approach is probably the most practical one right now for the ESP8266.

        1. Yes, from the development point of view they are very different.
          At hobby level, the CC3200 is not that attractive, exactly because of the price. You can definitely swap it for a router which is a more popular environment, unless you need battery power.

          @halherta I have only used the ESP with AT commands and that is tricky too. I believe expressif is waiting for the community to design a better FW, or at least check for inputs on how to make a better one.
          On the other hand, once the toolchain is much easier to use(1 click install, VM image or something) i don’t see why it would not be used. Make it user friendly enough and it will kick the ass of spark core.

          1. @Bogdan, In order for the ESP8266 to really take off in the hobbyist /maker community it’ll have to get a heck of a lot easier to use than simply 1-click install of the toolchain or a VM. It’ll need to be as easy to use as Arduino and/or mbed i.e. in addition to the toolchain, a high level api and an easy to use IDE are essential.

            Alternatively due to the higher performance and larger memory densities of new microcontrollers, an on-board Lua interpreter is very attractive as it is relatively easy to use and eliminates the need to have a compiler and IDE. This is where nodemcu’s esp8266 Lua interpreter comes in. It still seems to be a little buggy but it is on the right track.

            This approach is increasingly being used in hobbyist platforms such as the Tessel (Javascript/Lua), espruino (Javascript), sparkcore (Javascript?), and MicroPython (Python). I believe that MicroPython has also been ported (partially)to the ESP8266 http://hackaday.com/2014/11/29/micro-python-now-runs-on-the-esp8266-contributors-wanted-to-get-wifi-working/

            The ‘interpreter on a devboard with WiFi’ approach is I believe the way forward especially as more hobbyists tend to move away from C. C will always be king for optimized embedded development, while the interpreter approach will be ideal for fast prototyping.

            Lua is ideal as an embedded system interpreted prog lang as it is very lightweight and has a successful history of being implemented on microcontrollers via the ‘eLua’ project.

  6. You dont need to do CE testing if you dont plan to sell in Europe. USA requires no CE mark. I do CE testing quite a bit at work on custom modules built in house. If you do need to get CE or another IEC related test done its going to be super duper expensive, and your engineer better be good, compliance testing can be a game ender for alot of products that dont have deep pocket funding.

    1. Please Rob. Stop the FUD about CE test/certification.

      My company (none different from most other small passionate product developments/engineering services companies) have helped out companies/start-ups push out products for years onto the market. The cost is usually somewhere in the order of $7k-$70k (in EU) with third party (notified body) tests/cert depending on how many retests you need to do and also depending on the product type/category. The engineering-work for doing iterations of the products for retests are usually quite small as it is mostly small modifications (filters/protection/pcb-layout changes etc). My experience is that those costs never go over the retest/cert costs.

      A rule of thumb is that the certification are a mere fraction of the total cost of taking the product to market. Not considering tooling, factory setup, sourcing and purchasing stock/components the development costs alone are usually many times the cert costs. This rule of thumb is very obvious for a product like a car or a smartphone that you would consider to have a high complexity and high dev cost. But if your making a simple blinking LED with a 555 and a battery then the third party CE cert cost is going to of course be unproportionate. On the other hand you don’t need a third party test/cert for CE.

      To put a legal CE mark on a product does not need to cost that much at all as you can do self certification (many products on the market are self certificated – google: CE Self Certification) as long as you can uphold the right documentation (Deceleration of conformity, Technical Files etc) and comply with the harmonized standards. Most engineering companies that I know of (including us) own most of the equipment needed for preliminary testing and after that you just need to rent a EMC test lab+technician for a couple of hours (costs around $200/hour here in Sweden) per test round.

      When we started out with our company we had nothing. We solved it by going to our University and loaned the equipment or did the tests at the Uni. In some cases we befriended other development/test companies and bribed with beer and got to do some testing for next to nothing. If I had listened to guys like you before we started out we would have never dared to go ahead with developing products.

      Seeing big obstacles is the biggest obstacles of all. I prefer to see it as a challenge or a problem to be solved. The rest is just perspiration, stubbornness, a little bit of luck and inventiveness along the way.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.