Arduino Development; There’s A Makefile For That

Hardware and software combined, Arduino does many things right. It lowers the entry level into embedded systems development with a nifty hardware abstraction layer. It aims for cross-platform compatibility by supporting Windows, Mac OSX, and Linux operation systems. It throws out the need for an external programmer to get you up-and-blinkin’ those LEDs quickly.

One thing most of us never cease to curse about, though, is the IDE. Many have cried out wildly against the Java-based text-editor for its cryptic compiling-and-linking process, its inability to accommodate bare C or C++ source files, and (shh!) its lack of Vim keybindings. Fortunately, our cries have been heard, and the like many community-based projects, the community fights back with a custom solution.

Calling all Grumpy Engineers: The Arduino-Makefile

Enter the Arduino Makefile.

What began as [Sudar’s] lightweight program to escape the IDE has become a fully-blown, feature rich Makefile that has evolved and adapted to grow with the changes of Arduino. With a community of 47 contributors, the Makefile enables you to escape from the IDE entirely by writing code in the cushy text editor of your choice and compiling with a simple incantation of make into your terminal, be you in Linux, Mac, or Windows.

Without further ado, let’s take a walking tour of the project’s highlights.

Cryptic Shortcuts–Begone!

For many beginners, writing (or even editting) a Makefile can cause some serious confusion with most Makefile authors’ shameless shortcut use. Make no mistake, the cryptic syntax of many Makefiles forms a concise list of instructions for compiling and linking your executable, but this recipe does seem a bit hard to parse for the uninitiated. What’s more, Makefiles also tend to throw bizarre errors (trailing whitespace anyone?) that are difficult to track down–especially when we want to spend most of our time bringing up embedded systems, not understanding the mechanics and syntax of a Makefile. Fortunately [Sudar] and the rest of the development team have made the interface very human readable.

Their solution: A two-part Makefile. For a simple project, your Makefile need be no longer than this snippet (inspired from the Makefile Examples):

BOARD_TAG    = uno
include $(ARDMK_DIR)/Arduino.mk

Project-specific settings (like which board you’re using) are outlined in this brief Makefile that then includes the larger Makefile which contains the actual nuts and bolts for building your code. The assumption here is that you’ve defined two environment variables, both ARDMK_DIR and ARDUINO_DIR that point towards the (1) Arduino Makefile directory and the (2) Arduino Installation directory. In addition, if you have additional libraries, you can include them with a line in your top-level Makefile that defines the filepath.

 USER_LIB_PATH += /home/my_username/my_libraries_directory

You’ll also need to add all libraries you’re using (both user-added and built-in) to the list of libraries like so:

ARDUINO_LIBS += Wire \
                SPI \
                my_custom_lib

The benefit of a “split-Makefile” setup is that the short, top-level Makefile hides the gritty compiler gymnastics involved in compiling and linking against the default and user-added Arduino Libraries. On the flip-side, this “mini” Makefile becomes a brief, informative summary of a few minor details, such as which Arduino Libraries are being used, that are likely more relevant to the author and future developers.

Raw C and C++ is In–if you prefer such things

Tired of that *.ino file extension? Tired of having to constrain yourself into those setup and loop functions? With the Makefile, you can quickly wish these away and write your code in raw C or C++. You can even neglect to #include <Arduino.h> if you want to work in vanilla C or C++ and disregard the Arduino libraries altogether.

That said, if you’re mixing C and C++, keep in mind that you’ll need to insert guards around your C header files like so:

#ifdef __cplusplus
extern "C" {
#endif

/// the rest of my C header file 

#ifdef __cplusplus
}
#endif

Adding Project Libraries

Possibly my favorite aspect of the Arduino Makefile is its flexibility to accomodate a richer file structure and painlessly split your project into multiple files. Let’s say I have a one-off project where it makes sense to include a custom library. Given the flexibility of the Makefile, you can define:

    USER_LIB_PATH+=.
    ARDUINO_LIBS += my_custom_library

in your project Makefile and produce a directory tree like so.

 ├── main.ino
 ├── Makefile
 ├── my_custom_library
 ├── my_custom_library.cpp
 └── my_custom_library.h

If you need more flexibility, you can also split your source code across several directories, keeping the Makefile in the same directory of your code that acts like a “main.” (In this case, main.ino defines setup  and loop.) To do so, your Makefile will, instead, have:

    USER_LIB_PATH+=../libs
    ARDUINO_LIBS += my_custom_library

in your project Makefile, and your directory structure will look like so:

 ├── libs
 │   └── my_custom_library
 │     ├── my_custom_library.cpp
 │     └── my_custom_library.h
 └── src
  ├── main.ino
  └── Makefile

Finally, we don’t need to constrain ourselves to writing just C++ classes to split projects into multiple files.  Since we’re really working with vanilla C++, you can freely split your project into multiple source (*.c, *.cpp, *.ino, *.pde) and header (*.h, *.hpp) files that live in the same directory as the Makefile, and the Makefile will compile them into one executable.

Other Micros are Fair Game Too

Finally, the Arduino-Makefile is also compatible with a host of other microcontrollers and programmers, not just the ATMEGA328P on the Uno. In short, this feature is an exposition of the features of AVRDUDE, the program for downloading and uploading code to various AVR microcontrollers. Last, but not least, it’s also Teensy-compatible.

Define Our Standard

If you’re looking to get comfortable with the Arduino-Makefile workflow, have a quick look at their examples directory for a host of different use-cases. You can also take a peek at my i2c_demultiplexing_demo source from a couple weeks back for yet another example. At the end of the day, Arduino, with its giant library collection, makes project prototyping fast. For bigger projects, though, we don’t tend to see any standard practices for file organization to make projects easier to navigate. That’s where you come in. With the flexibility of the Makefile, you get it all: the text editor you always wanted, the separate header and implementation files, a clean directory… Now it’s your shot to take this tool and refine your workflow into a method worth sharing with the rest of us.

32 thoughts on “Arduino Development; There’s A Makefile For That

  1. Uhm, well, it’s a Makefile. Nice, but not as grand as you make it sound. And when you’re on the verge of working “in vanilla C or C++ and disregard the Arduino libraries altogether” then you might as well use avr-gcc in the first place (for which similar Makefiles already exist).

    1. True, but I see this as a good tool to get people used to the Arduino environment to transistion away from that environment.

      People have weird mindsets about the entire platform no thanks to the Arduino’s tendencies to mask everything of interest.

    2. I agree! I think that if you do not like arduino IDE then just move on to avr-gcc. It’s the next logical step to take and opens up a lot more to whomever decides to go down that path. Although there is a bit of a learning curve and you need to get a programmer, this is just the nature of the beast if you want to keep down the path. The Arduino IDE is for beginners and quick nonsense projects in my oppinion. If you are an advanced programmer then play with the big boy toys.

  2. If one is working on windows platform it is even simpler to work with Atmel studio. Now version 7 is available.
    The Atmel AVRISP MkII programmer is cheap and dispense of a boot loader that take space in flash.

    1. +1

      Anyone who understands what a Makefile is should really just bypass the entire Arduino situation all together

      AVR-Gcc/AVRDUDE in Linux or Atmel Studio in Windows… I like them both the same

      1. yes and no.

        I wrote some ‘commercial’ arduino code (used in some projects that you can build) and while I’m ok at doing makefiles (my project did use them about 3 or 4 yrs ago when I started) my target audience of ‘makers’ will usually not understand the makefile idea and they need the IDE.

        this is what I like about arduino; I can live with the cli and emacs and gcc and makefiles; but others can just load the ino/cpp files into the IDE and have at it. everyone can be happy.

        I call that a huge win, actually.

        1. I agree, It’s easier for the knowledgeable to work around the crappy IDE than for the newbie to deal with make.

          I use Atmel Studio, which isn’t great but it’s very good. Then I release my code to my students and the public, who can use the Arduino IDE, keeping it easy.

          I will try this Makefile, I’ve tried other before. But it usually means giving up the Arduino IDE compatibility, which misses the point of Arduino.

          If one wants straight AVR code, use avr-gcc/avrdude tool set. If you loose Arduino IDE compatibility the is it Arduino still?

  3. You don’t even need to list your libraries anymore, my Makefile for a sketch that uses 9 libraries:

    BOARD_TAG = leonardo
    MONITOR_PORT = /dev/ttyUSB0
    include /usr/share/arduino/Arduino.mk

    Then I just call “make upload” and it builds, flashes and reboots the pro micro.

  4. It took me a long time to get into the whole Arduino thing, because of the uhmmm… let’s call it “lacking” IDE.

    But then I found out about VisualMicro which just lets me use Visual Studio 2010 or later, which I also use for work everyday. And other plugins for VisualStudio such as WholeTomato VisualAssistX really help to make Arduino programming fun. Not to mention the fact that VisualMicro is smart enough to do little things like close the serial terminal when it needs to upload a sketch, and reopen it afterwards.

    I like command line tools a lot, and I understand how a Makefile can make things easier for scripting things, but for the Arduino audience (i.e. beginning programmers) I think Atmel Studio or Visual Studio with VisualMicro is the way to go.

    1. yeah this is over two years late, but I have recently started using Visual Micro inside of VS Community edition 2015 (and now 2017)… and it’s pretty sweet. Code completion, definitions, source control… nice.

  5. I am using a similar build script for the Teensy-3.1 in my electronic drum project… I am not a fan of the Arduino IDE or framework, but Paul has invested tons of time to get a very simple real time audio library for the Teensy, and I am using that heavily. I have my own libraries for other things, and don’t have a single .ino file, but still get the advantages of powerful audio routines without needing to write it all myself. Best of both worlds!

  6. Neat.
    Now you just need to put these instructions back into the arduino-makefile README, or at least a TUTORIAL file in the repo.

    Even then, it’s kind of a roundabout way to go. Lots of settings, paths, files, etc to all manage. Again, we are confronted with the paradox of the Arduino IDE: It is a terrible piece of junk, but it is very easy to use, and to get something working. It still emulates the odd compile path of building up the core.a archive, then linking it at the end.

    One of these days, I am just going to make an Arudino veneer on top of NetBeans. Regular old NetBeans. MicroChip uses it for their whole IDE, and other than the slowness, it is actually quite good.
    Yes, an Arduino plug for netBeans already exists, but it’s a pretty horrible roundabout thing.
    Someone could just roll a whole ArudinoBeans package, with support for just a simple “New Arduino Project”, all the examples, etc.

  7. What would be pretty awesome is having the ability to use Arduino libraries in an AVR development setup (Atmel Studio). Sure you’ll have to be aware of the various hardware resources used by the various libraries but it would mean that you can easily leverage the hundreds of Arduino Libraries in your code without having to be tied to the limited Arduino IDE and the ‘setup, loop’ structure.

    And this is possible. Simply create a static library containing all the Arduino Core libraries. Be sure to comment out the main function and setup and loop function prototypes in main.cpp. Additional libraries can be included as source…at least this is what the Arduino IDE does behind the scenes.

  8. Say what you like .The great benefit of the Arduino IDE is that people with no knowledge of microprocessors can follow simple instructions and have some leds blinking in an hour on a new board they can buy for less than $3 -the Arduino Pro ,and then develop their skills from there . Its a lead in to greater things of which a makefile might be considered one.

  9. Arduino has launched a new project called “Arduino Builder” which is a command line binary tool to compile Arduino code.
    It is currently in beta but very soon it will replace the Java builder included in the IDE.

    It takes care of a lot of steps like automatic protype generation and dealing with libraries and dependencies

    If you want to use your IDE of choice but wanto to make sure yoy’re 100% compatible with the Arduino IDE this is the tool you should be using (it also supports all the different implementations of Arduino provided by the community)

    enjoy

    Here is the source code https://github.com/arduino/arduino-builder
    (from the readme)
    “This tool is able to parse Arduino Hardware specifications, properly run gcc and produce compiled sketches.

    An Arduino sketch differs from a standard C program in that it misses a main (provided by the Arduino core), function prototypes are not mandatory, and libraries inclusion is automagic (you just have to #include them). This tool generates function prototypes and gathers library paths, providing gcc with all the needed -I params.”

    1. that’s a typical arduino “my way or the highway” response that’s not very supportive of an open approach to software development, you’ll be recommending the closed-source single platform atmel studio next, oh wait….

  10. Folks,

    I spent about 4 hours hacking on Sudar’s teensy makefile trying to compile for stm32 bluepill on Ubuntu 16.04 x64. Gave up, my stm32duino install puts files (compiler, headers, libs, core, not in the ‘arduino’ directory, which makes Sudar’s makefile project croak in a big way.

    I did find a command line option though. Open your arduino sketch and compile with pref verbose on. Then copy the second arduino-builder line from the console and paste into a file named ‘go’ in your sketch directory. minimize the arduino IDE window (if you close the IDE it will remove the temporary build directories in /tmp). If you want, while editing go, remove the ‘-verbose’ arg.

    here’s my arduino-builder line
    /home/holla/arduino-1.8.2/arduino-builder -compile -logger=machine -hardware /home/holla/arduino-1.8.2/hardware -hardware /home/holla/.arduino15/packages -hardware /home/holla/Arduino/hardware -tools /home/holla/arduino-1.8.2/tools-builder -tools /home/holla/arduino-1.8.2/hardware/tools/avr -tools /home/holla/.arduino15/packages -built-in-libraries /home/holla/arduino-1.8.2/libraries -libraries /home/holla/Arduino/libraries -fqbn=Arduino_STM32:STM32F1:genericSTM32F103C:device_variant=STM32F103C8,upload_method=DFUUploadMethod,cpu_speed=speed_72mhz -ide-version=10802 -build-path /tmp/arduino_build_201126 -warnings=none -build-cache /tmp/arduino_cache_97419 -prefs=build.warn_data_percentage=75 -verbose /home/holla/Arduino/BlinkSTM32/BlinkSTM32.ino

    I can paste into the command and it compiles my sketch.

    here’s the output from my ‘go’ command with -verbose arg removed
    ===info ||| Progress {0} ||| [0.00]
    ===info ||| Progress {0} ||| [2.94]
    ===info ||| Progress {0} ||| [5.88]
    ===info ||| Progress {0} ||| [8.82]
    ===info ||| Progress {0} ||| [11.76]
    ===info ||| Progress {0} ||| [14.71]
    ===info ||| Progress {0} ||| [17.65]
    ===info ||| Progress {0} ||| [20.59]
    ===info ||| Progress {0} ||| [23.53]
    ===info ||| Progress {0} ||| [26.47]
    ===info ||| Progress {0} ||| [29.41]
    ===info ||| Progress {0} ||| [32.35]
    ===info ||| Progress {0} ||| [35.29]
    ===info ||| Progress {0} ||| [38.24]
    ===info ||| Progress {0} ||| [41.18]
    ===info ||| Progress {0} ||| [44.12]
    ===info ||| Progress {0} ||| [47.06]
    ===info ||| Progress {0} ||| [50.00]
    ===info ||| Progress {0} ||| [52.94]
    ===info ||| Progress {0} ||| [55.88]
    ===info ||| Progress {0} ||| [58.82]
    ===info ||| Progress {0} ||| [61.76]
    ===info ||| Progress {0} ||| [64.71]
    ===info ||| Progress {0} ||| [67.65]
    ===info ||| Progress {0} ||| [70.59]
    ===info ||| Progress {0} ||| [73.53]
    ===info ||| Progress {0} ||| [76.47]
    ===info ||| Progress {0} ||| [79.41]
    ===info ||| Progress {0} ||| [82.35]
    ===info ||| Progress {0} ||| [85.29]
    ===info ||| Progress {0} ||| [88.24]
    ===info ||| Progress {0} ||| [91.18]
    ===info ||| Progress {0} ||| [94.12]
    ===info ||| Progress {0} ||| [97.06]
    ===info ||| Progress {0} ||| [100.00]
    ===info ||| Sketch uses {0} bytes ({2}%%) of program storage space. Maximum is {1} bytes. ||| [13044 65536 19]
    ===info ||| Global variables use {0} bytes of dynamic memory. ||| [2816]

    Now I just edit my sketch ino file in vi and run go to compile. You can also copy the upload command from the console to go file to upload on compile success.

    This solution is simple and avoids makefile hacking and all the stuff required to compile arduino sketches from command line. BTW Sudar’s solution works great on atmega328 compiles. I’m not knocking it at all. I’ve used it or other predecessors for years now.

    Good luck and take the blue pill.
    Craig

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.