There are many ways to update an embedded system in the field. Images can fly through the air one a time, travel by sneaker or hitch a ride on other passing data. OK, maybe that’s a stretch, but there are certainly a plethora of ways to get those sweet update bytes into a target system. How are those bytes assembled, and what are the tools that do the assembly? This is the problem I needed to solve.
Recall, my system wasn’t a particularly novel one (see the block diagram below). Just a few computers asking each other for an update over some serial busses. I had chosen to bundle the payload firmware images into the binary for the intermediate microcontroller which was to carry out the update process. The additional constraint was that the blending of the three firmware images (one carrier and two payload) needed to happen long after compile time, on a different system with a separate toolchain. There were ultimately two options that fit the bill.
Performing over-the-air updates of devices in the field can be a tricky business. Reliability and recovery is of course key, but even getting the right bits to the right storage sectors can be a challenge. Recently I’ve been working on a project which called for the design of a new pathway to update some small microcontrollers which were decidedly inconvenient.
There are many pieces to a project like this; a bootloader to perform the actual updating, a robust communication protocol, recovery pathways, a file transfer mechanism, and more. What made these micros particularly inconvenient was that they weren’t network-connected themselves, but required a hop through another intermediate controller, which itself was also not connected to the network. Predictably, the otherwise simple “file transfer” step quickly ballooned out into a complex onion of tasks to complete before the rest of the project could continue. As they say, it’s micros all the way down.
It wasn’t long ago that we introduced you to a web site, the Godbolt compiler explorer, that allows the visitor to compile code using a slew of compilers and compare their output. We suspect some number of readers said, “Wow! I can use that!”, while perhaps everyone else said, “Huh?” Well if you were in the second group, you ought to watch [What’s a Creel’s] video below where he walks through using the website. He looks at four different algorithms using four different compilers and it is a good example of how you might use the tool to make decisions about how you write software.
At first glance, it might not seem to make sense to write shell scripts in C/C++. After all, the whole point to a shell script is to knock out something quick and dirty. However, there are cases where you might want to write a quick C program to do something that would be hard to do in a traditional scripting language, perhaps you have a library that makes the job easier, or maybe you just know C and can knock it out faster.
While it is true that C generates executables, so there’s no need for a script, usually, the setup to build an executable is not what you want to spend your time on when you are just trying to get something done. In addition, scripts are largely portable. But sending an executable to someone else is fairly risky — but your in luck because C shell scripts can be shared as… well, as scripts. One option is to use a C interpreter like Cling. This is especially common when you are using something like Jupyter notebook. However, it is another piece of software you need on the user’s system. It would be nice to not depend on anything other than the system C compiler which is most likely gcc.
Luckily, there are a few ways to do this and none of them are especially hard. Even if you don’t want to actually script in C, understanding how to get there can be illustrative.
A lot of programming languages these days feature lambda functions, or what I would be just as happy to call anonymous functions. Some people make a big deal out of these but the core idea is very simple. Sometimes you need a little snippet of code that you only need in one place — most commonly, as a callback function to pass another function — so why bother giving it a name? Depending on the language, there can be more to it that, especially if you get into closures and currying.
For example, in Python, the map function takes a function as an argument. Suppose you have a list and you want to capitalize each word in the list. A Python string has a capitalize method and you could write a loop to apply it to each element in the list. However, map and a lambda can do it more concisely:
The anonymous function here takes an argument x and calls the capitalize method on it. The map call ensures that the anonymous function is called once for each item.
Modern C++ has lambda expressions. However, in C you have to define a function by name and pass a pointer — not a huge problem, but it can get messy if you have a lot of callback functions that you use only one time. It’s just hard to think up that many disposable function names. However, if you use gcc, there are some nonstandard C features you can use to get most of what you want out of lambda expressions.
Source control is often the first step when starting a new project (or it should be, we’d hope!). Breaking changes down into smaller chunks and managing the changes between them makes it easier to share work between developers and to catch and revert mistakes after they happen. As project complexity increases it’s often desirable to add other nice to have features on top of it like automatic build, test, and deployment.
These are less common for firmware but automatic builds (“Continuous Integration” or CI) is repetitively easy to setup and instantly gives you an eye on a range of potential problems. Forget to check in that new header? Source won’t build. Tweaked the linker script and broke something? Software won’t build. Renamed a variable but forgot a few references? Software won’t build. But just building the software is only the beginning. [noseglasses] put together a tool called elf_diff to make tracking binary changes easier, and it’s a nifty addition to any build pipeline.
In firmware-land, where flash space can be limited, it’s nice to keep a handle on code size. This can be done a number of ways. Manual inspection of .map files (colloquially “mapfiles”) is the easiest place to start but not conducive to automatic tracking over time. Mapfiles are generated by the linker and track the compiled sizes of object files generated during build, as well as the flash and RAM layouts of the final output files. Here’s an example generated by GCC from a small electronic badge. This is a relatively simple single purpose device, and the file is already about 4000 lines long. Want to figure out how much codespace a function takes up? That’s in there but you’re going to need to dig for it.
elf_diff automates that process by wrapping it up in a handy report which can be generated automatically as part of a CI pipeline. Fundamentally the tool takes as inputs an old and a new ELF file and generates HTML or PDF reports like this one that include readouts like the image shown here. The resulting table highlights a few classes of binary changes. The most prominent is size change for the code and RAM sections, but it also breaks down code size changes in individual symbols (think structures and functions). [noseglasses] has a companion script to make the CI process easier by compiling a pair of firmware files and running elf_diff over them to generate reports. This might be a useful starting point for your own build system integration.
Thanks [obra] for the tip! Have any tips and tricks for applying modern software practices to firmware development? Tell us in the comments!
If you haven’t seen the quadcopter, it is basically a PC board with props. The firmware is open source but uses the Keil IDE. The CPU is an STM32 with 64K of program memory. In addition, the drone sports a wireless module, a digital compass, an altimeter, and a gyro with an accelerometer.
Although the post is really about the quadcopter, [Tim] also gives information about the Blue Pill which could be applied to other STM32 boards, as well. On the hardware side, he’s using a common USB serial port and a Python-based loader.
On the software side, he shows how to set up the linker and, using gcc, control output ports. Of course, there’s more to go to work the other peripherals, and Tim’s planning to investigate CMSIS to make that work easier. Our earlier post on STM32 prompted [Wassim] over on Hackaday.io to review a bunch of IDEs. That could be helpful, too.