Software Development

758 Articles

Tracking Binary Changes: Learn The DIFF-erent Ways Of The ELF

April 7, 2019 by 3 Comments

Source control is often the first step when starting a new project (or it should be, we’d hope!). Breaking changes down into smaller chunks and managing the changes between them makes it easier to share work between developers and to catch and revert mistakes after they happen. As project complexity increases it’s often desirable to add other nice to have features on top of it like automatic build, test, and deployment.

These are less common for firmware but automatic builds (“Continuous Integration” or CI) is repetitively easy to setup and instantly gives you an eye on a range of potential problems. Forget to check in that new header? Source won’t build. Tweaked the linker script and broke something? Software won’t build. Renamed a variable but forgot a few references? Software won’t build. But just building the software is only the beginning. [noseglasses] put together a tool called elf_diff to make tracking binary changes easier, and it’s a nifty addition to any build pipeline.

In firmware-land, where flash space can be limited, it’s nice to keep a handle on code size. This can be done a number of ways. Manual inspection of .map files (colloquially “mapfiles”) is the easiest place to start but not conducive to automatic tracking over time. Mapfiles are generated by the linker and track the compiled sizes of object files generated during build, as well as the flash and RAM layouts of the final output files. Here’s an example generated by GCC from a small electronic badge. This is a relatively simple single purpose device, and the file is already about 4000 lines long. Want to figure out how much codespace a function takes up? That’s in there but you’re going to need to dig for it.

elf_diff automates that process by wrapping it up in a handy report which can be generated automatically as part of a CI pipeline. Fundamentally the tool takes as inputs an old and a new ELF file and generates HTML or PDF reports like this one that include readouts like the image shown here. The resulting table highlights a few classes of binary changes. The most prominent is size change for the code and RAM sections, but it also breaks down code size changes in individual symbols (think structures and functions). [noseglasses] has a companion script to make the CI process easier by compiling a pair of firmware files and running elf_diff over them to generate reports. This might be a useful starting point for your own build system integration.

Thanks [obra] for the tip! Have any tips and tricks for applying modern software practices to firmware development? Tell us in the comments!

WebAssembly: What Is It And Why Should You Care?

April 4, 2019 by 45 Comments

If you keep up with the field of web development, you may have heard of WebAssembly. A relatively new kid on the block, it was announced in 2015, and managed to garner standardised support from all major browsers by 2017 – an impressive feat. However, it’s only more recently that the developer community has started to catch up with adoption and support.

So, what is it? What use case is so compelling that causes such quick browser adoption? This post aims to explain the need for WebAssembly, a conceptual overview of the technical side, as well as a small hands-on example for context.

Continue reading “WebAssembly: What Is It And Why Should You Care?” →

Easy Git Repository Summaries With Web-git-sum

March 25, 2019 by 16 Comments

For those hosting their own git repositories there are a number of solutions for creating convenient web-accessible front ends, but [mitxela] wasn’t quite satisfied with any of them. After trying a number of alternatives and reflecting on his requirements, he realized that all he really needed was a summary page listing the latest commits, and a file tree with a list of branches and tags. To accomplish this, he created web-git-sum. It’s a bash script that runs on git’s post-receive hook and generates only two files: a summary page and an index of the repository. You can see a demo of the output at git.mitxela.com.

[mitxela]’s writeup goes into some detail on how git repositories work, how those repositories are served over HTTP, and covers a few of the different options for providing convenient and accessible web front ends. Not all repositories are alike, and what works well for one may not work or scale well for another.

Intrigued by the idea of a private git server? We covered exactly how to set one up (spoiler: it’s really easy.)

Show Your Skills With A Bootable CV

March 23, 2019 by 10 Comments

It’s a thankless task, searching for a job. You send off your CV, or resume, and it joins a thousand other destined for the round file. What on earth can you do to make your career stand out, and catch the eye of the recruiter?

Your bootable CV isn't eye-catching if the recruiter uses GitHub to view the PDF.
Your bootable CV isn’t eye-catching if the recruiter uses GitHub to view the PDF.

If you are [Pablo Jiménez Mateo], the answer is straightforward enough. Simply combine the document as a PDF with an x86 bootloader, to make a readable document that will also boot an x86 computer system. He can do this relatively easily by prepending the bootloader file to the PDF, as long as the “%PDF” header of the CV remains within the first 1024 bytes it will remain a readable document. Which it does, though as our GitHub screenshot shows, not in all PDF readers.

A bootable PDF is pretty cool and we have to salute his effort in getting it in front of us in the hope of  career boost, but it would be fair to admit that it’s a trick that has been done before. So it’s time to turn attention to the bootloader itself, whose code comes in the form of an extremely well-commented assembly file that loads some sprites and a border to a VGA screen that looks as though it might be the first room in a top-down adventure game. Through the code we can gain an appreciation of just how simple a bootloader can be, and that in itself makes this project worth a second look.

If writing your own bootloader interests you, that’s certainly a subject we’ve covered in the past. It’s possible to make bootable images very small indeed, even down to fitting in a Tweet.

Finally, An Open Source Calculator

March 12, 2019 by 104 Comments

Microsoft has released the code for the Calculator app. This move is the latest in Microsoft’s efforts to capitalize on the Open Source community. Previous efforts have been the Open Sourcing of an extremely old version of DOS, and shoehorning Linux into Windows somehow in a way that’s marginally more user-friendly than spinning up a VM or popping over to your Linux partition. Oh yeah, Microsoft bought Github. Can’t forget that.

The release of the code for the Calculator app means now you too can truly verify all your calculations are correct. To build the Calculator app, you’ll need a Windows 10 computer and Visual Studio. You might think that this is the same code that’s been shipping for 30 years — it’s a simple calculator, right? Not so: the Calculator for Windows 8 had a strange and odd bug where the square root of 4, minus two, did not equal zero. Floating point is hard, kids.

Of special interest to the community, it’s now possible to disable telemetry sent from the Calculator app to Microsoft servers. Yes, the Calculator app knows you forgot how to divide, and wow man, six times nine, you needed help with that?  Fortunately, telemetry can be disabled in developer’s builds by disabling the SEND_TELEMETRY build flag. Now Microsoft won’t know you don’t do math so good.

At the time of this writing, we could not be bothered to contact Microsoft to find out when the pinball game or Ski Free will be updated and Open Sourced.

The Bakery That Runs On Emacs

March 6, 2019 by 23 Comments

When it comes to managing ingredients and baking at a professional bakery, we know that most people would turn to an SQL database and emacs.  Really, what else do you need? Okay, so maybe there are a few who would think that emacs couldn’t help you with this, so, here’s how [Piers] uses emacs and PostgresSQL to manage the day to day needs at his bakery.

[Piers] had tried a spreadsheet to keep track of things, but didn’t really like it when he had to create a new recipe:  “lots of tedious copying, pasting and repetition of formulae” is how he put it. As a ex-professional programmer, [Piers] was familiar with emacs and so set up a daily worksheet in emacs using org-mode. Each morning he runs org-capture to create the template for the day’s work. Some code in the org file (run with org-babel) can run a query on the database. He’s created some code to set up each day’s journal entry and to run the complicated database queries that he needs.

There is a list of things that [Piers] is working on next, including ingredient order management and accounting, but it works for him. And to stop any potential flame wars that might break out, it’s good to mention that the system does just that: It works for him. There are other possibilities. Take a look at Al’s Editor Wars article, or Elliot’s rebuttal, or, ignore the wars and read this article on baking with steam.

“Good Code Documents Itself” And Other Hilarious Jokes You Shouldn’t Tell Yourself

March 5, 2019 by 159 Comments

Code documentation — is there anything more exciting than spending your time writing extensive comments? If I had to guess, your answer is probably somewhere along the lines of “uhm, yes, everything is more exciting than that”. Plus, requesting to document your code is almost like an insult to your well thought out design, this beautiful creation you implemented so carefully that it just has to be obvious what is happening. Writing about it is just redundant, the code is all you need.

As a result, no matter if it’s some open source side project or professional software development, code documentation usually comes in two flavors: absent and useless. The dislike for documenting ones code seems universal among programmers of any field or language, no matter where in the world they are. And it’s understandable, after all, you’re in it for the coding, implementing all the fun stuff. If you wanted to tell stories, you would have chosen a different path in life.

This reluctance has even formed whole new paradigms and philosophies claiming how comments are actually harmful, and anyone trying to weasel their way out of it can now happily rehash all those claims. But, to exaggerate a bit, we’re essentially villainizing information this way. While it is true that comments can be counterproductive, it’s more the fundamental attitude towards them that causes the harm here.

In the end, code documentation is a lot like error handling, we are told early on how it’s important and necessary, but we fail to understand why and instead grow to resent doing it again for that same old teacher, supervisor, or annoying teammate. But just like error handling, we are the ones who can actually benefit the most from it — if done right. But in order to do it right, we need to face some harsh truths and start admitting that there is no such thing as self-documenting code, and maybe we simply don’t understand what we’re actually doing if we can’t manage to write a few words about it.

So let’s burst some bubbles!

Continue reading ““Good Code Documents Itself” And Other Hilarious Jokes You Shouldn’t Tell Yourself” →