Software Development

840 Articles

Assembly Language For Real

August 25, 2020 by 48 Comments

We all probably know that for ultimate control and maximum performance, you need assembly language. No matter how good your compiler is, you’ll almost always be able to do better by using your human smarts to map your problem onto a computer’s architecture. Programming in assembly for PCs though is a little tricky. A lot of information about PC assembly language dates back from when assembly was more common, but it also covers old modes that, while still available, aren’t the best answer for the latest processors. [Gpfault] has launched a series on 64-bit x86 assembly that tries to remedy that, especially if you are working under Windows.

So far there are three entries. The first covers setting up your toolchain and creating a simple program that does almost nothing. But it is a start.

Continue reading “Assembly Language For Real” →

Frances Allen Optimised Your Code Without You Even Knowing

August 25, 2020 by 9 Comments

In 2020, our digital world and the software we use to create it are a towering structure, built upon countless layers of abstraction and building blocks — just think about all the translations and interactions that occur from loading a webpage. Whilst abstraction is undoubtedly a great thing, it only works if we’re building on solid ground; if the lower levels are stable and fast. What does that mean in practice? It means low-level, compiled languages, which can be heavily optimised and leveraged to make the most of computer hardware. One of the giants in this area was Frances Allen, who recently passed away in early August. Described by IBM as “a pioneer in compiler organization and optimization algorithms,” she made numerous significant contributions to the field. Continue reading “Frances Allen Optimised Your Code Without You Even Knowing” →

Don’t Let Endianness Flip You Around

August 4, 2020 by 26 Comments

Most of the processor architectures which we come into contact with today are little-endian systems, meaning that they store and address bytes in a least-significant byte (LSB) order. Unlike in the past, when big-endian architectures, including the Motorola 68000 and PowerPC, were more common, one can often just assume that all of the binary data one reads from files and via communication protocols are in little-endian order. This will often work fine.

The problem comes with for example image formats that use big-endian formatted integers, including TIFF and PNG. When dealing directly with protocols in so-called ‘network order’, one also deals with big-endian data. Trying to use these formats and protocol data verbatim on a little-endian system will obviously not work.

Fortunately, it is very easy to swap the endianness of any data which we handle. Continue reading “Don’t Let Endianness Flip You Around” →

Busting GPS Exercise Data Out Of Its Garmin-controlled IoT Prison

July 28, 2020 by 21 Comments

If you take to the outdoors for your exercise, rather than walking the Sisyphusian stair machine, it’s nice to grab some GPS-packed electronics to quantify your workout. [Bunnie Huang] enjoys paddling the outrigger canoe through the Singapore Strait and recently figured out how to unpack and visualize GPS data from his own Garmin watch.

By now you’ve likely heard that Garmin’s systems were down due to a ransomware attack last Thursday, July 23rd. On the one hand, it’s a minor inconvenience to not be able to see your workout visualized because of the system outage. On the other hand, the services have a lot of your personal data: dates, locations, and biometrics like heart rate. [Bunnie] looked around to see if he could unpack the data stored on his Garmin watch without pledging his privacy to computers in the sky.

Obviously this isn’t [Bunnie’s] first rodeo, but in the end you don’t need to be a 1337 haxor to pull this one off. An Open Source program called GPSBabel lets you convert proprietary data formats from a hundred or so different GPS receivers into .GPX files that are then easy to work with. From there he whipped up less than 200 lines of Python to plot the GPS data on a map and display it as a webpage. The key libraries at work here are Folium which provides the pretty browsable map data, and Matplotlib to plot the data.

These IoT devices are by all accounts amazing, listening for satellite pings to show us how far and how fast we’ve gone on web-based interfaces that are sharable, searchable, and any number of other good things ending in “able”. But the flip side is that you may not be the only person seeing the data. Two years ago Strava exposed military locations because of an opt-out policy for public data sharing of exercise trackers. Now Garmin says they don’t have any indications that data was stolen in the ransomware attack, but it’s not a stretch to think there was a potential there for such a data breach. It’s nice to see there are Open Source options for those who want access to exercise analytics and visualizations without being required to first hand over the data.

Beyond Printf(): Better Logging Practices For Faster Debugging

July 21, 2020 by 27 Comments

All of us who do some programming know that logging is a time-tested way to output messages about the internal state of our code. With varying degrees of adjustable granularity, these messages allow us to keep track of not only the state of the application, but also its overall health. When things do end up going FUBAR, these log messages are usually the first thing we look at as a software equivalent of a Flight Data Recorder in an airplane.

Spending some time and care in not only designing the logging system, but also in deciding what should be logged and at what level, can make our future self appreciate life a lot more. We’re all familiar with the practice of ‘printf-debugging’, where logging is added as part of the usual post-crash autopsy as one tries to figure out what exactly went wrong. It’s one way of doing it, and eventually it works, but we can do much better.

People are lazy, and you’re only going to stick to good logging practices if they are at least as easy if not easier than sprinkling printf() statement throughout the code. Yet at the same time we want to be able to handle things like log levels and newlines without too much extra typing. The most successful logging libraries are built with this

Continue reading “Beyond Printf(): Better Logging Practices For Faster Debugging” →

The rust language logo being branded onto a microcontroller housing

Will 2020 Be The Year Of Rust In The Linux Kernel?

July 15, 2020 by 55 Comments

One problem with modern programming languages is the reach their overly enthusiastic early adopters have nowadays thanks to the internet. As a result, everyone else’s first encounter with them are oftentimes some crude, fanboyish endeavors to rewrite every single established software project in that shiny new language — just because — which may leave an off-putting taste behind. However, Rust certainly seems to have outgrown this state by now, and with its rising popularity within the general developer population, it’s safe to say it will stick around. Will it fully replace C one day? Probably not, but there’s a big chance for coexistence, and [Nick Desaulniers] got the ball rolling for that within the Linux kernel.

Now, before you storm off pledging your allegiance to C by finding a new operating system: nothing is happening yet. [Nick] simply tested the waters for a possible future of Rust within the Linux kernel code base, which is something he’s planning to bring up for discussion in this year’s Linux Plumbers Conference — the annual kernel developer gathering. The interesting part is [Linus Torvalds]’s respone on the LKML thread, which leaves everyone hoping for a hearty signature Rust rant akin to his C++ one disappointed. Instead, his main concern is that a soft and optional introduction of the support in the build system would leave possible bugs hidden, and therefore should be automatically enabled if a Rust compiler is present — essentially implying that he seems otherwise on board.

We’ll see what comes of it, but with Rust language team lead [Josh Triplett] stating that enhancements benefiting and advancing a kernel integration are certainly imaginable for Rust itself, we might see interesting collaborations coming up in the near future. If you don’t want to wait for that and use Rust already today in a user-land driver, check out this 35c3 talk. And if that doesn’t go far enough for you, here’s a whole (non-Linux) kernel written in Rust.

EasyOCR Makes OCR, Well, Easy

July 9, 2020 by 21 Comments

Working on embedded systems used to be easier. You had a microcontroller and maybe a few pieces of analog or digital I/O, and perhaps communications might be a serial port. Today, you have systems with networks and cameras and a host of I/O. Cameras are strange because sometimes you just want an image and sometimes you want to understand the image in some way. If understanding the image involves reading text in the picture, you will want to check out EasyOCR.

The Python library leverages other open source libraries and supports 42 different languages. As the name implies, using it is pretty easy. Here’s the setup:


import easyocr
reader = easyocr.Reader(['th','en'])
reader.readtext('test.jpg')

The results include four points that define the bounding box of each piece of text, the text, and a confidence level. The code takes advantage of the GPU, but you can run it in a CPU-only mode if you prefer. There are a few other options, including setting the algorithm’s scanning behavior, how it handles multiple processors, and how it converts the image to grayscale. The results look impressive.

According to the project’s repository, they incorporated several existing neural network algorithms and conventional algorithms, so if you want to dig into details, there are links provided to both code and white papers. If you need some inspiration for what to do with OCR, maybe this past project will give you some ideas. Or you could cheat at games.