Software Development

757 Articles

The X Macro: A Historic Preprocessor Hack

March 16, 2023 by 45 Comments

If we told you that a C preprocessor hack dated back to 1968, you’d be within your rights to remind us that C didn’t exist in 1968. However, assemblers with preprocessors did, and where there is a preprocessor, there is an opportunity to do clever things. One of those things is the so-called X macro, which saw a lot of use in DEC System 10 code but probably dates back even earlier. You can still use it today if you like, even though there are, of course, other arguably better ways to get the same result. However, the X macro can be very efficient, and you may well run into it in some code, too.

Background

Preprocessing used to be a staple of programming. The idea is that code is manipulated purely at the text level before it is compiled. These days, languages with a preprocessor usually handle it as part of the compiler, but you can also use an external preprocessor like m4 for more sophisticated uses.

Modern languages tend to provide other ways to accomplish many of the tasks handled by the preprocessor. For example, if you have a constant you want to set at compile time, you could say:

int X = 32;
y = X;

But then you’ve created a real variable along with the overhead that might entail. A smart compiler might optimize it away for you, but you can be sure by writing:

#define X 32
y = X;

A modern compiler would prefer you to write:

const int X=32;
y = X;

But there are still some common uses for macros, like including header files. You can also make more sophisticated macros with arguments so you don’t incur a function call penalty, although modern usage would be to mark those functions as inline.

The Problem

Which brings us to the X macro. With all great hacks, there is first a problem to solve. Imagine you have a bunch of electronic parts you want to deal with in your code. You don’t want a database, and you don’t want to carry a bunch of strings around, so you define an enumerated type:

Continue reading “The X Macro: A Historic Preprocessor Hack” →

Efficient X86_64 Emulation With Box86

March 12, 2023 by 5 Comments

Running applications on a different architecture than the one for which they were compiled is a common occurrence, not in the least with Apple’s architectural migration every decade or so. It’s also commonly used with for example ARM, OpenRISC, and RISC-V systems to run applications that are only available for x86 or x86_64. While QEMU and kin are often used here, they’re pretty resource heavy, which is where an option like Box86 and its 64-bit sibling Box64 are attractive options. Unlike QEMU, both offer dynamic recompilation and redirection of dynamic library calls to native libraries, including those for SDL and OpenGL.

Both are available on GitHub under an MIT license, with Box64 probably the most interesting these days as applications and games have moved on to a 64-bit only world. The only hard requirement that Box64 has for a host system is that it is little-endian, which is a pretty easy requirement to meet. The most recent release was on March 10th, with Box86 0.3 and Box64 0.2.2. As essentially a translation layer, it does not offer full compatibility with every bit of software out there, but it’s already good enough to run Steam, GoG, and Epic Game Store clients and install and run Windows games via Wine for x86.

A simple set of benchmarks comparing it with QEMU and FEX (another emulator) shows it to run both more applications, and with significantly better performance.

Videos Teach Bare Metal RP2040

March 9, 2023 by 31 Comments

When we write about retrocomputers, we realize that back in the day, people knew all the details of their computer. You had to, really, if you wanted to get anything done. These days, we more often pick peripherals and just assume our C or other high level code will fit and run on the CPU.

But sometimes you need to get down to the bare metal and if your desire is to use bare metal on the RP2040, [Will Thomas] has a YouTube channel to help you. The first video explains why you might want to do this followed by some simple examples. Then you’ll find over a dozen other videos that give you details.

Any video that starts, “Alright, Monday night. I have no friends. It is officially bare metal hours,” deserves your viewing. Of course, you have to start with the traditional blinking LED. But subsequent videos talk about the second core, GPIO, clocks, SRAM, spinlocks, the UART, and plenty more.

As you might expect, the code is all in assembly. But even if you want to program using C without the SDK, the examples will be invaluable. We like assembly — it is like working an intricate puzzle and getting anything to work is satisfying. We get it. But commercially, it rarely makes sense to use assembly anymore. On the other hand, when you need it, you really need it. Besides, we all do things for fun that don’t make sense commercially.

We like assembly, especially on platforms where most people don’t use it. Tackling it on a modern CPU is daunting, but if you want to have a go, we know someone who can help.

Continue reading “Videos Teach Bare Metal RP2040” →

FOSDEM 2023: An Open-Source Conference, Literally

March 1, 2023 by 6 Comments

Every year, on the first weekend of February, a certain Brussels university campus livens up. There, you will find enthusiasts of open-source software and hardware alike, arriving from different corners of the world to meet up, talk, and listen. The reason they all meet there is the conference called FOSDEM, a long-standing open-source software conference which has been happening in Belgium since 2000. I’d like to tell you about FOSDEM because, when it comes to conferences, FOSDEM is one of a kind.

FOSDEM is organized in alignment with open-source principles, which is to say, it reminds me of an open-source project itself. The conference is volunteer-driven, with a core of staff responsible for crucial tasks – yet, everyone can and is encouraged to contribute. Just like a large open-source effort, it’s supported by university and company contributions, but there’s no admission fees for participants – for a conference, this means you don’t have to buy a ticket to attend. Last but definitely not least, what makes FOSDEM shine is the community that it creates.

FOSDEM’s focus is open software – yet, for hackers of the hardware world, you will find a strong hardware component to participate in, since a great number of FOSDEM visitors are either interested in hardware, or even develop hardware-related things day-to-day. It’s not just that our hardware can’t live without software, and vice-versa – here, you will meet plenty of pure software, a decent amount of pure hardware, and a lot of places where the two worlds are hard to distinguish. All in all, FOSDEM is no doubt part of hacker culture in Europe, and today, I will tell you about my experience of FOSDEM 2023. Continue reading “FOSDEM 2023: An Open-Source Conference, Literally” →

Tiny Machine Learning On As Little As 2 KB Of RAM

February 24, 2023 by 6 Comments

All of the machine language stuff coming out lately doesn’t affect you if you are developing with embedded microcontrollers, right? Perhaps not. Microsoft Research India wants you to use their EdgeML tool to do machine learning tasks such as gesture recognition in tiny devices like an Arduino Uno. According to the developers, you might need as little as 2 KB of RAM. There’s no network connection required and the work is using Tensorflow underneath, so it is compatible with much of what you’ll find for bigger computers.

If you add processing power, you can get more capability. For example, one of the demonstrations is a wake-word recognizer on a Raspberry Pi Zero (although the page for that demo seems to be missing at the moment; try the GesturePod, instead).

The system generally uses Python, but there are efficient C++ implementations for selected algorithms. The code lives on GitHub. There are also a number of research papers about each tool that you can find on the GitHub page. There’s also a recent paper on MinUn, an attempt to make things even more efficient for ARM microcontrollers. In particular, MinUn can store approximate numbers to save space, allows for variable precision of tensors, and tries to reduce memory fragmentation, an important feature for CPUs that don’t have memory management units.

If you haven’t studied TensorFlow yet, start here. Why use something like this with a microcontroller? How about smarter robots?

ESP32 Web Updater Allows File System Management And OTA Updates

February 17, 2023 by 10 Comments

Earlier versions of the Arduino IDE made uploading files to an ESP32’s SPIFFS filesystem easy via the ESP32FS plugin. Sadly, that’s no longer possible under the rewritten Arduino 2.0 IDE. Thankfully, [myhomethings] has stepped up to solve the problem with a new tool that also adds some new functionality.

The tool in question is the ESP32 Web Updater and SPIFFS File Manager. It features a web interface courtesy of the ESPAsyncWebServer library. Simply dialing into the ESP32’s IP address will grant one access to the interface. Once connected files can be uploaded to the ESP32, or deleted at will. Text files can be created and populated through the interface as well, and the SPIFFS file system can also be formatted if required. Plus, as a bonus, the interface allows for handy over-the-air firmware updates. One need only export a compiled binary from the Arduino IDE, and then load the resulting *.bin file into the ESP32 via the web interface. It does come with the caveat that if new firmware is uploaded that doesn’t include the ESP32 Web Updater itself, there will be no way to do further firmware updates in this manner.

For those working on projects that may need regular file system management, the tool may be very useful. Alternatively, if you just need to do OTA updates on an ESP32, we recently featured a way of doing them through GitHub.

Modernizing C Arrays For Greater Memory Safety

February 10, 2023 by 75 Comments

Lately, there has been a push for people to stop using programming languages that don’t promote memory safety. But as we still haven’t seen the death of some languages that were born in the early 1960s, we don’t think there will be much success in replacing the tremendous amount of software that uses said “unsafe” languages.

That doesn’t mean it’s a hopeless cause, though. [Kees Cook] recently posted how modern C99 compilers offer features to help create safer arrays, and he outlines how you can take advantage of these features. Turns out, it is generally easy to do, and if you get errors, they probably point out unexpected behavior in your original code, so that’s a plus.

We don’t think there’s anything wrong with C and C++ if you use them as you should. Electrical outlets are useful until you stick a fork in one. So don’t stick a fork in one. We really liked the recent headline we saw from [Sarah Butcher]: “If you can’t write safe C++ code, it’s because you can’t write C++.” [Cook’s] post makes a similar argument.  C has advanced quite a bit and the fact that 30-year-old code doesn’t use these new features isn’t a good excuse to give up on C.

Continue reading “Modernizing C Arrays For Greater Memory Safety” →