Archives

Linux Fu: Preprocessing Beyond Code

December 28, 2023 by 15 Comments

If you glanced at the title and thought, “I don’t care — I don’t write C code,” then hang on a minute. While it is true that C has a preprocessor and you can notoriously do strange and — depending on your point of view — horrible or wonderful things with it, there are actually other options and you don’t have to use any of them with a C program. You can actually use the C preprocessor with almost any kind of text file. And it’s not the only preprocessor you can abuse this way. For example, the m4 preprocessor is wildly complex, vastly underused, and can handle C source code or anything else you care to send to it.

Definitions

I’ll define a preprocessor as a program that transforms its input file into an output file, reacting to commands that are probably embedded in the file itself. Most often, that output is then sent to some other program to do the “real” work. That covers cpp, the C preprocessor. It also covers things like sed. Honestly, you can easily create custom preprocessors using C, awk, Python, Perl, or any other programming language. There are many other standard programs that you could think of as preprocessors, for example, tr. However, one of the most powerful is made to preprocess complex input files called m4. For some reason — maybe because of its complexity — you don’t see much m4 in the wild.

Continue reading “Linux Fu: Preprocessing Beyond Code” →

Linux Fu: Easy Kernel Debugging

November 22, 2023 by 2 Comments

It used to be that building the Linux kernel was not easy. Testing and debugging were even worse. Nowadays, it is reasonably easy to build a custom kernel and test or debug it using virtualization. But if you still find it daunting, try [deepseagirl’s] script to download, configure, build, and debug the kernel.

The Python program takes command line arguments so you can select a kernel version and different operations. The script can download the source, patch the configuration, build the kernel, and then package it into a Debian package you can boot under qemu. From there, you can test and even debug with gdb. No risk of hosing your everyday system and no need to understand how to configure everything to run.

Continue reading “Linux Fu: Easy Kernel Debugging” →

Linux Fu: Customizing Printf

October 24, 2023 by 13 Comments

When it comes to programming in C and, sometimes, C++, the printf function is a jack-of-all-trades. It does a nice job of quickly writing output, but it can also do surprisingly intricate formatting. For debugging, it is a quick way to dump some data. But what if you have data that printf can’t format? Sure, you can just write a function to pick things apart into things printf knows about. But if you are using the GNU C library, you can also extend printf to use custom specifications. It isn’t that hard, and it makes using custom data types easier.

An Example

Suppose you are writing a program that studies coin flips. Even numbers are considered tails, and odd numbers are heads. Of course, you could just print out the number or even mask off the least significant bit and print that. But what fun is that?

Here’s a very simple example of using our new printf specifier “%H”:

printf("%H %H %H %H\n",1,2,3,4);
printf("%1H %1H\n",0,1);

When you have a width specification of 1 (like you do in the second line) the output will be H or T. If you have anything else, the output will be HEADS or TAILS.

Continue reading “Linux Fu: Customizing Printf” →

Linux Fu: Deep Git Rebasing

October 17, 2023 by 22 Comments

If you spend much time helping people with word processor programs, you’ll find that many people don’t really use much of the product. They type, change fonts, save, and print. But cross-references? Indexing? Largely, those parts of the program go unused. I’ve noticed the same thing with Git. We all use it constantly. But do we? You clone a repo. Work on it. Maybe switch branches and create a pull request. That’s about 80% of what you want to do under normal circumstances. But what if you want to do something out of the ordinary? Git is very flexible, but you do have to know the magic incantations.

For example, suppose you mess up a commit message — we never do that, of course, but just pretend. Or you accidentally added a file you didn’t want in the commit. Git has some very useful ways to deal with situations like this, especially the interactive rebase.

Continue reading “Linux Fu: Deep Git Rebasing” →

Linux Fu: Build A Better Ls

September 8, 2023 by 27 Comments

Ask someone to name all the things they can find in a room. Only a few will mention air. Ask a Linux command line user about programs they use and they may well forget to mention ls. Like air, it is seemingly invisible since it is so everpresent. But is it the best it can be? Sure, you can use environment variables and aliases to make it work a little nicer, but, in fact, it is much the same ls we have used for decades. But there have always been moves to make better ls programs. One of them, exa, was recently deprecated in favor of one of its forks, eza.

One thing we liked about eza is that it is a single file. No strange installation. No multiple files to coordinate. Put it on your path, and you are done. So installation is easy, but why should you install it?

Continue reading “Linux Fu: Build A Better Ls” →

In Praise Of RPN (with Python Or C)

June 21, 2023 by 39 Comments

HP calculators, slide rules, and Forth all have something in common: reverse polish notation or RPN. Admittedly, slide rules don’t really have RPN, but you work problems on them the same way you do with an RPN calculator. For whatever reason, RPN didn’t really succeed in the general marketplace, and you might wonder why it was ever a thing. The biggest reason is that RPN is very easy to implement compared to working through proper algebraic, or infix, notation. In addition, in the early years of computers and calculators, you didn’t have much to work with, and people were used to using slide rules, so having something that didn’t take a lot of code that matched how users worked anyway was a win-win.

What is RPN?

If you haven’t encountered RPN before, it is an easy way to express math without ambiguity. For example, what’s 5 + 3 * 6?  It’s 23 and not 48. By order of operations you know that you have to multiply before you add, even if you wrote down the multiplication second. You have to read through the whole equation before you can get started with math, and if you want to force the other result, you’ll need parentheses.

With RPN, there is no ambiguity depending on secret rules or parentheses, nor is there any reason to remember things unnecessarily. For instance, to calculate our example you have to read all the way through once to figure out that you have to multiply first, then you need to remember that is pending and add the 5. With RPN, you go left to right, and every time you see an operator, you act on it and move on. With RPN, you would write 3 6 * 5 +.

While HP calculators were the most common place to encounter RPN, it wasn’t the only place. Friden calculators had it, too. Some early computers and calculators supported it but didn’t name it. Some Soviet-era calculators used it, too, including the famous Elektronika B3-34, which was featured in a science fiction story in a Soviet magazine aimed at young people in 1985. The story set problems that had to be worked on the calculator.

Continue reading “In Praise Of RPN (with Python Or C)” →

Linux Fu: Easy And Easier Virtual Networking

June 14, 2023 by 15 Comments

One of the best things about Linux is that there are always multiple ways to do anything you want to do. However, some ways are easier than others. Take, for example, virtual networking. There are plenty of ways to make a bunch of Internet-connected computers appear to be on a single private network. That’s nothing new, of course. Linux and Unix have robust networking stacks. Since 2018, though, Wireguard has been the go-to solution; it has a modern architecture, secure cryptography, and good performance.

There’s only one problem: it is relatively difficult to set up. Not impossible, of course. But it is a bit difficult, depending on what you want to accomplish.

How Difficult?

You must set up a wireguard server and one or more clients. You’ll need to pick a range of IP addresses. You might need to turn on routing. You have to generate keys. You might need to configure DNS and other routing options. You’ll certainly need to modify firewall rules. You’ll also need to distribute keys.

None of these steps are terribly difficult, but it is a lot to keep straight. The wg program and wg-quick script do most of the work, but you have a lot of decisions and configuration management to keep straight.

Browse the official “quick start,” and you’ll see that it isn’t all that quick. The wg-quick script is better but only handles some use cases. If you want really limited use cases, there are third-party tools to do a lot of the rote work, but if you need to change anything, you’ll still need to figure it all out.

That being said, once you have it set up, it pretty much works without issue and works well. But that initial setup can be very frustrating. Continue reading “Linux Fu: Easy And Easier Virtual Networking” →