Linux Fu: Modernize Your Command Line

August 29, 2018 by Al Williams 19 Comments

If you use Linux and its associated tools on the desktop or on a Raspberry Pi, or on a server, you probably have used the command line. Some people love it and some people hate it. However, many of us have been using Linux for years and sometimes Unix before that, and we tend to use the same old tried-and-true tools. [Remy Sharp] had a recent post talking about how he had created aliases to replace those old tools with great modern replacements and it is definitely worth a read.

We’ll be honest, when we first saw the post we almost skipped reading it. A lot of Linux tip posts are pretty uninteresting unless you are a total beginner. But [Remy] has a lot of really great tools and how he has them installed including bat, which is like cat but with syntax coloring (see picture above), and fzf — a command line history search on steroids. He even shows how to join fzf and bat to make a very cool file browser from the command line (see below).

Continue reading “Linux Fu: Modernize Your Command Line” →

Linux Fu: Scripting For Binary Files

June 29, 2018 by Al Williams 31 Comments

If you ever need to write a binary file from a traditional language like C, it isn’t all that hard to do. About the worst thing you might have to deal with is attempts to fake line endings across Windows and Linux, but there’s usually a way to turn that off if it is on by default. However, if you are using some type of scripting language, binary file support might be a bit more difficult. One answer is to use a tool like xxd or t2b (text-to-binary) to handle the details. You can find the code for t2b on GitHub including prebuilt binaries for many platforms. You should be able to install xxd from your system repository.

These tools take very different approaches. You might be familiar with tools like od or hexdump for producing readable representations of binary files. The xxd tool can actually do the same thing — although it is not as flexible. What xxd can even reverse itself so that it can rebuild a binary file from a hex dump it creates (something other tools can’t do). The t2b tool takes a much different approach. You issue commands to it that causes it to write an original hex file.

Both of these approaches have some merit. If you are editing a binary file in a scripting language, xxd makes perfect sense. You can convert the file to text, process it, and then roll it back to binary using one program. On the other hand, if you are creating a binary file from scratch, the t2b program has some advantages, too.

I decided to write a few test scripts using bash to show how it all works. These aren’t production scripts so they won’t be as hardened as they could be, but there is no reason they couldn’t be made as robust as you were willing to make them.

Continue reading “Linux Fu: Scripting For Binary Files” →

Linux Fu: Watch That Filesystem

June 7, 2018 by Al Williams 31 Comments

The UNIX Way™ is to cobble together different, single-purpose programs to get the effect you want, for instance in a Bash script that you run by typing its name into the command line. But sometimes you want the system to react to changes in the system without your intervention. For example, you might like to watch a directory and kick off some program automatically when a file appears from a completed FTP transaction, without having to sit there and refresh the directory yourself.

The simple but ugly way to do this just scans the directory periodically. Here’s a really dumb shell script:

#!/bin/bash
while true
 do
   for I in `ls`
    do cat $I; rm $I
   done
 sleep 10
done

Just for an example, I dump the file to the console and remove it, but in real life, you’d do something more interesting. This is really not a good script because it executes all the time and it just isn’t a very elegant solution. (If you think I should use for I in *, try doing that in an empty directory and you’ll see why I use the ls command instead.)

Continue reading “Linux Fu: Watch That Filesystem” →

Linux Fu: Counter Rotate Keys!

June 1, 2018 by Al Williams 22 Comments

If you’ve done anything with a modern Linux system — including most variants for the Raspberry Pi — you probably know about sudo. This typically allows an authorized user to elevate themselves to superuser status to do things.

However, there is a problem. If you have sudo access, you can do anything — at least, anything the sudoers file allows you to do. But what about extremely critical operations? We’ve all seen the movies where launching the nuclear missile requires two keys counter-rotated at the same time and third firing key. Is there an equivalent for Linux systems?

It isn’t exactly a counter-rotating key, but the sudo_pair project — a prelease open-source project from Square — gives you something similar. The project is a plugin for sudo that allows you to have another user authorize a sudo request. Not only do they authorize it, but they get to see what is happening, and even abort it if something bad is happening.

Continue reading “Linux Fu: Counter Rotate Keys!” →

Linux Fu: File Aliases, Links, And Mappings

March 16, 2018 by Al Williams 37 Comments

Have you heard it said that everything in Linux is a file? That is largely true, and that’s why the ability to manipulate files is crucial to mastering Linux Fu.

One thing that makes a Linux filesystem so versatile is the ability for a file to be many places at once. It boils down to keeping the file in one place but using it in another. This is handy to keep disk access snappy, to modify a running system, or merely to keep things organized in a way that suits your needs.

There are several key features that lend to this versatility: links, bind mounts, and user space file systems immediately come to mind. Let’s take a look at how these work and how you’ll often see them used.

Continue reading “Linux Fu: File Aliases, Links, And Mappings” →

Linux Fu: Regular Expressions

March 9, 2018 by Al Williams 37 Comments

If you consider yourself a good cook, you may or may not know how to make a souffle or baklava. But there are certain things you probably do know how to do that form the basis of many recipes. For example, you can probably boil water, crack an egg, and brown meat. With Linux or Unix systems, you can make the same observation. You might not know how to set up a Wayland server or write a kernel module. But there are certain core skills like file manipulation and editing that will serve you no matter what you do. One of the pervasive skills that often gives people trouble is regular expressions. Many programs use these as a way to specify search patterns, usually in text strings such as files.

If you aren’t comfortable with regular expressions, that’s easy to fix. They aren’t that hard to learn and there are some great tools to help you. Many tools use regular expressions and the core syntax is the same. The source of confusion is that the details beyond core syntax have variations.

Let’s look at the foundation you need to understand regular expression well.

Continue reading “Linux Fu: Regular Expressions” →

Linux Fu: A Little Help For Bash

February 2, 2018 by Al Williams 23 Comments

It isn’t uncommon these days for a programmer’s editor to offer you help about what you are typing, ranging from a pop up with choices to a full-blown code template. If you have written a million lines of code in the language, this might even annoy you. However, if you use it only occasionally, these can be very helpful. I’ve used Unix and Linux for many years, but I realize that there are people who don’t use it every day. With the Raspberry Pi, Linux servers, and Windows 10 having a bash shell, there are more people using a shell “every once in a while” than ever before. Could you use a little help? If so, you might try bashelp: a little something I put together while writing about bash completion.

There’s good news and bad news. The good news is that Unix has a built-in help command — man — and has for some time. The bad news is that you need to stop what you are typing and enter a man command to use it. Man, by the way, is short for manual.

There are GUI front-ends to man (like yelp, on the left) and you can even use a web browser locally or remotely. However, none of these are connected to what you are typing. You have to move to another window, enter your search term, then go back to your typing. That got me to thinking about how to get a sort of context-sensitive inline help for bash.