Linux Fu: Keep An Eye On That File

October 4, 2018 by Al Williams 38 Comments

One of the things that’s nice about Linux or Unix compared to many other operating systems is there’s a good chance a Linux program will spew out informational messages to a log somewhere. Many commands even have a way to turn on more logs. I know that Windows has the event viewer, but many programs don’t have much to say which makes it difficult to know what’s happening when things go wrong.

The problem is, sometimes programs tell you too much information. How do you find what you want to know? It looks cool on a movie where the hacker is in front of a terminal scrolling 500 lines a second of some log file, but in real life, it is hard to read a moving screen, although with some practice you can sometimes — unreliably — pick out a keyword as it whizzes by.

Like most Unix things, there’s a tool for that. In fact, unsurprisingly, there are many tools for that. If you are using the tail command, that’s certainly one of them. But there are others you should consider.

Continue reading “Linux Fu: Keep An Eye On That File” →

Linux Fu: Modernize Your Command Line

August 29, 2018 by Al Williams 19 Comments

If you use Linux and its associated tools on the desktop or on a Raspberry Pi, or on a server, you probably have used the command line. Some people love it and some people hate it. However, many of us have been using Linux for years and sometimes Unix before that, and we tend to use the same old tried-and-true tools. [Remy Sharp] had a recent post talking about how he had created aliases to replace those old tools with great modern replacements and it is definitely worth a read.

We’ll be honest, when we first saw the post we almost skipped reading it. A lot of Linux tip posts are pretty uninteresting unless you are a total beginner. But [Remy] has a lot of really great tools and how he has them installed including bat, which is like cat but with syntax coloring (see picture above), and fzf — a command line history search on steroids. He even shows how to join fzf and bat to make a very cool file browser from the command line (see below).

Continue reading “Linux Fu: Modernize Your Command Line” →

Linux Fu: Scripting For Binary Files

June 29, 2018 by Al Williams 31 Comments

If you ever need to write a binary file from a traditional language like C, it isn’t all that hard to do. About the worst thing you might have to deal with is attempts to fake line endings across Windows and Linux, but there’s usually a way to turn that off if it is on by default. However, if you are using some type of scripting language, binary file support might be a bit more difficult. One answer is to use a tool like xxd or t2b (text-to-binary) to handle the details. You can find the code for t2b on GitHub including prebuilt binaries for many platforms. You should be able to install xxd from your system repository.

These tools take very different approaches. You might be familiar with tools like od or hexdump for producing readable representations of binary files. The xxd tool can actually do the same thing — although it is not as flexible. What xxd can even reverse itself so that it can rebuild a binary file from a hex dump it creates (something other tools can’t do). The t2b tool takes a much different approach. You issue commands to it that causes it to write an original hex file.

Both of these approaches have some merit. If you are editing a binary file in a scripting language, xxd makes perfect sense. You can convert the file to text, process it, and then roll it back to binary using one program. On the other hand, if you are creating a binary file from scratch, the t2b program has some advantages, too.

I decided to write a few test scripts using bash to show how it all works. These aren’t production scripts so they won’t be as hardened as they could be, but there is no reason they couldn’t be made as robust as you were willing to make them.

Continue reading “Linux Fu: Scripting For Binary Files” →

Linux Fu: The Great Power Of Make

June 22, 2018 by Al Williams 76 Comments

Over the years, Linux (well, the operating system that is commonly known as Linux which is the Linux kernel and the GNU tools) has become much more complicated than its Unix roots. That’s inevitable, of course. However, it means old-timers get to slowly grow into new features while new people have to learn all in one gulp. A good example of this is how software is typically built on a Linux system. Fundamentally, most projects use make — a program that tries to be smart about running compiles. This was especially important when your 100 MHz CPU connected to a very slow disk drive would take a day to build a significant piece of software. On the face of it, make is pretty simple. But today, looking at a typical makefile will give you a headache, and many projects use an abstraction over make that further obscures things.

In this article, I want to show you how simple a makefile can be. If you can build a simple makefile, you’ll find they have more uses than you might think. I’m going to focus on C, but only because that’s sort of the least common denominator. A make file can build just about anything from a shell prompt.

Continue reading “Linux Fu: The Great Power Of Make” →

Linux Fu: Watch That Filesystem

June 7, 2018 by Al Williams 31 Comments

The UNIX Way™ is to cobble together different, single-purpose programs to get the effect you want, for instance in a Bash script that you run by typing its name into the command line. But sometimes you want the system to react to changes in the system without your intervention. For example, you might like to watch a directory and kick off some program automatically when a file appears from a completed FTP transaction, without having to sit there and refresh the directory yourself.

The simple but ugly way to do this just scans the directory periodically. Here’s a really dumb shell script:

#!/bin/bash
while true
 do
   for I in `ls`
    do cat $I; rm $I
   done
 sleep 10
done

Just for an example, I dump the file to the console and remove it, but in real life, you’d do something more interesting. This is really not a good script because it executes all the time and it just isn’t a very elegant solution. (If you think I should use for I in *, try doing that in an empty directory and you’ll see why I use the ls command instead.)

Continue reading “Linux Fu: Watch That Filesystem” →

Linux Fu: Counter Rotate Keys!

June 1, 2018 by Al Williams 22 Comments

If you’ve done anything with a modern Linux system — including most variants for the Raspberry Pi — you probably know about sudo. This typically allows an authorized user to elevate themselves to superuser status to do things.

However, there is a problem. If you have sudo access, you can do anything — at least, anything the sudoers file allows you to do. But what about extremely critical operations? We’ve all seen the movies where launching the nuclear missile requires two keys counter-rotated at the same time and third firing key. Is there an equivalent for Linux systems?

It isn’t exactly a counter-rotating key, but the sudo_pair project — a prelease open-source project from Square — gives you something similar. The project is a plugin for sudo that allows you to have another user authorize a sudo request. Not only do they authorize it, but they get to see what is happening, and even abort it if something bad is happening.

Continue reading “Linux Fu: Counter Rotate Keys!” →

Linux Fu: File Aliases, Links, And Mappings

March 16, 2018 by Al Williams 37 Comments

Have you heard it said that everything in Linux is a file? That is largely true, and that’s why the ability to manipulate files is crucial to mastering Linux Fu.

One thing that makes a Linux filesystem so versatile is the ability for a file to be many places at once. It boils down to keeping the file in one place but using it in another. This is handy to keep disk access snappy, to modify a running system, or merely to keep things organized in a way that suits your needs.

There are several key features that lend to this versatility: links, bind mounts, and user space file systems immediately come to mind. Let’s take a look at how these work and how you’ll often see them used.

Continue reading “Linux Fu: File Aliases, Links, And Mappings” →