Linux Fu: Controlling the Terminal

A Linux terminal has a lot more features than the TeleType of yore. On a TeleType, text spews out and scrolls up and is gone forever. A real terminal can use escape characters to do navigate around and emulate most of what you like about GUIs. However, doing this at the lowest level is a chore and limits portability. Luckily, all the hard work has already been done.

First, there’s a large database of terminal capabilities available for you to use: terminfo.  And in addition, there’s a high-level library called curses or ncurses that simplifies writing programs to control the terminal display. Digging deep into every nook and cranny of ncurses could take years. Instead, I’m going to talk about using a program that comes with ncurses to control the terminal, called tput. Using these two commands, you can figure out what kind of terminal you’re dealing with, and then manipulate it nearly to your heart’s content. Let’s get started!

Continue reading “Linux Fu: Controlling the Terminal”

Linux Fu: Pimp Your Pipes

One of the best things about working at the Linux (or similar OS) command line is the use of pipes. In simple terms, a pipe takes the output of one command and sends it to the input of another command. You can do a lot with a pipe, but sometimes it is hard to work out the right order for a set of pipes. A common trick is to attack it incrementally. That is, do one command and get it working with the right options and inputs. Then add another command until that works. Keep adding commands and tweaking until you get the final results.

That’s fine, but [akavel] wanted better and used Go to create “up” — an interactive viewer for pipelines.

Pipe Philosophy

Pipes can do a lot. They match in with the original Unix philosophy of making each tool do one thing really well. Pipe is really good at allowing Linux commands to talk to each other. If you want to learn all about pipes, have a look at the Linux Info project’s guide. They even talk about why MSDOS pipes were not really pipes at all. (One thing that write up doesn’t touch on is the named pipe. Do a “man fifo” if you want to learn more for now and perhaps that will be the subject of a future Linux Fu.)

This program — called up — continuously runs and reruns your pipeline as you make changes to the pipe. This way, every change you make is instantly reflected in the output. Here’s the video, here’s a quick video which shows off the interactive nature of up.

Installing

The GitHub page assumes you know how to install a go program. I tried doing a build but I didn’t have a few dependencies. Turns out the easy way to do it was to run this line:

go get -u github.com/akavel/up

This put the executable in ~/go/bin — which isn’t on my path. You can, of course, copy or link it to some directory that’s on your path or add that directory to your path. You could also set an alias, for example. Or, like I did in the video, just specify it every time.

Perfect?

This seems like a neat simple tool. What could be better? Well, I was a little sad that you can’t use emacs or vi edit keys on the pipeline, at least not as far as I could tell. This is exactly the kind of thing where you want to back up into the middle and change something. You can use the arrow keys, though, so that’s something. I also wished the scrollable window had a search feature like less.

Otherwise, though, there’s not much to dislike about the little tool. If writing a pipeline is like using a C compiler, up makes it more like writing an interactive Basic program.

Linux Fu: Modernize Your Command Line

If you use Linux and its associated tools on the desktop or on a Raspberry Pi, or on a server, you probably have used the command line. Some people love it and some people hate it. However, many of us have been using Linux for years and sometimes Unix before that, and we tend to use the same old tried-and-true tools. [Remy Sharp] had a recent post talking about how he had created aliases to replace those old tools with great modern replacements and it is definitely worth a read.

We’ll be honest, when we first saw the post we almost skipped reading it. A lot of Linux tip posts are pretty uninteresting unless you are a total beginner. But [Remy] has a lot of really great tools and how he has them installed including bat, which is like cat but with syntax coloring (see picture above), and fzf — a command line history search on steroids. He even shows how to join fzf and bat to make a very cool file browser from the command line (see below).

Continue reading “Linux Fu: Modernize Your Command Line”

Linux Fu: Scripting for Binary Files

If you ever need to write a binary file from a traditional language like C, it isn’t all that hard to do. About the worst thing you might have to deal with is attempts to fake line endings across Windows and Linux, but there’s usually a way to turn that off if it is on by default. However, if you are using some type of scripting language, binary file support might be a bit more difficult. One answer is to use a tool like xxd or t2b (text-to-binary) to handle the details. You can find the code for t2b on GitHub including prebuilt binaries for many platforms. You should be able to install xxd from your system repository.

These tools take very different approaches. You might be familiar with tools like od or hexdump for producing readable representations of binary files. The xxd tool can actually do the same thing — although it is not as flexible. What xxd can even reverse itself so that it can rebuild a binary file from a hex dump it creates (something other tools can’t do). The t2b tool takes a much different approach. You issue commands to it that causes it to write an original hex file.

Both of these approaches have some merit. If you are editing a binary file in a scripting language, xxd makes perfect sense. You can convert the file to text, process it, and then roll it back to binary using one program. On the other hand, if you are creating a binary file from scratch, the t2b program has some advantages, too.

I decided to write a few test scripts using bash to show how it all works. These aren’t production scripts so they won’t be as hardened as they could be, but there is no reason they couldn’t be made as robust as you were willing to make them.

Continue reading “Linux Fu: Scripting for Binary Files”

Linux Fu: Watch That Filesystem

The UNIX Way™ is to cobble together different, single-purpose programs to get the effect you want, for instance in a Bash script that you run by typing its name into the command line. But sometimes you want the system to react to changes in the system without your intervention. For example, you might like to watch a directory and kick off some program automatically when a file appears from a completed FTP transaction, without having to sit there and refresh the directory yourself.

The simple but ugly way to do this just scans the directory periodically. Here’s a really dumb shell script:

#!/bin/bash
while true
 do
   for I in `ls`
    do cat $I; rm $I
   done
 sleep 10
done

Just for an example, I dump the file to the console and remove it, but in real life, you’d do something more interesting. This is really not a good script because it executes all the time and it just isn’t a very elegant solution. (If you think I should use for I in *, try doing that in an empty directory and you’ll see why I use the ls command instead.)

Continue reading “Linux Fu: Watch That Filesystem”

Linux Fu: Counter Rotate Keys!

If you’ve done anything with a modern Linux system — including most variants for the Raspberry Pi — you probably know about sudo. This typically allows an authorized user to elevate themselves to superuser status to do things.

However, there is a problem. If you have sudo access, you can do anything — at least, anything the sudoers file allows you to do. But what about extremely critical operations? We’ve all seen the movies where launching the nuclear missile requires two keys counter-rotated at the same time and third firing key. Is there an equivalent for Linux systems?

It isn’t exactly a counter-rotating key, but the sudo_pair project — a prelease open-source project from Square — gives you something similar. The project is a plugin for sudo that allows you to have another user authorize a sudo request. Not only do they authorize it, but they get to see what is happening, and even abort it if something bad is happening.

Continue reading “Linux Fu: Counter Rotate Keys!”

Linux Fu: File Aliases, Links, and Mappings

Have you heard it said that everything in Linux is a file? That is largely true, and that’s why the ability to manipulate files is crucial to mastering Linux Fu.

One thing that makes a Linux filesystem so versatile is the ability for a file to be many places at once. It boils down to keeping the file in one place but using it in another. This is handy to keep disk access snappy, to modify a running system, or merely to keep things organized in a way that suits your needs.

There are several key features that lend to this versatility: links, bind mounts, and user space file systems immediately come to mind. Let’s take a look at how these work and how you’ll often see them used.

Continue reading “Linux Fu: File Aliases, Links, and Mappings”