Linux Fu: Monitor Disks

If you want a quick view of a Linux system’s process load, you can use top or — slightly nicer — htop. But what if you want a quick snapshot of how the disk system is doing? There are a few tools you can use, some of which are not nearly as common as top.

First, iotop

Most similar to top is iotop. This program shows you the total and current disk read and write numbers for the file system and also shows you who is eating up the most disk I/O.  This screen looks busy:

Continue reading “Linux Fu: Monitor Disks”

Ubuntu (Finally) Officially Lands On The Raspberry Pi. But Will Anyone Notice?

The Raspberry Pi has been with us for over eight years now, and during that time it has seen a myriad operating system ports. It seems that almost anything can be run on the little computer, but generally the offerings have seen minority uptake in the face of the officially supported Raspbian, or as it’s now called, Raspberry Pi OS.

Maybe that could change, with the arrival of an Ubuntu release for the platform. For those of you pointing out that this is nothing new, what makes the new version 20.10 release special is that it’s the first official full Ubuntu release, rather than an unofficial port.

So Raspberry Pi 4 owners can now install the same full-fat Ubuntu they have on their PCs, and with the same official Ubuntu support. What does this really do for them that Raspberry Pi OS doesn’t? Underneath they share Debian underpinnings, and they both benefit from a huge quantity of online resources should the user find themselves in trouble. Their repositories both contain almost every reasonable piece of software that could be imagined, so the average Pi user might be forgiven for a little confusion.

We don’t expect this news to take the Pi desktop world by storm then. Ubuntu is a powerful distribution, but it’s fair to say that it is not the least bloated among distributions, and that some of its quirks such as Snap applications leave many users underwhelmed. By contrast Raspberry Pi OS is relatively lightweight, and crucially it’s optimised for the Pi. Its entire support base online is specific to the Pi hardware, so the seeker of solutions need not worry about encountering some quirk in an explanation that pertains only to PC platforms.

It’s fair to say though, that this release is almost certainly not targeted at the casual desktop user. We’d expect that instead it will be in the Ubuntu portfolio for commercial and enterprise users, and in particular for the new Raspberry Pi 4 Compute Module in which it will no doubt form the underpinnings of many products without their owners ever realising it.

[via OMG Ubuntu]

Linux Fu: Troubleshooting Incron

You probably know about cron, a program that lets you schedule programs to run at various times. We’ve also talked about incron, which is very similar but instead of time, it reacts to changes in the file system. If you ever wanted to write a program that, say, detects a change in a file and automatically uploads it to a programmer, backs it up, e-mails it somewhere, or anything else, then incron might be for you. Although we’ve talked about it before, incron has some peculiarities that make it very difficult to debug problems, so I thought I’d share some of the tricks I use when working with incron.

I was thinking about this because I wanted to set up a simple system where I have a single document directory under git control. Changing a markdown file in that folder would generate Word document and PDF equivalents. Conversely, changing a Word document would produce a markdown version.

This is easy to do with pandoc — it speaks many different formats. The trick is running it only on changed files and as soon as they change. The task isn’t that hard, but it does take a bit to debug since it’s a bit nontrivial.

Continue reading “Linux Fu: Troubleshooting Incron”

Linux Fu: Global Search And Replace With Ripgrep

If you are even a casual Linux user, you probably know how to use grep. Even if you aren’t a regular expression guru, it is easy to use grep to search for lines in a file that match anything from simple strings to complex patterns. Of course, grep is fine for looking, but what if you want to find things and change them. Maybe you want to change each instance of “HackADay” to “Hackaday,” for example. You might use sed, but it is somewhat hard to use. You could use awk, but as a general-purpose language, it seems a bit of overkill for such a simple and common task. That’s the idea behind ripgrep which actually has the command name rg. Using rg, you can do things that grep can do using more modern regular expressions and also do replacements.

A Note on Installing Ripgrep

Your best bet is to get ripgrep from your repositories. When I tried running KDE Neon, it helpfully told me that I could install a version using apt or take a Snap version that was newer. I usually hate installing a snap, but I did anyway. It informed me that I had to add –classic to the install line because ripgrep could affect files outside the Snap sandbox. Since the whole purpose of the program is to change files, I didn’t think that was too surprising, so I did the install.

Continue reading “Linux Fu: Global Search And Replace With Ripgrep”

Linux Fu: The Linux Android Convergence

The Android phone that you carry in your pocket is basically a small computer running Linux. So why is it so hard to get to a usable Linux environment on your phone? If you could run Linux, you could turn your cell phone into an ultra-portable laptop replacement.

Of course, the obvious approach is just to root the phone and clean-slate install a Linux distribution on it. That’s pretty extreme and, honestly, you would probably lose a lot of phone function unless you go with a Linux-specific phone like the PinePhone. However, using an installer called AnLinux, along with a terminal program and a VNC client, you can get a workable setup without nuking your phone’s OS, or even having root access. Let’s see what we can do. Continue reading “Linux Fu: The Linux Android Convergence”

Should You Build For Windows, Mac, IOS, Android, Or Linux? Yes!

The holy grail of computer languages is to write code once and have it deploy effortlessly everywhere. Java likes to take credit for the idea, but UCSD P-Code was way before that and you could argue that mainframes had I/O abstraction like Fortran unit numbers even earlier. More modern efforts include Qt, GTK, and other things. Naturally, all of these fall short in some way. Now Google enters the fray with Flutter.

Flutter isn’t new, but in the past, it only handled Android and iOS. Now it can target desktop platforms and can even produce JavaScript. We haven’t played with the system enough to say how successful it is, but you can try it in your browser if you want some first-hand experience.

Continue reading “Should You Build For Windows, Mac, IOS, Android, Or Linux? Yes!”

Linux-Fu: Making AWK A Bit Easier

awk is a kind of Swiss Army knife for text files. However, some of its limitations are often a bit annoying. I’ve used a simple set of functions to make awk a bit better, although I will warn you: it does require GNU extensions to awk. That is, you must use gawk and not other versions. Your system probably maps /usr/bin/awk to something and that something might be gawk. But it could also be mawk or some other flavor. If you use a Debian-based distro, update-alternatives is your friend here. But for the purposes of this post, I’m going to assume you are using gawk.

By the end of the post, you’ll see how to use my awk add-on functions to split up a line into fields even when there is no single character to separate all fields. In addition, you’ll be able to refer to the fields using names you decide. You won’t have to remember that $2 is the time field. You’ll say Fields_fields["time"] instead.

The Problem

awk does a lot of common work for you when you use it to process text files. It reads files a record at a time. Normally, a record is a single line. Then it splits the line on fields using whitespace, or some other choice of field separators. You can write code that manipulates the line or individual fields. This default behavior is great, especially since you can change the end of record character and the field separator. A surprising number of files fit this sort of format.

Until, of course, they don’t. If you have data coming from a data logging instrument or some database, it could be formatted in a variety of ways. Some fields might have structured data with a variety of separators. This isn’t a deal-breaker. Since you can get at the whole line, you can do almost anything you want, but the logic is harder and the whole point to using awk is to make things easier.

For example, suppose you had a file from a data recorder that had an eight-digit serial number, followed by a six-character tag, and then two floating point numbers separated by colons. The pattern might look like

^([0-9]{8})([a-zA-Z0-9]{6})([-+.0-9]+),([-+.0-9]+)$

This would be hard to handle with the conventional field splitting and you’d normally just write code to split everything apart.

Continue reading “Linux-Fu: Making AWK A Bit Easier”