bash

58 Articles

Linux Fu: Mixing Bash And Python

May 4, 2021 by 24 Comments

Although bash scripts are regularly maligned, they do have a certain simplicity and ease of creation that makes them hard to resist. But sometimes you really need to do some heavy lifting in another language. I’ll talk about Python, but actually, you can use many different languages with this technique, although you might need a little adaptation, depending on your language of choice.

Of course, you don’t have to do anything special to call another program from a bash script. After all, that’s what it’s mainly used for: calling other programs. However, it isn’t very handy to have your script spread out over multiple files. They can get out of sync and if you want to send it to someone or another machine, you have to remember what to get. It is nicer to have everything in one file.

Continue reading “Linux Fu: Mixing Bash And Python” →

AI Makes Linux Do What You Mean, Not What You Say

April 20, 2021 by 30 Comments

We are always envious of the Star Trek Enterprise computers. You can just sort of ask them a hazy question and they will — usually — figure out what you want. Even the automatic doors seemed to know the difference between someone walking into a turbolift versus someone being thrown into the door during a fight. [River] decided to try his new API keys for the private beta of an AI service to generate Linux commands based on a description. How does it work? Watch the video below and find out.

Some examples work fairly well. In response to “email the Rickroll video to Jeff Bezos,” the system produced a curl command and an e-mail to what we assume is the right place. “Find all files in the current directory bigger than 1 GB” works, too.

Continue reading “AI Makes Linux Do What You Mean, Not What You Say” →

Linux Fu: Shell Script File Embedding

April 9, 2021 by 25 Comments

You need to package up a bunch of files, send them somewhere, and do something with them at the destination. It isn’t an uncommon scenario. The obvious answer is to create an archive — a zip or tar file, maybe — and include a shell script that you have to tell the user to run after unpacking.

That may be obvious, but it assumes a lot on the part of the remote user. They need to know how to unpack the file and they also need to know to run your magic script of commands after the unpack. However, you can easily create a shell script that contains a file — even an archive of many files — and then retrieve the file and act on it at run time. This is much simpler from the remote user’s point of view. You get one file, you execute it, and you are done.

In theory, this isn’t that hard to do, but there are a lot of details. Shell scripts are not compiled — at least, not typically — so the shell only reads what it needs to do the work. That means if your script is careful to exit, you can add as much “garbage” to the end of it as you like. The shell will never look at it, so it’s possible to store the payload there.

Continue reading “Linux Fu: Shell Script File Embedding” →

BASH Template Promises Safer Scripts

December 17, 2020 by 21 Comments

Many bash scripts start out as something quick and dirty but then become so useful that they live for years, indeed sometimes seeing more use than our traditional programs. Now that you can even run bash well under Windows (although, you’ve always been able to run it there if you tried), there are even more opportunities for your five-minute bash script to proliferate. [Maciej] decided he was tired of always having to patch up his quick and dirty scripts to be more robust, so he created (and shared) his boilerplate template for scripts.

Probably most of us have at least some basic template we start with, even if it just our last script project. What’s nice about [Maciej’s] template is that he documents what’s going on with each part of it. It is also relatively short without a lot of excess stuff. Of course, you’ll probably customize it, but it is a great place to start.

Continue reading “BASH Template Promises Safer Scripts” →

Linux Fu: Literate Regular Expressions

September 11, 2020 by 20 Comments

Regular expressions — the things you feed to programs like grep — are a bit like riding a bike. It seems impossible until you learn to do it, and then it’s easy. Part of their bad reputation is because they use a very concise and abbreviated syntax that alarms people. To help people who don’t use regular expressions every day, I created a tool that lets you write them in something a little closer to plain English. Actually, I’ve written several versions of this over the years, but this incarnation that targets grep is the latest. Unlike some previous versions, this time I did it all using Bash.

Those who don’t know regular expressions might freak out when they see something like:

[0-9]{5}(-[0-9]{4})?

How long does it take to figure out what that does? What if you could write that in a more literate way? For example:

digit repeat 5 \

start_group \

   - digit repeat 4 \

end_group optional

Not as fast to type, sure. But you can probably deduce what it does: it reads US Zipcodes.

I’ve found that some of the most popular tools I’ve created over the years are ones that I don’t need myself. I’m sure you’ve had that experience, too. You know how to operate a computer, but you create a menu system for people who don’t and they love it. That’s how it is with this tool. You might not need it, but there’s a good chance you know someone who does. Along the way, the code uses some interesting features of Bash, so even if you don’t want to be verbose with your regular expressions, you might pick up a trick or two.

Continue reading “Linux Fu: Literate Regular Expressions” →

Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts

August 18, 2020 by 12 Comments

You normally think of a critical section — that is, a piece of a program that excludes other programs from using a resource — as a pretty advanced technique. You certainly don’t often think of them as part of shell scripting but it turns out they are surprisingly useful for certain scripts. Most often, a critical section is protecting some system resource like a shared memory location, but there are cases where a shell script needs similar protection. Luckily, it is really easy to add critical sections to shell scripts, and I’ll show you how.

Sometimes Scripts Need to Be Selfish

One very common case is where you want a script to run exactly one time. If the same script runs again while the original is active, you want to exit after possibly printing a message. Another common case is when you are updating some file and you need undisturbed access while making the change.

That was actually the case that got me thinking about this. I have a script — may be the subject of a future Linux-Fu — that provides dynamic DNS by altering a configuration file for the DNS server. If two copies of the script run at the same time, it is important that only one of them does modifications. The second copy can run after the first is totally complete.

Continue reading “Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts” →

A Shell? A Programming Language? Relax! It’s Both!

August 8, 2020 by 55 Comments

Every time we publish a Linux hack that uses a shell script, someone will chime in about how awful it is to program shell scripts. While we like the ubiquity and efficiency, we can’t disagree that the shell is a bit of a hack itself. [Axel Lijencrantz] wants to change your shell to be a full-blow programming language called Crush.

On the face of it, it looks like a shell. Want to see the contents of the current directory? Simple: ls.

The difference is underneath. In Crush, ls is a built-in and it returns data in rows like a database. You can manipulate that database with SQL-like commands: ls | where {type=="directory"}.

Continue reading “A Shell? A Programming Language? Relax! It’s Both!” →