Linux Fu: Mixing Bash And Python

May 4, 2021 by Al Williams 24 Comments

Although bash scripts are regularly maligned, they do have a certain simplicity and ease of creation that makes them hard to resist. But sometimes you really need to do some heavy lifting in another language. I’ll talk about Python, but actually, you can use many different languages with this technique, although you might need a little adaptation, depending on your language of choice.

Of course, you don’t have to do anything special to call another program from a bash script. After all, that’s what it’s mainly used for: calling other programs. However, it isn’t very handy to have your script spread out over multiple files. They can get out of sync and if you want to send it to someone or another machine, you have to remember what to get. It is nicer to have everything in one file.

Continue reading “Linux Fu: Mixing Bash And Python” →

Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts

August 18, 2020 by Al Williams 12 Comments

You normally think of a critical section — that is, a piece of a program that excludes other programs from using a resource — as a pretty advanced technique. You certainly don’t often think of them as part of shell scripting but it turns out they are surprisingly useful for certain scripts. Most often, a critical section is protecting some system resource like a shared memory location, but there are cases where a shell script needs similar protection. Luckily, it is really easy to add critical sections to shell scripts, and I’ll show you how.

Sometimes Scripts Need to Be Selfish

One very common case is where you want a script to run exactly one time. If the same script runs again while the original is active, you want to exit after possibly printing a message. Another common case is when you are updating some file and you need undisturbed access while making the change.

That was actually the case that got me thinking about this. I have a script — may be the subject of a future Linux-Fu — that provides dynamic DNS by altering a configuration file for the DNS server. If two copies of the script run at the same time, it is important that only one of them does modifications. The second copy can run after the first is totally complete.

Continue reading “Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts” →

Give A Man A Phish, And You Entertain Him For A Day

August 16, 2020 by Sven Gregori 11 Comments

With millions of phishing attempts happening daily, we’ve probably all had our fair share of coming across one. For the trained or naturally suspicious eye, it’s usually easy to spot them — maybe get a good chuckle out of the ridiculously bad ones along the way — and simply ignore them. Unfortunately, they wouldn’t exist if they weren’t successful enough in the big picture, so it might be a good idea to inform the targeted service about the attempt, in hopes they will notify users to act with caution. And then there’s [Christian Haschek], who decided to have some fun and trying to render the phished data useless by simply flooding it with garbage.

After his wife received a text message from “their bank”, [Christian] took a closer look at the URL it was pointing to, and found your typical copy of the real login form at a slightly misspelled address. As the usual goal is to steal the victim’s credentials, he simply wrote a shell script that sends random generated account numbers and PINs for all eternity via cURL, potentially lowering any value the attackers could get from their attempt.

As the form fields limit the input length of the account number and PIN, he eventually wondered if the server side will do the same, or whether it would crash if longer data is sent to it. Sadly, he’ll never know, because after he modified the script, the site itself returned a 404 and had disappeared.

In the quest against phishing attacks, this should count as a success, but as [Christian] seemed to enjoy himself, he yearned for more and decided to take a look at a similar attempt he saw mentioned earlier on Reddit. Despite targeting the same bank, the server-side implementation was more sophisticated, hinting at a different attack, and he definitely got his money worth this time — but we don’t want to give it all away here.

Rest assured, [Christian Haschek] continues the good fight, whether by annoying attackers as he did with ZIP-bombing random WordPress login attempts or battling child pornography with a Raspberry Pi cluster. Well, unless he’s busy hunting down an unidentified device hooked up in his own network.

(Banner image by Tumisu)

Linux Fu: Debugging Bash Scripts

December 11, 2019 by Al Williams 23 Comments

A recent post about debugging constructs surprised me. There were quite a few comments about how you didn’t need a debugger, as long as you had printf. For that matter, we’ve all debugged systems where you had nothing but an LED to flash or otherwise turn on to communicate with the user. However, it is hard to deny that a debugger can help with complex code.

To say you only need printf would be like saying you only need machine language. Technically accurate — you can do anything in machine language. But it sure makes things easier to have an assembler or some language to help you work out your problem. If you write a simple bash script, you can use the equivalent to printf — maybe that’s the echo command, although there is usually a printf command on a typical system, if you want to use it. However, there are other things you can do with bash including a pretty cool debugger if you know how to find it.

I assume you already know how to use echo and printf, but let’s dig into how to use trace execution line by line without the need for echo statements on every other line. Along the way, you’ll learn how to get started with the bash debugger.

Continue reading “Linux Fu: Debugging Bash Scripts” →

Give Your Raspberry Pi SD Card A Break: Log To RAM

April 8, 2019 by Dan Maloney 82 Comments

The fragility of SD cards is the weak link in the Raspberry Pi ecosystem. Most of us seem to have at least one Pi tucked away somewhere, running a Magic Mirror, driving security cameras, or even taking care of a media library. But chances are, that Pi is writing lots and lots of log files. Logging is good — it helps when tracking down issues — but uncontrolled logging can lead to problems down the road with the Pi’s SD card.

[Erich Styger] has a neat way to avoid SD card logging issues on Raspberry Pi, he calls it a solution to reduce “thrashing” of the SD card. The problem is that flash memory segments wear out after a fairly low number of erase cycles, and the SD card’s wear-leveling algorithm will eventually cordon off enough of the card to cause file system issues. His “Log2Ram” is a simple Unix shell script that sets up a mount point for logging in RAM rather than on the SD card.

The idea is that any application or service sending log entries to /var/log will actually be writing them to virtual log files, which won’t rack up any activity on the SD card. Every hour, a cron job sweeps the virtual logs out to the SD card, greatly reducing its wear. There’s still a chance to lose logging data before it’s swept to disk, but if you have relatively stable system it’s a small price to pay for the long-term health of a Pi that’s out of sight and out of mind.

One thing we really like about [Erich]’s project is that it’s a great example of shell scripting and Linux admin concepts. If you need more information on such things, check out [Al Williams’] Linux-Fu series. It goes back quite a way, so settle in for some good binge reading.

Linux Fu: Controlling The Terminal

November 28, 2018 by Al Williams 17 Comments

A Linux terminal has a lot more features than the TeleType of yore. On a TeleType, text spews out and scrolls up and is gone forever. A real terminal can use escape characters to do navigate around and emulate most of what you like about GUIs. However, doing this at the lowest level is a chore and limits portability. Luckily, all the hard work has already been done.

First, there’s a large database of terminal capabilities available for you to use: terminfo. And in addition, there’s a high-level library called curses or ncurses that simplifies writing programs to control the terminal display. Digging deep into every nook and cranny of ncurses could take years. Instead, I’m going to talk about using a program that comes with ncurses to control the terminal, called tput. Using these two commands, you can figure out what kind of terminal you’re dealing with, and then manipulate it nearly to your heart’s content. Let’s get started!

Continue reading “Linux Fu: Controlling The Terminal” →

Set Up A Headless Raspberry Pi, All From Another Computer’s Command Line

November 24, 2018 by Donald Papp 17 Comments

There are differences between setting up a Raspberry Pi and installing an OS on any other computer, but one thing in common is that if you do enough of them, you seek to automate the process any way you can. That is the situation [Peter Lorenzen] found himself in, and his solution is a shell script to install and configure the Raspberry Pi for headless operation, with no need to connect either a keyboard or monitor in the process.

[Peter]’s tool is a script called rpido, and with it the process for setting up a new Raspberry Pi for headless operation is super streamlined. To set up a new Pi, all [Peter] needs to do is:

Plug an SD card into his laptop (which happens to be running Ubuntu.)
Run: rpido -w -h myhostname -s which downloads and installs the newest version of Raspbian lite, does some basic setup (such as setting the hostname), configures for headless operation, and launches a root shell.
Use the root shell to do any further tweaks or checks (like launching raspi-config for additional changes.)
Exit the shell, remove the SD card from his laptop, and install the card into the Raspberry Pi.

There are clear benefits to [Peter]’s script compared to stepping through a checklist of OS install and setup tasks, not to mention the advantage of not needing to plug in a keyboard and monitor. Part of the magic is that [Peter] is mounting the SD card’s filesystem in a chroot environment. Given the right tools, the ARM binaries intended for the Pi run on his (Intel) Ubuntu laptop. It’s far more convenient to make changes to the contents of the SD card in this way, before it goes to its new home in a Pi.

Not everything has to revolve around an SD card, however. [Jonathan Bennet] showed that it’s possible to run a Raspberry Pi without an SD card by using the PXE boot feature, allowing it to boot and load its file system from a server on the same network, instead of a memory card.