The fragility of SD cards is the weak link in the Raspberry Pi ecosystem. Most of us seem to have at least one Pi tucked away somewhere, running a Magic Mirror, driving security cameras, or even taking care of a media library. But chances are, that Pi is writing lots and lots of log files. Logging is good — it helps when tracking down issues — but uncontrolled logging can lead to problems down the road with the Pi’s SD card.
[Erich Styger] has a neat way to avoid SD card logging issues on Raspberry Pi, he calls it a solution to reduce “thrashing” of the SD card. The problem is that flash memory segments wear out after a fairly low number of erase cycles, and the SD card’s wear-leveling algorithm will eventually cordon off enough of the card to cause file system issues. His “Log2Ram” is a simple Unix shell script that sets up a mount point for logging in RAM rather than on the SD card.
The idea is that any application or service sending log entries to /var/log will actually be writing them to virtual log files, which won’t rack up any activity on the SD card. Every hour, a cron job sweeps the virtual logs out to the SD card, greatly reducing its wear. There’s still a chance to lose logging data before it’s swept to disk, but if you have relatively stable system it’s a small price to pay for the long-term health of a Pi that’s out of sight and out of mind.
One thing we really like about [Erich]’s project is that it’s a great example of shell scripting and Linux admin concepts. If you need more information on such things, check out [Al Williams’] Linux-Fu series. It goes back quite a way, so settle in for some good binge reading.
A Linux terminal has a lot more features than the TeleType of yore. On a TeleType, text spews out and scrolls up and is gone forever. A real terminal can use escape characters to do navigate around and emulate most of what you like about GUIs. However, doing this at the lowest level is a chore and limits portability. Luckily, all the hard work has already been done.
First, there’s a large database of terminal capabilities available for you to use:
terminfo. And in addition, there’s a high-level library called
ncurses that simplifies writing programs to control the terminal display. Digging deep into every nook and cranny of
ncurses could take years. Instead, I’m going to talk about using a program that comes with
ncurses to control the terminal, called
tput. Using these two commands, you can figure out what kind of terminal you’re dealing with, and then manipulate it nearly to your heart’s content. Let’s get started!
Continue reading “Linux Fu: Controlling the Terminal”
There are differences between setting up a Raspberry Pi and installing an OS on any other computer, but one thing in common is that if you do enough of them, you seek to automate the process any way you can. That is the situation [Peter Lorenzen] found himself in, and his solution is a shell script to install and configure the Raspberry Pi for headless operation, with no need to connect either a keyboard or monitor in the process.
[Peter]’s tool is a script called
rpido, and with it the process for setting up a new Raspberry Pi for headless operation is super streamlined. To set up a new Pi, all [Peter] needs to do is:
- Plug an SD card into his laptop (which happens to be running Ubuntu.)
rpido -w -h myhostname -s which downloads and installs the newest version of Raspbian lite, does some basic setup (such as setting the hostname), configures for headless operation, and launches a root shell.
- Use the root shell to do any further tweaks or checks (like launching
raspi-config for additional changes.)
- Exit the shell, remove the SD card from his laptop, and install the card into the Raspberry Pi.
There are clear benefits to [Peter]’s script compared to stepping through a checklist of OS install and setup tasks, not to mention the advantage of not needing to plug in a keyboard and monitor. Part of the magic is that [Peter] is mounting the SD card’s filesystem in a chroot environment. Given the right tools, the ARM binaries intended for the Pi run on his (Intel) Ubuntu laptop. It’s far more convenient to make changes to the contents of the SD card in this way, before it goes to its new home in a Pi.
Not everything has to revolve around an SD card, however. [Jonathan Bennet] showed that it’s possible to run a Raspberry Pi without an SD card by using the PXE boot feature, allowing it to boot and load its file system from a server on the same network, instead of a memory card.
Sound eXchange, or SoX, the “Swiss Army knife of audio manipulation” has been around for as long as the Linux kernel, and in case you’re not familiar with it, is a command line tool to play, record, edit, generate, and process audio files. [porkostomus] was especially interested about the generating part, and wrote a little shell script that utilizes SoX’s built-in synthesizer to compose 8-bit style music.
The script comes with a simple yet straightforward user interface to record the lead and bass parts into a text file, and play them back later on. Notes from C2 to C5 are currently supported, and are mapped to the keyboard in a two-row piano layout. The output file format itself is just a plain text listing of the played note, wave form, and note length. This lets you easily edit the song or even generate it from an alternative source, for example MIDI. Also note that there are no initial audio files required here, SoX will generate them as needed.
Admittedly, the command line interface may not be the most convenient way to create music, but nevertheless, it is a way — and that is [porkostomus]’s main mission here. Also, SoX is fun — and versatile, you can apply its audio effects even on images, or decode strange signals sent from a helicopter with it.
Truly good ideas tend to apply in all situations. The phrase is “never run with scissors”, not “don’t run with scissors unless you are just going into the next room.” Software development methodology is a good idea and most of us have our choice of tools. But what if you are developing a significant amount of bash or similar script? Should you just wing it because bash isn’t a “real” programming language? [Oscar] says no, and if you are writing more than two or three lines of script, we agree.
We’ve made the argument before (and many of you have disagreed) that bash is a programming language. Maybe not the greatest and certainly not the sexiest, but bash is near ubiquitous on certain kinds of systems and for many tasks is pretty productive. [Oscar] shows how he uses a source code formatter, a linter, and a unit test framework to bring bash scripting in line with modern software development. We are pretty sure he uses source control, too, but that seems so elementary that it doesn’t come up outside of a link to his repository in GitHub.
Continue reading “Software Development in… Bash”
It is easy to dismiss bash — the typical Linux shell program — as just a command prompt that allows scripting. Bash, however, is a full-blown programming language. I wouldn’t presume to tell you that it is as fast as a compiled C program, but that’s not why it exists. While a lot of people use shell scripts as an analog to a batch file in MSDOS, it can do so much more than that. Contrary to what you might think after a casual glance, it is entirely possible to write scripts that are reliable and robust enough to use in many embedded systems on a Raspberry Pi or similar computer.
I say that because sometimes bash gets a bad reputation. For one thing, it emphasizes ease-of-use. So while it has features that can promote making a robust script, you have to know to turn those features on. Another issue is that a lot of the functionality you’ll use in writing a bash script doesn’t come from bash, it comes from Linux commands (or whatever environment you are using; I’m going to assume some Linux distribution). If those programs do bad things, that isn’t a problem specific to bash.
One other limiting issue to bash is that many people (and I’m one of them) tend to write scripts using constructs that are compatible with older shells. Often times bash can do things better or neater, but we still use the older ways. For example:
Continue reading “Linux Fu: Better Bash Scripting”
It used to be one of the joys of writing embedded software was never having to deploy shell scripts. But now with platforms like the Raspberry Pi becoming very common, Linux shell scripts can be a big part of a system–even the whole system, in some cases. How do you know your shell script is error-free before you deploy it? Of course, nothing can catch all errors, but you might try ShellCheck.
When you compile a C program, the compiler usually catches most of your gross errors. The shell, though, doesn’t look at everything until it runs which means you might have an error just waiting for the right path of an if statement or the wrong file name to occur. ShellCheck can help you identify those issues before deployment.
If you don’t like pasting your script into a Web page, you can install the checker locally by visiting GitHub. The readme file there also explains what kind of things the tool can catch. It can even integrate with common editors (as seen in the video below).
Continue reading “Lint for Shell Scripters”