If you ever need to write a binary file from a traditional language like C, it isn’t all that hard to do. About the worst thing you might have to deal with is attempts to fake line endings across Windows and Linux, but there’s usually a way to turn that off if it is on by default. However, if you are using some type of scripting language, binary file support might be a bit more difficult. One answer is to use a tool like xxd or t2b (text-to-binary) to handle the details. You can find the code for t2b on GitHub including prebuilt binaries for many platforms. You should be able to install xxd from your system repository.
These tools take very different approaches. You might be familiar with tools like od or hexdump for producing readable representations of binary files. The xxd tool can actually do the same thing — although it is not as flexible. What xxd can even reverse itself so that it can rebuild a binary file from a hex dump it creates (something other tools can’t do). The t2b tool takes a much different approach. You issue commands to it that causes it to write an original hex file.
Both of these approaches have some merit. If you are editing a binary file in a scripting language, xxd makes perfect sense. You can convert the file to text, process it, and then roll it back to binary using one program. On the other hand, if you are creating a binary file from scratch, the t2b program has some advantages, too.
I decided to write a few test scripts using bash to show how it all works. These aren’t production scripts so they won’t be as hardened as they could be, but there is no reason they couldn’t be made as robust as you were willing to make them.
Continue reading “Linux Fu: Scripting for Binary Files”
Truly good ideas tend to apply in all situations. The phrase is “never run with scissors”, not “don’t run with scissors unless you are just going into the next room.” Software development methodology is a good idea and most of us have our choice of tools. But what if you are developing a significant amount of bash or similar script? Should you just wing it because bash isn’t a “real” programming language? [Oscar] says no, and if you are writing more than two or three lines of script, we agree.
We’ve made the argument before (and many of you have disagreed) that bash is a programming language. Maybe not the greatest and certainly not the sexiest, but bash is near ubiquitous on certain kinds of systems and for many tasks is pretty productive. [Oscar] shows how he uses a source code formatter, a linter, and a unit test framework to bring bash scripting in line with modern software development. We are pretty sure he uses source control, too, but that seems so elementary that it doesn’t come up outside of a link to his repository in GitHub.
Continue reading “Software Development in… Bash”
It isn’t uncommon these days for a programmer’s editor to offer you help about what you are typing, ranging from a pop up with choices to a full-blown code template. If you have written a million lines of code in the language, this might even annoy you. However, if you use it only occasionally, these can be very helpful. I’ve used Unix and Linux for many years, but I realize that there are people who don’t use it every day. With the Raspberry Pi, Linux servers, and Windows 10 having a bash shell, there are more people using a shell “every once in a while” than ever before. Could you use a little help? If so, you might try
bashelp: a little something I put together while writing about bash completion.
There’s good news and bad news. The good news is that Unix has a built-in help command —
man — and has for some time. The bad news is that you need to stop what you are typing and enter a
man command to use it.
Man, by the way, is short for manual.
There are GUI front-ends to
yelp, on the left) and you can even use a web browser locally or remotely. However, none of these are connected to what you are typing. You have to move to another window, enter your search term, then go back to your typing. That got me to thinking about how to get a sort of context-sensitive inline help for bash.
Continue reading “Linux Fu: A Little Help for Bash”
If you aren’t a Linux user and you watch someone who knows what they are doing use Bash — the popular command line interpreter — you might get the impression they type much faster than they actually do. That’s because experienced Linux users know that pressing the tab key will tend to complete what they are typing, so you can type just a few characters and get a much longer line of text. The feature is very smart so you may not have realized it, but it knows a good bit about what you could type. For example, if you try to unzip a file, it knows the expected file name probably has a .zip extension.
How does that happen? At first, you might think, “who cares how it happens?” The problem is when you write a shell script or a program that runs on Linux, the completion gets dumb. Someone has to make Bash smart about each command line program and if you are the author then that someone is you.
Continue reading “Linux Fu: Custom Bash Command Completion”
It is easy to dismiss bash — the typical Linux shell program — as just a command prompt that allows scripting. Bash, however, is a full-blown programming language. I wouldn’t presume to tell you that it is as fast as a compiled C program, but that’s not why it exists. While a lot of people use shell scripts as an analog to a batch file in MSDOS, it can do so much more than that. Contrary to what you might think after a casual glance, it is entirely possible to write scripts that are reliable and robust enough to use in many embedded systems on a Raspberry Pi or similar computer.
I say that because sometimes bash gets a bad reputation. For one thing, it emphasizes ease-of-use. So while it has features that can promote making a robust script, you have to know to turn those features on. Another issue is that a lot of the functionality you’ll use in writing a bash script doesn’t come from bash, it comes from Linux commands (or whatever environment you are using; I’m going to assume some Linux distribution). If those programs do bad things, that isn’t a problem specific to bash.
One other limiting issue to bash is that many people (and I’m one of them) tend to write scripts using constructs that are compatible with older shells. Often times bash can do things better or neater, but we still use the older ways. For example:
Continue reading “Linux Fu: Better Bash Scripting”
My apologies if you speak the Queen’s English since that title probably has a whole different meaning to you than I intended. In fact, I’m talking about Git, the version control system. Last time I talked about how the program came to be and offered you a few tutorials. If you are a dyed-in-the-wool software developer, you probably don’t need to be convinced to use Git. But even if you aren’t, there are a lot of things you can do with Git that don’t fit the usual mold.
Continue reading “Stupid Git Tricks”
It used to be one of the joys of writing embedded software was never having to deploy shell scripts. But now with platforms like the Raspberry Pi becoming very common, Linux shell scripts can be a big part of a system–even the whole system, in some cases. How do you know your shell script is error-free before you deploy it? Of course, nothing can catch all errors, but you might try ShellCheck.
When you compile a C program, the compiler usually catches most of your gross errors. The shell, though, doesn’t look at everything until it runs which means you might have an error just waiting for the right path of an if statement or the wrong file name to occur. ShellCheck can help you identify those issues before deployment.
If you don’t like pasting your script into a Web page, you can install the checker locally by visiting GitHub. The readme file there also explains what kind of things the tool can catch. It can even integrate with common editors (as seen in the video below).
Continue reading “Lint for Shell Scripters”