It used to be one of the joys of writing embedded software was never having to deploy shell scripts. But now with platforms like the Raspberry Pi becoming very common, Linux shell scripts can be a big part of a system–even the whole system, in some cases. How do you know your shell script is error-free before you deploy it? Of course, nothing can catch all errors, but you might try ShellCheck.
When you compile a C program, the compiler usually catches most of your gross errors. The shell, though, doesn’t look at everything until it runs which means you might have an error just waiting for the right path of an if statement or the wrong file name to occur. ShellCheck can help you identify those issues before deployment.
If you don’t like pasting your script into a Web page, you can install the checker locally by visiting GitHub. The readme file there also explains what kind of things the tool can catch. It can even integrate with common editors (as seen in the video below).
For example, a common issue with shell scripts is to forget to quote an environment variable that could hold a file name that contains spaces. For example, consider this really simple script:
#!/bin/bash echo Processing $1 cat $1 >$2
This works fine as long as the files have no spaces in the name. However, suppose the name of the input file is “Hack A Day.txt.” Then you get:
$ ./bad "Hack A Day.txt" hello Processing Hack A Day.txt cat: Hack: No such file or directory cat: A: No such file or directory cat: Day.txt: No such file or directory
The $1 expands to three words: Hack, A, and Day.txt. For echo, that’s not a problem. It just prints them. But the last line turns into:
cat Hack A Day.txt >hello
That’s not going to work (unless there are three files with those names at which point it will just not work correctly). The answer, of course, is to put quotes around $1 (and $2, for that matter) on the cat line. Won’t hurt on the echo line either.
ShellCheck will find those kinds of problems and more. For example, here’s part of the output from a more ambitious script:
Line 5: if [ "$0" == "$BASH_SOURCE" ] ^-- SC2128: Expanding an array without an index only gives the first element. Line 8: echo E.g.: . $0 ^-- SC2086: Double quote to prevent globbing and word splitting. Line 15: alias pathed=". $BASH_SOURCE" ^-- SC2128: Expanding an array without an index only gives the first element. ^-- SC2139: This expands when defined, not when used. Consider escaping. Line 29: if [ "$PATHEDIT_OLD" = "$PATHEDIT_NEW" -o ! -s "$PATHEDIT_FN" ] ^-- SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.
Like any similar tool, you do have to interpret the results. For example, the echo statement in line 8 doesn’t really care if the $0 has quotes around it or not, but the tool can’t figure that out.
Will ShellCheck catch everything? No. But it will catch things that you might not find for quite some time after deploying a shell script.
If you don’t think you can do a lot in a bash script, think again. If that’s not practical enough for you, consider scraping data off a web page, as an example.
That is handy even for the warnings about deprecated syntax in older scripts.
You can find a lot of scripts to test it with using,
file /usr/bin/* |grep BourneNice find. I usually make a few scripts that’s in the a-few-hundred-lines region per year. Sob this will b come in handy.
I wonder if it can check usage of external utilities and make a list and/or warn of things that isn’t installed in the dist by default…
Ugh, I hate lint. Always clogs up my dryer. :P
Not liking shell scripts? this is blasphemy!
Only in shell scripts can you make a one-liner that uses sed to edit part of itself as text using external variables and then execute it with eval and attain Code (dirtiness) Nirvana!
The ShellCheck in the debian (sid, at least) repo as is, or integrated into the syntastic vim plugin ( apt install vim-syntastic && vam-install syntastic ), by the way.
It seems great… and then you learn it’s written in Haskell. -_-
The only issue I heard with Haskell-based stuff is compiling it from source, but this probably isn’t an issue as long as it’s packaged for your distribution. Do you have something else in mind? Genuinely asking.
Exactly. I just wanted to get it running on OSX. Install instructions are simple… as long as you’ve already bootstrapped a haskell compiler environment. *sigh* And no binary packages available. Not going to go to the trouble of brew/macports just for this. Yeah – that’s my own choice… why am I complaining? Don’t know.
% brew install shellcheck
Error: shellcheck: no bottle available!
Shell?
Wake up!
We’re in the 21th century!
Please write your stuff in real languages!
Why don’t you just shut up? Shell is perfectly fine to use in some cases.
And in some cases, shell programming is by far the best choice.
Heh. Even Microsoft woke up and smelled the roses with Powershell. GUIs are so 20th century :)
Your name seems very well chosen… “doof” is the german word for “stupid”.
@Topic: Thanks, might be usefull one day!
Yeah, long before the time I go online and deal with “ShellCheck” the mutex will bring the system to its knees.
I am sorry I didn’t show up earlier to jump on this topic.
This is wrong on so many levels. Like “fixing” a frayed and crumbling extension cord by wrapping it with duct tape.
Nobody should be writing shell scripts in this day and age!! Period!
Use ruby, or even python if you must.
About 13 years ago I discovered perl (so use perl instead of shell scripting for crying out loud). Never wrote a shell script ever again. There is so much weird and lame stuff involved in bash (or csh, but don’t get me started). It was a perverse hack job back in the day when there were no other alternatives. So …..
If I ever see a situation where I need to do looping or testing, I jump to ruby. People call ruby “perl done right”. I have my own soft sentimental and nostalgic spot for perl, but am loathe to recommend it to anyone these days given the existence of ruby and python.
But shell scripts are so 1980’s — if you are into some kind of retro-computing, have at it. My advice is to invest your precious brain cells elsewhere. Use ruby or python for real work and do some Haskell on the side for recreation.
is Perl already installed on common Linux distributions? Doing BATCH on Windows is a real pita, but the problem with Perl (which i like) on this OS is that you have to install it first – something you might not be allowed to do for example at work – and it’s not a small thing too… My “strawberry”-folder is 460MB with 16.000(!!!) files…
It is certainly installed on every linux distro I have ever used. Either that or it (or ruby) is readily installed via a package manager. Heck, I remember almost 10 years ago in Best Buy (filthy place!) I was playing with a Macbook and got to the command line and typed “perl -v” and there it was! With windows, all bets are off though.
In the Red Hat kickstart environment, there’s no perl. Shell and python only. The admin tools no longer depend on perl either.
Bash, perl, ruby, python, they all have their places, especially when it comes to what’s available (sometimes all you’ve got is bash, or worse, cmd). Bash is great for snapping off little automations or doing something *right now*.
If it starts to grow though and become a fixture in the system, that’s when I start looking at porting into a real language.
Besides, is there anything more haker-tastic than stringing together grep, sed and awk commands?
I adopted perl with the express purpose of not becoming intimate with sed and awk (and bash scripting). And I did this at least 12 years ago (I recognized then that these venerable tools had passed their prime). There is something hackish (but I would not say hackerish) about using them these days. It might have been hacker-ish when bash and awk and sed were the only tools available, but I recognized that my brain cells were far better spent learning a modern language. These days I wouldn’t even call perl a modern language. The camel is the mascot for a good reason.
No one picked up on the bug in the example code?!? I expected more from HaD Trolls.
#!/bin/bash
echo Processing $1
cat $1 >$2
This works fine as long as the files have no spaces in the name. However, suppose the name of the input file is “Hack A Day.txt.” Then you get:
$ ./bad “Hack A Day.txt” hello
Processing Hack A Day.txt
cat: Hack: No such file or directory
cat: A: No such file or directory
cat: Day.txt: No such file or directory
The $1 expands to three words: Hack, A, and Day.txt. For echo, that’s not a problem. It just prints them. But the last line turns into:
$1 doesn’t expact to three words, it expands to only the first word ‘Hack’.
Nevermind, I’m the id10t. I didn’t look close enough to how the command was executed.