Imagine that you want to output multiple lines of text in Bash, or any shell script. Maybe it’s for a help string for a particularly convoluted shell script you’re writing. You could have a separate echo
command for each line. Or you could use the “here document“.
The “here document” construction takes the text between two delimiters and passes it, as if it were piped, to a command.
if [[$# == 0 ]] || [[ "$1" == "-h" ]]; then cat << EOF This is my help message. There are many like it but this one is mine. My help message is my best friend. EOF
All of the text, as written, with line breaks and spaces and all, get passed to cat
and your helpful formatted message is printed to the user.
You can use here documents interactively at the command prompt, and you can use them in functions:
#!/bin/bash help() { cat << EOF Here is your help. EOF }
More About Here Documents
More than just writing help scripts, here documents are useful any time that you want to enter multiple lines and have them piped to a command. We use them most often from the shell directly. There are a few nuances, however.
You probably figured out that the “EOF” string in the above examples marks the beginning and end of the here document. There’s nothing special about it. You can use any word that doesn’t appear in your document. So “<< END_OF_HERE_DOC” would work.
The end of document string has to be at the beginning of a line, and by itself. You should not have whitespace before or after it. It is usually a good idea to use something odd and more than a few characters as the end string, although — in theory — you could use even a single character.
But that requirement messes up code indentation — see the help()
function above. If you put a dash between the redirection brackets and the string, the shell will consume any leading tab characters (but not spaces!). That lets you indent your message so it doesn’t all run together. But here, you have to be careful about spaces in the declaration: cat <<- EOF
and cat <<-EOF
work, while cat << -EOF
doesn’t. (It expects the delimiter “-EOF”.) Still, it lets you indent your code if you can get it right, and use tabs instead of spaces.
#!/bin/bash help() { cat <<- EOF Here is your help. EOF }
Substitution
One thing that might not be obvious is that the here document can handle variable expansion. Try typing:
cat << EOF Your path is $PATH Your prompt is $PS1 EOF Your path is /usr/local/sbin:/usr/local/bin:... Your prompt is ($HOSTNAME \w)\$
That even works with things like command substitution. So you could put things in like $(ls)
, for example. Sometimes you don’t want that, though. In those cases, just quote the end string. For example:
cat << "EOF" Your path is $PATH Your prompt is $PS1 EOF Your path is $PATH Your prompt is $PS1
Peculiarities
You can even send the input to a program that does nothing like :
which might seem odd. But some people use that to comment out large blocks of script for debugging or other purposes. It is a bit wasteful, because the lines you don’t want get put in a temporary file and fed to the input of a program that will ignore them, but you might see it done.
Clearly, here documents are niche. I use them a lot for writing help strings in scripts, but they’re just the ticket when you need to pass multiple lines in a script or to a script as you typed multiple lines into stdin
.
Anyone who is writing bash scripts in this day and age where there are so many far superior languages to choose from needs help alright.
I don’t have python on my hpux and Solaris servers, I do have Bourne though
There are reasons to write in the scripting language that is guaranteed to be installed on all platforms you support. Of course, there are also reasons why such scripts are generally small programs used as utilities or “glue” and not as full-fledged applications.
Also, maintaining an existing script is much more likely than creating a new one.
But also, since this exists…
https://xinthecity.wordpress.com/2010/02/04/is-that-a-wenger-giant-swiss-army-knife-in-your-pocket-or-are-you-just-happy-to-see-me/
I demand that you spread your toast with it every morning. Or are you a hypocrite?
Wrong
Indeed, it is about getting the job done
“This is my help message. There are many like it but this one is mine.
My help message is my best friend.”
So, a continuing evolution of the Rifleman’s Creed?
A great use case is automating ftp.
ftp http://www.example.com <<EOF
user anonymous
get some_file.txt
bye
EOF
Oy, I’m getting tired of HaD commenters. Grow the he** up.
In other news, I’ve apparently been using here-docs without even realizing it, with the double-quote as the delimiter. Cool to know so much more can be done.
All generalizations suck. Yours, on top of that, shows your ignorance.
Bash (generally, shell) is a nice language. Not built for large projects, but with its expansion strategy, it sports a powerful macro language. It is ideal for small, interactive tasks and as a glue language — exactly what it was made for.
Know your tools. Pick the right one for the job.
What’s Linux specific about this?
Well, it has the common decency to fail under VMS. :-)
Not exactly true:
https://sourceforge.net/projects/gnv/files/bash/
Non-technical people may be excused for not knowing the difference between a family of OSes and a specific OS, but Hackaday should certainly know and shouldn’t promote generalizing one in place of the other.
It’s so wrong, but it’s the best kind of wrong. It’s time to activate the SIMH…
It works with ssh too, if you need some remote work:
ssh somehost /bin/sh <<eof
echo \$hostname
ls ; ps -ef|head -n 10
eof
When using here docs vs echo/printf, I think it should be noted (more clearly) that generally speaking the here doc will generate a temporary file. Even when printing help texts. That also means you get potentially a write on your disk, with all potential downsides of that. Use wisely, or, cavaet emptor.
You forgot to mention the “<<<" variant of the here document:
cat <<< "$text"
For short messages or text already stored in a variable this is more convenient than the << EOF syntax.
Cave canem!
The “here string” method adds a linefeed, just as “here document” does.
Another use case for heredocs: passing data to programs via file descriptor pseudo-files.
Example: executing a curl command with HTTP authorization, without having the username and password appear in the process name (that anyone could see by running ps):
curl –config /dev/fd/3 https://example.org/ 3<<EOF
–user username:password
EOF