The computer world looks different from behind a TeleType or other hardcopy terminal. Things that tend to annoy people about Unix or Linux these days were perfectly great when you were printing everything the computer said to you. Consider the brevity of most basic commands. When you copy a file, for example, it doesn’t really tell you much other than it returns you to the prompt when it is done. If you are on a modern computer working with normal-sized files locally, not a big deal. But if you are over a slow network or with huge files, it would be nice to have a progress bar. Sure, you could write your own version of copy, but wouldn’t it be nice to have some more generic options?
pv program can do some of the things you want. It monitors data through a pipe or, at least through its standard output. Think of it as
cat with a meter. Suppose you want to write a diskimage to
cat diskz.img >/dev/sdz
But you could also do:
pv diskz.img >/dev/sdz
By default, pv will show a progress bar, an elapsed time, an estimated end time, a rate, and a total number of bytes. You can turn any of that off or add things using command line options. You can also specify things like the size of the terminal if it should count lines instead of bytes, and, in the case where the program doesn’t know what it is reading, the expected size of the transfer.
Of course, the output of
pv is to
stderr so it shows up on the screen. This makes it tricky if you want to use it with something like dialog that wants the
stdout to be the progress indication. Luckily, you can redirect
stdout and then redirect
stdout to a file. Just be sure to do it in that order. For example:
pv -n /tmp/bigfile.iso 2>&1 >/tmp/bigfile.iso.backup | dialog --gauge "Copy Progress" 10 60
There is also
progress. It looks around for programs running like
dd, and many more, looks at their open files, and shows you progress information for those programs. It only does this once, so you’ll typically marry it with the watch command or use the
-M option. You can also monitor arbitrary programs with a little help. For example, suppose you have a big download in firefox:
watch progress -c firefox
Actually, if you are like me and have more than 32 instances of firefox running, that’s going to fail, but you get the idea.
The progress program can also take a pipe, but it still only monitors processes it knows about unless you use
-c. You can add to the list with the
-a option. For example:
tar czf /tmp/books.tgz ~/library/ebooks | progress -m
Note that since
gzip is one of the programs that progress knows about, it shows up on the list, too. You could, of course, tell it to only look for
As usual with Linux, there are always many ways to do anything. If you are a masochist, you could use
ptrace to look inside a program and figure things out from there. You could also do the same trick that
progress does and look through a program’s
If you want to add a progress bar to your shell scripts directly, try gum. Or maybe combine progress with widgets to put the result on your desktop?
11 thoughts on “Linux Fu: Making Progress”
Neato and handy. Will be trying this out later
dd has “status=progress”
mbuffer is also a handy tool.
dd_rescue is great even if not using to recover media
“When yoo copy a file”
Does “yoo” means “you”?
dd if=/dev/sdx | pv –size=80G | dd of=/backup/image.img
old but good works command, making a backup of a hdd :)
dd if=file1.img of=file2.img status=progress
How about a series on Windows Subsystem for Linux with articles on things like getting USB devices to work or gaining access to a DLNA server the windows host has no problem with, or how to fix crappy streaming performance in VLC when it’s fine in VLC for Windows?
Since microsoft allowed WSL2 on Windows 10 without kludgy hacks I decided to give it a try. The above are some of the issues with it. “Solutions” I’ve found only cover the Linux part, or are for old builds of WSL and/or Windows and MS has of course *changed stuff*.
Right now I just want to get my CanoScan LiDE 210 working with GIMP in WSL2 on Windows 10.
Funny, you should mention that I’m actually flashing a 3D printer right now over USB with WSL and waiting for it to finish so I certainly could do a piece on that. I don’t know about the VLC issue though
I’m surprised that pipeview isn’t a standard utility in Linux.
I’m curious, though, how do you have more than 32 instances of Firefox? Do you mean tabs?
I currently have 50 tabs open, but ps shows only 13 processes.
I’ve only been doing this for 13 years so I’m still leaning Linux.
I have multiple virtual desktops and each browser has way too many tabs. But I also have 12/24 processors and 64g of RAM so I am never motivated to do better lol
Please be kind and respectful to help make the comments section excellent. (Comment Policy)