Archives

Linux Fu: Leaning Down With Exec

December 30, 2019 by 11 Comments

Shell scripting is handy and with a shell like bash it is very capable, too. However, shell scripting isn’t always very efficient. Think about it. If you run grep or tr or sort to do some operation in a shell script, you are spawning a whole new process. That takes time and resources. But there are some answers to reducing — but not eliminating — the problem.

Have you ever written a program like this (in any language, but I’ll use C):

int foo(void)
{
  ...
  bar();

}

You hope the compiler doesn’t write assembly code like this:

_foo: 
....

      call _bar
      ret

Most optimizers should pick up on the fact that you can convert a call like this to a jump and let the ret statement in _bar return to foo’s caller. However, shell scripts are not that smart. If you have a shell script called MungeData and it calls another program or shell script called PostProcess on its last line, then you will have at one time three processes in play: your original shell, the shell running MungeData, and either the PostProcess program or a shell running the script. Not to mention, the processes to do things inside post process. So what do you do?

Continue reading “Linux Fu: Leaning Down With Exec” →

Linux Fu: WSL Tricks Blur The Windows/Linux Line

December 23, 2019 by 51 Comments

We have to admit, we have an odd fascination with WSL — the Windows subsystem for Linux. On the one hand, it gives us more options on Windows 10 for running the software we love. On the other hand, we wonder why we aren’t just running Linux. Sometimes it is because our cool laptop doesn’t work well on Linux. Other times we are using someone else’s computer that we aren’t allowed to reload or dual boot. Still, as long as we have to use Windows, we are glad to have WSL. A recent blog post by [Hanselman] shows some very cool tricks for using WSL that make it even better.

Exploring WSL

Did you know you can use WSL to run Linux commands in a Windows command shell? For example, you have a long directory and you want to run grep:

dir c:\archive\* | wsl grep -i hackaday

Of course, from bash you could access the same directory:

ls /mnt/c/archive | grep -i hackaday

Continue reading “Linux Fu: WSL Tricks Blur The Windows/Linux Line” →

Linux Fu: Stupid SSH Tricks

December 17, 2019 by 52 Comments

If you connect to remote computers over the Internet, it is a pretty good chance you use some form of SSH or secure shell. On Linux or Unix you’ll use the ssh command. Same goes for Linux-like environments on Windows like Cygwin or WSL. For native Windows, you might be using Putty. In its simplest form, ssh is just a terminal program that talks to a server using an encrypted connection. We think it is very hard to eavesdrop on anyone communicating with a remote computer via ssh.

There are several tricks for using ssh — some are pretty straightforward and some are things you might not think of as being in the domain of a terminal program. You probably know that ssh can copy files securely, and there are easy and hard ways to set up logging in with no password.

However, you can also mount a remote filesystem via ssh (actually, there are several ways to do that). You can use ssh to securely browse the web in your favorite browser, or even use it to tunnel specific traffic by port or even use it as a makeshift VPN. In fact, there’s so much ground to cover that this won’t be the last Linux Fu to talk about ssh. But enough setup, let’s get to the tricks.

Continue reading “Linux Fu: Stupid SSH Tricks” →

Linux Fu: Debugging Bash Scripts

December 11, 2019 by 23 Comments

A recent post about debugging constructs surprised me. There were quite a few comments about how you didn’t need a debugger, as long as you had printf. For that matter, we’ve all debugged systems where you had nothing but an LED to flash or otherwise turn on to communicate with the user. However, it is hard to deny that a debugger can help with complex code.

To say you only need printf would be like saying you only need machine language. Technically accurate — you can do anything in machine language. But it sure makes things easier to have an assembler or some language to help you work out your problem. If you write a simple bash script, you can use the equivalent to printf — maybe that’s the echo command, although there is usually a printf command on a typical system, if you want to use it. However, there are other things you can do with bash including a pretty cool debugger if you know how to find it.

I assume you already know how to use echo and printf, but let’s dig into how to use trace execution line by line without the need for echo statements on every other line. Along the way, you’ll learn how to get started with the bash debugger.

Continue reading “Linux Fu: Debugging Bash Scripts” →

Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly

October 14, 2019 by 7 Comments

Not every programmer likes creating GUI code. Most hacker types don’t mind a command line interface, but very few ordinary users appreciate them. However, if you write command line programs in Python, Gooey can help. By leveraging some Python features and a common Python idiom, you can convert a command line program into a GUI with very little effort.

The idea is pretty simple. Nearly all command line Python programs use argparse to simplify picking options and arguments off the command line as well as providing some help. The Gooey decorator picks up all your options and arguments and creates a GUI for it. You can make it more complicated if you want to change specific things, but if you are happy with the defaults, there’s not much else to it.

At first, this article might seem like a Python Fu and not a Linux Fu, since — at first — we are going to focus on Python. But just stand by and you’ll see how this can do a lot of things on many operating systems, including Linux.

Continue reading “Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly” →

Linux Fu: Shell Scripts In C, C++, And Others

September 17, 2019 by 44 Comments

At first glance, it might not seem to make sense to write shell scripts in C/C++. After all, the whole point to a shell script is to knock out something quick and dirty. However, there are cases where you might want to write a quick C program to do something that would be hard to do in a traditional scripting language, perhaps you have a library that makes the job easier, or maybe you just know C and can knock it out faster.

While it is true that C generates executables, so there’s no need for a script, usually, the setup to build an executable is not what you want to spend your time on when you are just trying to get something done. In addition, scripts are largely portable. But sending an executable to someone else is fairly risky — but your in luck because C shell scripts can be shared as… well, as scripts. One option is to use a C interpreter like Cling. This is especially common when you are using something like Jupyter notebook. However, it is another piece of software you need on the user’s system. It would be nice to not depend on anything other than the system C compiler which is most likely gcc.

Luckily, there are a few ways to do this and none of them are especially hard. Even if you don’t want to actually script in C, understanding how to get there can be illustrative.

Continue reading “Linux Fu: Shell Scripts In C, C++, And Others” →

Linux Fu: Interactive SSH Applications

September 6, 2019 by 4 Comments

[Drew DeVault] recently wrote up some interesting instructions on how to package up interactive text-based Linux commands for users to access via ssh. At first, this seems simple, but there are quite a few nuances to it and [Drew] does a good job of covering them.

One easy way — but not very versatile — is to create a user and make the program you want to run the default shell. The example used is to make /usr/bin/nethack the shell and now people can log in as that user and play nethack. Simple, right? However, there are better ways to get there.

Continue reading “Linux Fu: Interactive SSH Applications” →