Archives

Linux-Fu: Your Own Dynamic DNS

August 25, 2020 by 14 Comments

It is a problem as old as the Internet. You want to access your computer remotely, but it is behind a router that randomly gets different IP addresses. Or maybe it is your laptop and it winds up in different locations with, again, different IP addresses. There are many ways to solve this problem and some of them are better than others.

A lot of routers can report their IP address to a dynamic DNS server. That used to be great, but now it seems like many of them hound you to upgrade or constantly renew so you can see their ads. Some of them disappear, too. If your router vendor supplies one, that might be a good choice, until you change routers, of course. OpenWRT supports many such services and there are many lists of common services.

However, if you have a single public accessible computer, for example a Web server or even a cloud instance, and you are running your own DNS server, you really don’t need one of those services. I’m going to show you how I do it with an accessible Linux server running Bind. This is a common setup, but if you have a different system you might have to adapt a bit.

There are many ways to set up dynamic DNS if you are willing to have a great deal of structure on both sides. Most of these depend on setting up a secret key to allow for DNS updates and some sort of script that calls nsupdate or having the DHCP server do it. The problem is, I have a lot of client computers and many are set up differently. I wanted a system where the only thing needed on the client side was ssh. All the infrastructure remains on the DNS server.

Continue reading “Linux-Fu: Your Own Dynamic DNS” →

Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts

August 18, 2020 by 12 Comments

You normally think of a critical section — that is, a piece of a program that excludes other programs from using a resource — as a pretty advanced technique. You certainly don’t often think of them as part of shell scripting but it turns out they are surprisingly useful for certain scripts. Most often, a critical section is protecting some system resource like a shared memory location, but there are cases where a shell script needs similar protection. Luckily, it is really easy to add critical sections to shell scripts, and I’ll show you how.

Sometimes Scripts Need to Be Selfish

One very common case is where you want a script to run exactly one time. If the same script runs again while the original is active, you want to exit after possibly printing a message. Another common case is when you are updating some file and you need undisturbed access while making the change.

That was actually the case that got me thinking about this. I have a script — may be the subject of a future Linux-Fu — that provides dynamic DNS by altering a configuration file for the DNS server. If two copies of the script run at the same time, it is important that only one of them does modifications. The second copy can run after the first is totally complete.

Continue reading “Linux-Fu: One At A Time, Please! Critical Sections In Bash Scripts” →

Linux Fu: Remote Execution Made Easy

August 11, 2020 by 20 Comments

If you have SSH and a few other tools set up, it is pretty easy to log into another machine and run a few programs. This could be handy when you are using a machine that might not have a lot of memory or processing power and you have access to a bigger machine somewhere on the network. For example, suppose you want to reencode some video on a box you use as a media server but it would go much faster on your giant server with a dozen cores and 32 GB of RAM.

Remote Execution

However, there are a few problems with that scenario. First, you might not have the software on the remote machine. Even if you do, it might not be the version you expect or have all the same configuration as your local copy. Then there’s the file problem. the input file should come from your local file system and you’d like the output to wind up there, too. These aren’t insurmountable, of course. You could install the program on the remote box and copy your files back and forth manually. Or you can use Outrun.

There are a few limitations, though. You do need Outrun on both machines and both machines have to have the same CPU architecture. Sadly, that means you can’t use this to easily run jobs on your x86-64 PC from a Raspberry Pi. You’ll need root access to the remote machine, too. The system also depends on having the FUSE file system libraries set up.

Continue reading “Linux Fu: Remote Execution Made Easy” →

Linux-Fu: Help Messages For Shell Scripts And Here Documents

August 3, 2020 by 20 Comments

Imagine that you want to output multiple lines of text in Bash, or any shell script. Maybe it’s for a help string for a particularly convoluted shell script you’re writing. You could have a separate echo command for each line.  Or you could use the “here document“.

The “here document” construction takes the text between two delimiters and passes it, as if it were piped, to a command.

if [[$# == 0 ]] || [[ "$1" == "-h" ]]; then
cat << EOF
This is my help message. There are many like it but this one is mine.
My help message is my best friend.
EOF

All of the text, as written, with line breaks and spaces and all, get passed to cat and your helpful formatted message is printed to the user.

Continue reading “Linux-Fu: Help Messages For Shell Scripts And Here Documents” →

Linux Fu: Keep In Sync

July 23, 2020 by 28 Comments

Once upon a time, computers were very expensive and you were lucky to have shared access to one computer. While that might seem to be a problem, it did have one big advantage: all of your files were on that computer.

Today, we all probably have at least a desktop and one laptop. Your phone is probably a pretty good computer by most standards. You might have multiple computers and a smattering of tablets. So what do you do to keep your files accessible everywhere? Why not run your own peer-to-peer synchronization service? Your files are always under your control and encrypted in motion. There’s no central point of failure. You can do it with one very slick piece of Open Source software called syncthing. It runs on Windows, Linux, Mac, BSD, and Solaris. There are also Android clients. We haven’t tested it, but one caveat is that the unofficial iOS support sounds a little spotty.

The joke about the cloud — that it’s just other people’s servers — is on point here. Some people don’t like their files sitting on a third-party server. Even if your files are encrypted or you don’t care, you still have the problem of what happens if you can’t reach the server — may be on an airplane with no WiFi — or the server goes down. Sure, Google and Microsoft don’t go dark very often, but they can and do. Even if you build your own cloud, it runs on your servers. Syncthing is serverless: it simply makes sure that all files are up-to-date on all your end devices. Continue reading “Linux Fu: Keep In Sync” →

Linux-Fu: Parallel Universe

June 29, 2020 by 38 Comments

At some point, you simply run out of processing power. Admittedly, that point keeps getting further and further away, but you can still get there. If you run out of CPU time, the answer might be to add more CPUs. However, sometimes there are other bottlenecks like memory or disk space. However, it is also likely that you have access to multiple computers. Who doesn’t have a few Raspberry Pis sitting around their network? Or maybe a server in the basement? Or even some remote servers “in the cloud.” GNU Parallel is a tool that lets you spread work across multiple tasks either locally to remote machines. In some ways, it is simple, since it looks sort of like xargs but with parallel execution. On the other hand, it has myriad options and configurations that can make it a little daunting to use. Continue reading “Linux-Fu: Parallel Universe” →

Linux-Fu: Automation For Chrome And The Desktop By Matching Screenshots

June 15, 2020 by 11 Comments

I will be the first to admit it. This is almost not — at least not specifically — a Linux article. The subject? An automation tool for Chrome or Firefox. But before you hit the back button, hear me out. Sure, this Chrome plugin started out as a tool to automatically test web pages and automate repetitive tasks in the browser. However, it can extend that power to all programs on your computer. So, in theory, you can use it to graphically build macros that can interact with desktop applications in surprisingly sophisticated ways. In theory, anyway; there are a few problems.

The program has a few different names. Most documentation says UI Vision RPA, although there are some references to Kantu, which appears to be an older name. RPA is an acronym for Robotic Process Automation, which is an industry buzz word.

Let’s take it for a spin and see what it’s all about.

Continue reading “Linux-Fu: Automation For Chrome And The Desktop By Matching Screenshots” →