Automate Your Life With Node-RED (Plus A Dash Of MQTT)

January 15, 2020 by Mike Szczys 54 Comments

For years we’ve seen a trickle of really interesting home automation projects that use the Node-RED package. Each time, the hackers behind these projects have raved about Node-RED and now I’ve joined those ranks as well.

This graphic-based coding platform lets you quickly put together useful operations and graphic user interfaces (GUIs), whether you’re the freshest greenhorn or a seasoned veteran. You can use it to switch your internet-connected lights on schedule, or at the touch of a button through a web-app available to any device on your home network. You can use it as an information dashboard for the weather forecast, latest Hackaday articles, bus schedules, or all of them at once. At a glance it abstracts away the complexity of writing Javascript, while also making it simple to dive under hood and use your 1337 haxor skills to add your own code.

You can get this up and running in less than an hour and I’m going to tackle that as well as examples for playing with MQTT, setting up a web GUI, and writing to log files. To make Node-RED persistent on your network you need a server, but it’s lean enough to run from a Raspberry Pi without issue, and it’s even installed by default in BeagleBone distributions. Code for all examples in this guide can be found in the tutorial repository. Let’s dive in!

Continue reading “Automate Your Life With Node-RED (Plus A Dash Of MQTT)” →

A Vintage Phone In 2020

January 14, 2020 by Jenny List 72 Comments

When we make a telephone call in 2020 it is most likely to be made using a smartphone over a cellular or IP-based connection rather than a traditional instrument on a pair of copper wires to an exchange. As we move inexorably towards a wireless world in which the telephone line serves only as a vehicle for broadband Internet, it’s easy to forget the last hundred years or more of telephone technology that led up to the present.

The iconic British telephone of the 1960s and 1970s, the GPO model 746. Mine is from 1971. (That isn’t my phone number)

In a manner of speaking though, your telephone wall socket hasn’t forgotten. If you like old phones, you can still have one, and picture yourself in a 1950s movie as you twirl the handset cord round your finger while you speak. Continue reading “A Vintage Phone In 2020” →

Linux Fu: Leaning Down With Exec

December 30, 2019 by Al Williams 11 Comments

Shell scripting is handy and with a shell like bash it is very capable, too. However, shell scripting isn’t always very efficient. Think about it. If you run grep or tr or sort to do some operation in a shell script, you are spawning a whole new process. That takes time and resources. But there are some answers to reducing — but not eliminating — the problem.

Have you ever written a program like this (in any language, but I’ll use C):

int foo(void)
{
  ...
  bar();

}

You hope the compiler doesn’t write assembly code like this:

_foo: 
....

      call _bar
      ret

Most optimizers should pick up on the fact that you can convert a call like this to a jump and let the ret statement in _bar return to foo’s caller. However, shell scripts are not that smart. If you have a shell script called MungeData and it calls another program or shell script called PostProcess on its last line, then you will have at one time three processes in play: your original shell, the shell running MungeData, and either the PostProcess program or a shell running the script. Not to mention, the processes to do things inside post process. So what do you do?

Continue reading “Linux Fu: Leaning Down With Exec” →

Linux Fu: Stupid SSH Tricks

December 17, 2019 by Al Williams 52 Comments

If you connect to remote computers over the Internet, it is a pretty good chance you use some form of SSH or secure shell. On Linux or Unix you’ll use the ssh command. Same goes for Linux-like environments on Windows like Cygwin or WSL. For native Windows, you might be using Putty. In its simplest form, ssh is just a terminal program that talks to a server using an encrypted connection. We think it is very hard to eavesdrop on anyone communicating with a remote computer via ssh.

There are several tricks for using ssh — some are pretty straightforward and some are things you might not think of as being in the domain of a terminal program. You probably know that ssh can copy files securely, and there are easy and hard ways to set up logging in with no password.

However, you can also mount a remote filesystem via ssh (actually, there are several ways to do that). You can use ssh to securely browse the web in your favorite browser, or even use it to tunnel specific traffic by port or even use it as a makeshift VPN. In fact, there’s so much ground to cover that this won’t be the last Linux Fu to talk about ssh. But enough setup, let’s get to the tricks.

Continue reading “Linux Fu: Stupid SSH Tricks” →

Linux Fu: Debugging Bash Scripts

December 11, 2019 by Al Williams 23 Comments

A recent post about debugging constructs surprised me. There were quite a few comments about how you didn’t need a debugger, as long as you had printf. For that matter, we’ve all debugged systems where you had nothing but an LED to flash or otherwise turn on to communicate with the user. However, it is hard to deny that a debugger can help with complex code.

To say you only need printf would be like saying you only need machine language. Technically accurate — you can do anything in machine language. But it sure makes things easier to have an assembler or some language to help you work out your problem. If you write a simple bash script, you can use the equivalent to printf — maybe that’s the echo command, although there is usually a printf command on a typical system, if you want to use it. However, there are other things you can do with bash including a pretty cool debugger if you know how to find it.

I assume you already know how to use echo and printf, but let’s dig into how to use trace execution line by line without the need for echo statements on every other line. Along the way, you’ll learn how to get started with the bash debugger.

Continue reading “Linux Fu: Debugging Bash Scripts” →

Tales From The Sysadmin: Dumped Into The Grub Command Line

November 25, 2019 by Jonathan Bennett 53 Comments

Today I have a tale of mystery, of horror, and of hope. The allure of a newer kernel and packages was too much to resist, so I found myself upgrading to Fedora 30. All the packages had downloaded, all that was left was to let DNF reboot the machine and install all the new packages. I started the process and meandered off to find a cup of coffee: black, and darker than the stain this line of work leaves on the soul. After enough time had elapsed, I returned, expecting the warming light of a newly upgraded desktop. Instead, all that greeted me was the harsh darkness of a grub command line. Something was amiss, and it was bad.

(An aside to the reader, I had this experience on two different machines, stemming from two different root problems. One was a wayward setting, and the other an unusual permissions problem.)

How does the fledgling Linux sysadmin recover from such a problem? The grub command line is an inscrutable mystery to the uninitiated, but once you understand the basics, it’s not terribly difficult to boot your system and try to restore the normal boot process. This depends on what has broken, of course. If the disk containing your root partition has crashed, then sorry, this article won’t help.

In order to get a system booting, what exactly needs to happen? How does booting Linux work, even? Two components need to be loaded into memory: the kernel, and the initramfs. Once these two elements are loaded into memory, grub performs a jump into the kernel code, which takes over and finishes the machine’s boot. There is one more important detail that we care about — the kernel needs to know where to find the root partition. This is typically part of the kernel parameters, specified on the kernel boot line.

When working with an unfamiliar shell, the help command is a good starting point. grub runs in a very limited environment, and running the help command scrolls most of the text off the screen. There is an environment variable that helps out here, enabling output paging:set pager=1.
Continue reading “Tales From The Sysadmin: Dumped Into The Grub Command Line” →

Training Bats In The Random Forest With The Confusion Matrix

November 21, 2019 by Pat Whetman 4 Comments

When exploring the realm of Machine Learning, it’s always nice to have some real and interesting data to work with. That’s where the bats come in – they’re fascinating animals that emit very particular ultrasonic calls that can be recorded and analysed with computer software to get a fairly good idea of what species they are. When analysed with an FFT spectogram, we can see the individual call shapes very clearly.

Nyctalus noctula (noctule bat)

Myotis nattereri (natterera’s bat)

Plecotus auritus (brown long eared)

Pipistrellus pipistrellus (common pipistrelle)

Creating an open source classifier for bats is also potentially useful for the world outside of Machine Learning as it could not only enable us to more easily monitor bats themselves, but also the knock on effects of modern farming methods on the natural environment. Bats feed on moths and other night flying insects which themselves have been decimated in numbers. Even in the depths of the countryside here in the UK these insects are a fraction of the population that they used to be 30 years ago, but nobody seems to have monitored this decline.

So getting back to our spectograms, it would be perfectly reasonable to throw these images at a convolutional neural network (CNN) and use an image feature-recognition strategy. But I wanted to explore the depths of the mysterious Random Forest. Continue reading “Training Bats In The Random Forest With The Confusion Matrix” →