Go

19 Articles

Blowing Up Shell Scripts

September 18, 2024 by 24 Comments

One of the most universal experiences of any Linux or Unix user is working through a guide or handbook and coming across an almost unbelievably complex line of code meant to be executed with a shell. At the time of encountering a snippet like this it’s difficult to imagine any human ever having written it in the first place, but with some dedication it is possible to tease out what these small bits of code do when they’re typed into the terminal and run (unless it’s something like :(){ :|:& };: but that’s another story entirely). [noperator] recently built a tool which helps users in this predicament understand these shell scripts by expanding them into a more human-intelligible form.

The tool is named sol and does much more than expanding shell one-liners into a readable format. It also provides an interactive shell environment where the user can explore the exploded code in detail, modify it in any way they see fit, and collapse it back down to a single line so it can easily be sent to other users. It can be used with most of the major text editors as well as piped directly to standard input, and has a number of other options as well such as custom configurations and the ability to see non-standard bits of code that might not be compatible from one shell environment to another, as well as helping to translate those bits of code.

[noperator] has made the code available in the linked GitHub page for anyone curious about its use, and has a to-do list for future versions of the tool as well including adding support beyond bash. We’d definitely recommend a tool like this especially if you’re still relatively new to bash scripting (or shell scripting in general) and, as always, we’d just to remind everyone not to blindly copy and paste commands into their terminal windows. If you’re the type of person to go out on a limb and run crazy commands to see what they actually do, though, make sure you’re at least logged into the right computer first.

Hackaday Links Column Banner

Hackaday Links: March 10, 2024

March 10, 2024 by 17 Comments

We all know that we’re living in a surveillance state that would make Orwell himself shake his head, but it looks like at least one company in this space has gone a little rogue. According to reports, AI surveillance start-up Flock <<insert gratuitous “What the Flock?” joke here>> has installed at least 200 of its car-tracking cameras on public roads in South Carolina alone. That’s a serious whoopsie, especially since it’s illegal to install anything on state infrastructure without permission, which it appears Flock failed to obtain. South Carolina authorities are making a good show of being outraged about this, but it sort of rings hollow to us, especially since Flock now claims that 70% of the population (of the USA, we presume) is covered by their technology. Also, police departments across the country are in love with Flock’s service, which lets them accurately track the movements of potential suspects, which of course is everyone. No word on whether Flock will have to remove the rogue cameras, but we’re not holding our breath.

Continue reading “Hackaday Links: March 10, 2024” →

Machine Vision Automates Trainspotting With Unique Full-Length Portraits

May 2, 2023 by 10 Comments

As hobbies go, trainspotting is just as valid a choice as any — we don’t judge. But it does present certain logistical challenges, such as having to be in visual range of a train to be able to spot it. There’s also the fact that trains are very large objects, and they tend to move very fast. What’s a railfan to do?

If you’re also technically minded, you might try building an automatic trainspotting bot like [jo-m] has. It looks like the hardware end of “Trainbot” is pretty simple since it has been tested on both x86 and Raspberry Pi, and supports both video4linux and Pi cam. The magic is in the software, which is able to detect a train entering the frame, record images, and then stitch them together into one long image. The whole thing is coded in Go and has some interesting bits, like a custom image patch mapping package.

Trainbot gives an unusual view of a train, one that most of us accustomed to watching a train pass at a crossing have never seen. By stitching small chunks of the train as it passes, Trainbot is able to show the entire train in a single image, which would be impossible to do except for being very, very far away from the track. [jo-m] also built a web interface for Trainbot where you can check out the comings and goings yourself. Each passing train’s image is accompanied by data like its velocity and acceleration, length of the train, and time of passage. There’s also a GIF of the original source video, which is pretty cool.

Here in the States, we don’t have a lot of passenger trains to spot, but we do have some really long freight trains. It’d be interesting to see how this works with a train that’s over a mile long; that would be quite an image. Looks like someone at least has the hardware in place to give it a try.

Building Faster Rsync From Scratch In Go

June 1, 2022 by 53 Comments

For a quick file transfer between two computers, SCP is a fine program to use. For more complex, large, or regular backups, however, the go-to tool is rsync. It’s faster, more efficient, and usable in a wider range of circumstances. For all its perks, [Michael Stapelberg] felt that it had one major weakness: it is a tool written in C. [Michael] is philosophically opposed to programs written in C, so he set out to implement rsync from scratch in Go instead.

[Michael]’s path to deciding to tackle this project is a complicated one. His ISP upgraded his internet connection to 25 Gbit/s recently, which means that his custom router was the bottleneck in his network. To solve that problem he migrated his router to a PC with several 25 Gbit/s network cards. To take full advantage of the speed now theoretically available, he began using a tool called gokrazy, which turns applications written in Go into their own appliance. That means that instead of installing a full Linux distribution to handle specific tasks (like a router, for example), the only thing loaded on the computer is essentially the Linux kernel, the Go compiler and libraries, and then the Go application itself.

With a new router with hardware capable of supporting these fast speeds and only running software written in Go, the last step was finally to build rsync to support his tasks on his network. This meant that rsync itself needed to be built from scratch in Go. Once [Michael] completed this final task, he found that his implementation of rsync is actually much faster than the version built in C, thanks to the modernization found in the Go language and the fact that his router isn’t running all of the cruft associated with a standard Linux distribution.

For a software project of this scope, we find [Michael]’s step-by-step process worth taking note of for any problem any of us attempt to tackle. Not only that, refactoring a foundational tool like rsync is an involved task on its own, let alone its creation simply to increase network speeds beyond what most of us would already consider blazingly fast. We’re leaving out a ton of details on this build so we definitely recommend checking out his talk in the video below.

Thanks to [sarinkhan] for the tip!

Continue reading “Building Faster Rsync From Scratch In Go” →

TinyGo Brings Go To Arduino

September 4, 2019 by 25 Comments

Go — a modern programming language with roots at Google — is one of the new generation languages that would like to unseat C (and C++) for what we think of as traditional programming. It is only for PCs, though, right? Not so fast! TinyGo provides a compiler that — in their words — is for small places. How small? They can target code for the Arduino Uno or the BBC micro:bit. It can also produce code for x86 or ARM Linux (both 32- and 64-bit) as well as WebAssembly. They claim that a recent project to add ESP8266 and EPS32 support to LLVM will eventually enable TinyGo to target those platforms, too.

As you would expect, there are some subtle differences between TinyGo and the full-blown version. The compiler handles the entire program at once which is slower but offers more for optimization. Certain optimizations for interface methods are not used in TinyGo, and global variable handling changes to accommodate moving data from flash to RAM efficiently. TinyGo passes parameters in registers.

Continue reading “TinyGo Brings Go To Arduino” →

The V Programming Language: Vain Or Virtuous?

July 23, 2019 by 92 Comments

If you stay up to date with niche software news, your ears may recently have twitched at the release of a new programming language: V. New hobby-project programming languages are released all the time, you would correctly argue; what makes this one special? The answer is a number of design choices which promote speed and safety: V is tiny and very fast. It’s also in a self-proclaimed alpha state, and though it’s already been used to build some interesting projects, is still at an early stage.

Continue reading “The V Programming Language: Vain Or Virtuous?” →

Linux Fu: Pimp Your Pipes

November 7, 2018 by 17 Comments

One of the best things about working at the Linux (or similar OS) command line is the use of pipes. In simple terms, a pipe takes the output of one command and sends it to the input of another command. You can do a lot with a pipe, but sometimes it is hard to work out the right order for a set of pipes. A common trick is to attack it incrementally. That is, do one command and get it working with the right options and inputs. Then add another command until that works. Keep adding commands and tweaking until you get the final results.

That’s fine, but [akavel] wanted better and used Go to create “up” — an interactive viewer for pipelines.

Pipe Philosophy

Pipes can do a lot. They match in with the original Unix philosophy of making each tool do one thing really well. Pipe is really good at allowing Linux commands to talk to each other. If you want to learn all about pipes, have a look at the Linux Info project’s guide. They even talk about why MSDOS pipes were not really pipes at all. (One thing that write up doesn’t touch on is the named pipe. Do a “man fifo” if you want to learn more for now and perhaps that will be the subject of a future Linux Fu.)

This program — called up — continuously runs and reruns your pipeline as you make changes to the pipe. This way, every change you make is instantly reflected in the output. Here’s the video, here’s a quick video which shows off the interactive nature of up.

Installing

The GitHub page assumes you know how to install a go program. I tried doing a build but I didn’t have a few dependencies. Turns out the easy way to do it was to run this line:

go get -u github.com/akavel/up

This put the executable in ~/go/bin — which isn’t on my path. You can, of course, copy or link it to some directory that’s on your path or add that directory to your path. You could also set an alias, for example. Or, like I did in the video, just specify it every time.

Perfect?

This seems like a neat simple tool. What could be better? Well, I was a little sad that you can’t use emacs or vi edit keys on the pipeline, at least not as far as I could tell. This is exactly the kind of thing where you want to back up into the middle and change something. You can use the arrow keys, though, so that’s something. I also wished the scrollable window had a search feature like less.

Otherwise, though, there’s not much to dislike about the little tool. If writing a pipeline is like using a C compiler, up makes it more like writing an interactive Basic program.