Linux Fu: Making Progress

May 24, 2023 by Al Williams 12 Comments

The computer world looks different from behind a TeleType or other hardcopy terminal. Things that tend to annoy people about Unix or Linux these days were perfectly great when you were printing everything the computer said to you. Consider the brevity of most basic commands. When you copy a file, for example, it doesn’t really tell you much other than it returns you to the prompt when it is done. If you are on a modern computer working with normal-sized files locally, not a big deal. But if you are over a slow network or with huge files, it would be nice to have a progress bar. Sure, you could write your own version of copy, but wouldn’t it be nice to have some more generic options?

One Way

The pv program can do some of the things you want. It monitors data through a pipe or, at least through its standard output. Think of it as cat with a meter. Suppose you want to write a diskimage to /dev/sdz:

cat diskz.img >/dev/sdz

But you could also do:

pv diskz.img >/dev/sdz

By default, pv will show a progress bar, an elapsed time, an estimated end time, a rate, and a total number of bytes. You can turn any of that off or add things using command line options. You can also specify things like the size of the terminal if it should count lines instead of bytes, and, in the case where the program doesn’t know what it is reading, the expected size of the transfer.

Continue reading “Linux Fu: Making Progress” →

Linux Fu: C On Jupyter

May 11, 2023 by Al Williams 22 Comments

If you are a Pythonista or a data scientist, you’ve probably used Jupyter. If you haven’t, it is an interesting way to work with Python by placing it in a Markdown document in a web browser. Part spreadsheet, part web page, part Python program, you create notebooks that can contain data, programs, graphics, and widgets. You can run it locally and attach to it via a local port with a browser or, of course, run it in the cloud if you like. But you don’t have to use Python.

You can, however, use things with Jupyter other than Python with varying degrees of success. If you are brave enough, you can use C. And if you look at this list, you’ll see you can use things ranging from Javascript, APL, Fortran, Bash, Rust, Smalltalk, and even MicroPython.

Continue reading “Linux Fu: C On Jupyter” →

Linux Fu: Supercharge Bash History

May 5, 2023 by Al Williams 18 Comments

Having a history of shell commands is a great idea. It is, of course, enormously handy when you have to run something repetitively or you make a simple mistake that needs correction. However, as I’ve mentioned in the past, bash history isn’t without its problems. For one thing, by default, you don’t get history in one window from typing in another window. If you use a terminal multiplexer or a GUI, you are very likely to have many shells open. You can make them share history, but that comes with its own baggage. If you think about it, we have super fast computers with tons of storage compared to the “old days,” yet shell history is pretty much the same as it has been for decades. But [Rcaloras] did think about it and created Bashhub, a history database for bash, zsh, and probably some other shells, too.

You might think you don’t need anything more than what you have, and, of course, you don’t. However, Bashhub offers privately stored and encrypted history across machines. It also provides context about commands you’ve executed in the past. In other words, you can see the directory you were in, the exact time and date, the system you were on, and the last return code of the command.

Continue reading “Linux Fu: Supercharge Bash History” →

Run Linux By Emulating RISC-V On A RISC-V Microcontroller

April 23, 2023 by Jenny List 7 Comments

For years it was a given that it was impossible to run a Linux based operating system on a less powerful computer whose architecture lacked a memory management unit. There were projects such as uCLinux which sought to provide some tidbits to low computing power Linux users, but ultimately they came to naught. It is achievable after a fashion though, by using the limited architecture to emulate a more powerful one. It’s been done on AVR chips emulating ARM, on ARM chips, and now someone’s done it on an ESP32-C3 microcontroller, a RISC-V part running a RISC-V emulator. What’s going on?

RISC-V is an architecture specification that can be implemented at many levels from a simple microcontroller or even a pile of 74 logic to a full-fat application processor. The ESP32-C3 lies towards the less complicated end of this curve, though that’s not the whole reason for the emulation. The PSRAM storage is used by the C3 as data storage and can’t be used to run software, so to access all that memory capacity an emulator is required that in turn can use the PSRAM as its program memory. It’s a necessary trick for Espressif’s implementation of the architecture.

Surprisingly it’s not as slow as might be expected, with a boot-up time under two minutes. It’s not what we’d expect from our desktop powerhouses, but it’s not so long ago that certain lower-power full-fat processors could be just as lethargic. For past glories, see the AVR running Linux, and the RP2040.

Build Your Own Bootable Emacs Environment

April 22, 2023 by Bryan Cockfield 18 Comments

An old joke is that Emacs is a text editor with an operating system included, given that its extensibility and customization often goes far beyond traditional text editors. Part of its well-earned reputation comes from being built in Lisp which allows it to be expanded to do almost anything. Despite this in-joke in the community, though, you will still need an actual operating system to run it, but not much more than that.

This project uses User-Mode Linux (UML) as a foundation to load almost nothing other than an Emacs editor. UML is a virtualization technology that allows running multiple Linux kernel instances as separate virtual machines, so once the Linux environment is started and Emacs is compiled, the virtual machine can essentially boot straight into an Emacs environment. Some tools are needed outside of the Linux kernel like mount which allows the virtual file system to access the files needed to build Emacs, but as far as lightweight or minimalist Linux distributions go this one definitely gets at least an honorable mention.

While UML is virtualization software rather than a full-fledged Linux distribution, we would expect a similarly minimalist build could easily be done with something more hardware-based like Linux From Scratch. Emacs has been around for so long and had such a wide reach that it’s difficult to imagine a world without it. Even in more modern technology like browsers, knowing a little bit about Emacs can be an extremely powerful tool.

Before You Sudo Rm -rf /, Take Some Precautions

April 21, 2023 by Bryan Cockfield 40 Comments

Maintaining or administering a computer system remotely is a common enough task these days, but it’s also something that can go sideways on you quickly if you aren’t careful. How many of us are guilty of executing a command, having it fail, and only then realizing that we weren’t connected to the correct computer at all? [Callan] occasionally has this issue as well, but in at least one instance, he deleted all of the contents of the wrong server by mistake. To avoid that mistake again, he uses color codes in the command line in a fairly unique way.

The solution at first seems straightforward enough. Since the terminal he’s using allows for different colors to be displayed for the user and hostname on the bash prompt, different text and background colors are used for each server. The only problem with this is that his friends also have access to these servers, and one of them is red/green colorblind, which led to another near-catastrophic mix-up. To ensure no edge cases are missed, [Callan] built a script which runs on every new server he spins up which selects two random colors, checks that they contrast well with each other, don’t create problems for the colorblind, and then applies them to the bash prompt.

For a problem most of us have had at some point or another, it’s a fairly elegant solution that helps ensure we’re sending the right commands to the right computer. This adds a layer of automation to the process and, while some color combinations do look similar, there are enough to help out most of us in some way, especially since he has released the source code on his GitHub page. For other helpful server administration tips, we’d recommend the Linux-Fu article about deploying your own dynamic DNS.

Linux Server, Wakey, Wakey

April 21, 2023 by Al Williams 39 Comments

We all know we should save energy and not leave computers on all the time. It is probably better for the computer, too. But when you operate a home server, it isn’t feasible to just turn it on when you want to use it and then turn it off again. Or is it? [Daniel] decided that was exactly what he wanted to do, and it was quite an adventure to get there.

The trick is to use a Raspberry Pi — they don’t draw nearly the power a big computer does — to stay awake to facilitate the process. The Pi watches for ARP requests for the sleeping machine and replies on its behalf so that other network nodes can find the machine even when it isn’t on.

The server itself detects if it is idle in a cron job. When it finds that there are no SSH or other service connections for a set period of time, it suspends the machine to RAM, putting it in a low-power mode. Waking a sleeping computer up over the network is a solved problem, and [Daniel] investigated several wake-on-lan solutions.

There were several oddities to work out, including a Mac pinging an unused network share, and a router that was making NetBIOS queries. However, [Daniel] found a $30 router that could do port mirroring and that helped a lot with troubleshooting.

This is one of those things where his recipe won’t exactly fit your situation. But the post has a lot of good information and some nice tricks for troubleshooting any kind of network bizarreness.

Wireshark is a great tool for this kind of work, too. Another useful technique is recording network traffic and playing it back.