Linux Fu: Tracing System Calls

April 7, 2020 by Al Williams 11 Comments

One of the nice things about Linux and similar operating systems is that you can investigate something to any level you wish. If a program has a problem you can decompile it, debug it, trace it, and — if necessary — even dig into the source code for the kernel and most of the libraries the program is probably using. However, the tools to do this aren’t ones you use every day. One very interesting tool is strace. Using it you can see what system calls any program makes and that can sometimes give you important clues about how the program works or, probably more often, why it doesn’t work.

Let’s consider the least complex use of the command. Suppose you want to make symlink from testxmit.grc to the /tmp directory. That command is simple:

ln -sf testxmit.grc /tmp

But if you tell strace to run it, the command becomes:

strace ln -sf testxmit.grc /tmp

You might want to redirect the output to a file using the shell or the -o option, though. Some commands generate a lot and often the first page or two of output isn’t really what you care about anyway. Continue reading “Linux Fu: Tracing System Calls” →

Linux Fu: Mapping Files

March 20, 2020 by Al Williams 7 Comments

If you use C or C++, you have probably learned how to open a file and read data from it. Usually, we read a character or a line at a time. At least, it seems that way. The reality is there are usually quite a number of buffers between you and the hard drive, so your request for a character might trigger a read for 2,048 characters and then your subsequent calls return from the buffer. There may even be layers of buffers feeding buffers.

A modern computer can do so much better than reading using things using old calls like fgetc. Given that your program has a huge virtual address space and that your computer has a perfectly good memory management unit within it, you can ask the operating system to simply map the file into your memory space. Then you can treat it like any other array of characters and let the OS do the rest.

The operating system doesn’t necessarily read the entire file in at one time, it just reserves space for you. Any time you hit a page that isn’t in memory, the operating system grabs it for you invisibly. Pages that you don’t use very often may be discarded and reloaded later. Behind the scenes, the OS does a lot so you can work on very large files with no real effort. The call that does it all is mmap.

Continue reading “Linux Fu: Mapping Files” →

Linux Fu: The Linux Shuffle

February 3, 2020 by Al Williams 35 Comments

Computers are known to be precise and — usually — repeatable. That’s why it is so hard to get something that seems random out of them. Yet random things are great for games, encryption, and multimedia. Who wants the same order of a playlist or slide show every time?

It is very hard to get truly random numbers, but for a lot of cases, it isn’t that important. Even better, if you programming or using a scripting language, there are lots of things that you can use to get some degree of randomness that is sufficient for many purposes. Continue reading “Linux Fu: The Linux Shuffle” →

Understand Linux Htop Visually

January 30, 2020 by Al Williams 33 Comments

If you want to know exactly what’s going on in your Linux system, some of you might reach for top. For the connoisseur of system monitors, nothing less than htop will do. Not familiar with htop? [Umer Mansoor] did a beautiful job of explaining it graphically.

We’ve mentioned htop in a previous Linux Fu, but we’ve never gotten a chance to dig into it. And now, we don’t have to. Like top, the htop program is still text-based, but it has a much nicer interface with colors, and easier way to send signals to processes, and support for tree displays. You can even use the mouse with it if you want to.

[Umer] did a lot of work to take screenshots of htop at work and annotate them. Sure, you could read the man page, but we think this is a lot better.

Of course, there are other improvements to top. Glances is pretty interesting, for example. For serious system administration help, you can try Webmin or Cockpit.

Maze Solving Via Text Editing

January 12, 2020 by Al Williams 18 Comments

Linux scripters usually know about sed — the stream editor. It has a simple job: transform text as it whizzes from input to output. So if you wanted to solve a maze, this wouldn’t be the tool you’d think to use, right? Well, if you were [xsot], you’d disagree.

You build a maze using spaces for empty space and # for walls. There’s an S to mark the start position and an E to mark the end. Of course, the maze can also contain newlines. The sed script does an amazing job of solving the problem.

Continue reading “Maze Solving Via Text Editing” →

Linux Fu: Leaning Down With Exec

December 30, 2019 by Al Williams 11 Comments

Shell scripting is handy and with a shell like bash it is very capable, too. However, shell scripting isn’t always very efficient. Think about it. If you run grep or tr or sort to do some operation in a shell script, you are spawning a whole new process. That takes time and resources. But there are some answers to reducing — but not eliminating — the problem.

Have you ever written a program like this (in any language, but I’ll use C):

int foo(void)
{
  ...
  bar();

}

You hope the compiler doesn’t write assembly code like this:

_foo: 
....

      call _bar
      ret

Most optimizers should pick up on the fact that you can convert a call like this to a jump and let the ret statement in _bar return to foo’s caller. However, shell scripts are not that smart. If you have a shell script called MungeData and it calls another program or shell script called PostProcess on its last line, then you will have at one time three processes in play: your original shell, the shell running MungeData, and either the PostProcess program or a shell running the script. Not to mention, the processes to do things inside post process. So what do you do?

Continue reading “Linux Fu: Leaning Down With Exec” →

VGA Signal In A Browser Window, Thanks To Reverse Engineering

December 30, 2019 by Donald Papp 25 Comments

[Ben Cox] found some interesting USB devices on eBay. The Epiphan VGA2USB LR accepts VGA video on one end and presents it as a USB webcam-like video signal on the other. Never have to haul a VGA monitor out again? Sounds good to us! The devices are old and abandoned hardware, but they do claim Linux support, so one BUY button mash later and [Ben] was waiting patiently for them in the mail.

But when they did arrive, the devices didn’t enumerate as a USB UVC video device as expected. The vendor has a custom driver, support for which ended in Linux 4.9 — meaning none of [Ben]’s machines would run it. By now [Ben] was curious about how all this worked and began digging, aiming to create a userspace driver for the device. He was successful, and with his usual detail [Ben] explains not only the process he followed to troubleshoot the problem but also how these devices (and his driver) work. Skip to the end of the project page for the summary, but the whole thing is worth a read.

The resulting driver is not optimized, but will do about 7 fps. [Ben] even rigged up a small web server inside the driver to present a simple interface for the video in a pinch. It can even record its output to a video file, which is awfully handy. The code is available on his GitHub repository, so give it a look and maybe head to eBay for a bit of bargain-hunting of your own.