File Compression By Steganography

October 19, 2019 by Sharon Lin 29 Comments

In a world with finite storage and an infinite need for more storage space, data compression becomes a very necessary problem. Several algorithms for data compression may be more familiar – Huffman coding, LZW compression – and some a bit more arcane.

[Labunsky] decided to put to use his knowledge of steganography to create a wholly unique form of file compression, perhaps one that may gain greater notoriety among other information theorists.

Steganography refers to the method of concealing messages or files within another file, coming from the Greek words steganos for “covered or concealed” and graphe for “writing”. The practice has been around for ages, from writing in invisible ink to storing messages in moon cakes. The methods used range from hiding messages in images to evade censorship to hiding viruses in files to cause mayhem.

The developer explains that since every file is just a bit sequence, observing files leads to the realization that a majority of bits will be equal on the same places. Rather than storing all of the bits of a file, making modifications to the hard drive at certain locations can save storage space. What is important to avoid, however, is lossy file compression that can wreak havoc on quality during the compression stage.

The compression technique they ended up implementing is based on the F5 algorithm that embeds binary data into JPEG files to reduce total space in the memory. The compression uses libjpeg for JPEG decoding and encoding, pcre for POSIX regular expressions support, and tinydir for platform-independent filesystem traversal. One of the major modifications was to save computation resources by disabling a password-based permutative straddling that uniformly spreads data among multiple files.

One caveat – changing even one bit of the compressed file could lead to total corruption of all of the data stored, so use with caution!

Pack Your Bags – Systemd Is Taking You To A New Home

October 16, 2019 by Sven Gregori 177 Comments

Home directories have been a fundamental part on any Unixy system since day one. They’re such a basic element, we usually don’t give them much thought. And why would we? From a low level point of view, whatever location $HOME is pointing to, is a directory just like any other of the countless ones you will find on the system — apart from maybe being located on its own disk partition. Home directories are so unspectacular in their nature, it wouldn’t usually cross anyone’s mind to even consider to change anything about them. And then there’s Lennart Poettering.

In case you’re not familiar with the name, he is the main developer behind the systemd init system, which has nowadays been adopted by the majority of Linux distributions as replacement for its oldschool, Unix-style init-system predecessors, essentially changing everything we knew about the system boot process. Not only did this change personally insult every single Perl-loving, Ken-Thompson-action-figure-owning grey beard, it engendered contempt towards systemd and Lennart himself that approaches Nickelback level. At this point, it probably doesn’t matter anymore what he does next, haters gonna hate. So who better than him to disrupt everything we know about home directories? Where you _live_?

Although, home directories are just one part of the equation that his latest creation — the systemd-homed project — is going to ~~make people hate him even more~~ tackle. The big picture is really more about the whole concept of user management as we know it, which sounds bold and scary, but which in its current state is also a lot more flawed than we might realize. So let’s have a look at what it’s all about, the motivation behind homed, the problems it’s going to both solve and raise, and how it’s maybe time to leave some outdated philosophies behind us.

Continue reading “Pack Your Bags – Systemd Is Taking You To A New Home” →

Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly

October 14, 2019 by Al Williams 7 Comments

Not every programmer likes creating GUI code. Most hacker types don’t mind a command line interface, but very few ordinary users appreciate them. However, if you write command line programs in Python, Gooey can help. By leveraging some Python features and a common Python idiom, you can convert a command line program into a GUI with very little effort.

The idea is pretty simple. Nearly all command line Python programs use argparse to simplify picking options and arguments off the command line as well as providing some help. The Gooey decorator picks up all your options and arguments and creates a GUI for it. You can make it more complicated if you want to change specific things, but if you are happy with the defaults, there’s not much else to it.

At first, this article might seem like a Python Fu and not a Linux Fu, since — at first — we are going to focus on Python. But just stand by and you’ll see how this can do a lot of things on many operating systems, including Linux.

Continue reading “Linux Fu: Python GUIs For Command Line Programs (Almost) Instantly” →

WiringPi Library To Be Deprecated

September 18, 2019 by Maya Posch 60 Comments

Since the release of the original Raspberry Pi single board computer, the WiringPi library by [Gordon] has been the easy way to interface with the GPIO and peripherals – such as I2C and SPI – on the Broadcom SoCs which power these platforms. Unfortunately, [Gordon] is now deprecating the library, choosing to move on rather than deal with a community which he no longer recognizes.

Among the points which he lists are the (commercial) abuse of his code, and the increasing amount of emails and messages on social media from folk who either failed to read the friendly manual, or are simply rude and inconsiderate. As [Gordon] puts it, WiringPi was never meant to be statically linked into code, nor to be used with anything other than C and RTB BASIC programmers. He never supported the use of the library with other languages, or having it statically integrated into some Java/JavaScript/NodeJS project.

As this secondary use is what’s draining the fun out of the project, he has decided to put out one final release, before making it a closed-source project, for use by himself and presumably paying clients. What the impact of this will be has to be seen. Perhaps a new fork will become the new ‘WiringPi’?

Suffice it to say, none of this is a good thing. The illegal use of open source code and the support nightmare that gets poured on the authors of said code by less than informed users is enough to drive anyone away from putting their projects out there. Fighting abuse and junking the ‘spam’ is one way to deal with it, but who has the time and energy (and money) for this?

What are your thoughts on this news, and this issue in general? How should an open source developer deal with it?

Thanks to [Dirk-Jan Faber] for sending this one in.

What’s In A Name? Tales Of Python, Perl, And The GIMP

September 18, 2019 by Sven Gregori 58 Comments

In the older days of open source software, major projects tended to have their Benevolent Dictators For Life who made all the final decisions, and some mature projects still operate that way. Guido van Rossum famously called his language “Python” because he liked the British comics of the same name. That’s the sort of thing that only a single developer can get away with.

However, in these modern times of GitHub, GitLab, and other collaboration platforms, community-driven decision making has become a more and more common phenomenon, shifting software development towards democracy. People begin to think of themselves as “Python programmers” or “GIMP users” and the name of the project fuses irrevocably with their identity.

What happens when software projects fork, develop apart, or otherwise change significantly? Obviously, to prevent confusion, they get a new name, and all of those “Perl Monks” need to become “Raku Monks”. Needless to say, what should be a trivial detail — what we’ve all decided to call this pile of ones and zeros or language constructs — can become a big deal. Don’t believe us? Here are the stories of renaming Python, Perl, and the GIMP.

Continue reading “What’s In A Name? Tales Of Python, Perl, And The GIMP” →

Peek Into The Compiler’s Code — Lots Of Compilers

September 13, 2019 by Al Williams 12 Comments

We don’t know what normal people argue about, but we know we spend a lot of time arguing about the best microcontroller, which editor is the best, and what language or compiler does the best job. The problem with all those compilers is getting them loaded and digging into the generated code. If you too spend your time thinking about those things, you ought to have a look at [Matt Godbolt’s] Compiler Explorer. We know that hosting an IDE-like web page and compiling code is old hat — although [Matt’s] site has been around quite some time. But [Matt’s] doing it differently. The code you build on the left hand pane shows up as assembly language on the right hand side.

There are plenty of options, too. For example, here’s a bit of C code from the site’s example:

int square(int num) {
   return num * num;
}

Here’s the corresponding assembly from gcc 9.2 for x86-64:

square:
  push rbp
  mov rbp, rsp
  mov DWORD PTR [rbp-4], edi
  mov eax, DWORD PTR [rbp-4]
  imul eax, eax
  pop rbp
  ret

Continue reading “Peek Into The Compiler’s Code — Lots Of Compilers” →

Why Ada Is The Language You Want To Be Programming Your Systems With

September 10, 2019 by Maya Posch 168 Comments

The Ada programming language was born in the mid-1970s, when the US Department of Defense (DoD) and the UK’s Ministry Of Defence sought to replace the hundreds of specialized programming languages used for the embedded computer systems that increasingly made up essential parts of military projects. Instead, Ada was designed to be be a single language, capable of running on all of those embedded systems, that offered the same or better level of performance and reliability.

With the 1995 revision, the language also targeted general purpose systems and added support for object-oriented programming (OOP) while not losing sight of the core values of reliability, maintainability and efficiency. Today, software written in Ada forms the backbone of not only military hardware, but also commercial projects like avionics and air-traffic control systems. Ada code controls rockets like the Ariane 4 and 5, many satellites, and countless other systems where small glitches can have major consequences.

Ada might also be the right choice for your next embedded project. Continue reading “Why Ada Is The Language You Want To Be Programming Your Systems With” →