Linux Fu: Databases Are Next-Level File Systems

It is funny how exotic computer technology eventually either fails or becomes commonplace. At one time, having more than one user on a computer at once was high tech, for example. Then there are things that didn’t catch on widely like vector display or content-addressable memory. The use of mass storage — especially disk drives — in computers, though has become very widespread. But at one time it was an exotic technique and wasn’t nearly as simple as it is today.

However, I’m surprised that the filesystem as we know it hasn’t changed much over the years. Sure, compared to, say, the 1960s we have a lot better functionality. And we have lots of improvements surrounding speed, encoding, encryption, compression, and so on. But the fundamental nature of how we store and access files in computer programs is stagnant. But it doesn’t have to be. We know of better ways to organize data, but for some reason, most of us don’t use them in our programs. Turns out, though, it is reasonably simple and I’m going to show you how with a toy application that might be the start of a database for the electronic components in my lab.

You could store a database like this in a comma-delimited file or using something like JSON. But I’m going to use a full-featured SQLite database to avoid having a heavy-weight database server and all the pain that entails. Is it going to replace the database behind the airline reservation system? No. But will it work for most of what you are likely to do? You bet. Continue reading “Linux Fu: Databases Are Next-Level File Systems”

Branch Out Your SQLite Database With LiteTree

Whether you want some quick and dirty data storage, or simply don’t have that heavy requirements for your local database system, SQLite is always a good choice. With its portable single-file approach, bindings to all major languages, and availability on systems of all sizes, it is relatively easy to integrate a SQLite database in your undertakings. And if you tend to develop directly in your production environment, you may be interested to hear that the folks at [aergo] made this a lot more flexible (and interesting) by adding Git-style branching to the SQLite engine.

Similar to Git, each database operation is now stored as a commit with a unique id as reference point, and new branches will keep track how they diverge from their parent reference point. This essentially lets you modify your data set or database schema on the fly, while keeping your original data not only untouched, but fully isolated and functional. Unfortunately, merging branches is not yet supported, but it is planned for the near future.

In case you don’t see much use for git-alike functionality in a database, how about the other way around then: using Git as a database, among other tricks?

Stupid Git Tricks

My apologies if you speak the Queen’s English since that title probably has a whole different meaning to you than I intended. In fact, I’m talking about Git, the version control system. Last time I talked about how the program came to be and offered you a few tutorials. If you are a dyed-in-the-wool software developer, you probably don’t need to be convinced to use Git. But even if you aren’t, there are a lot of things you can do with Git that don’t fit the usual mold.

Continue reading “Stupid Git Tricks”