A Shell? A Programming Language? Relax! It’s Both!

Every time we publish a Linux hack that uses a shell script, someone will chime in about how awful it is to program shell scripts. While we like the ubiquity and efficiency, we can’t disagree that the shell is a bit of a hack itself. [Axel Lijencrantz] wants to change your shell to be a full-blow programming language called Crush.

On the face of it, it looks like a shell. Want to see the contents of the current directory? Simple: ls.

The difference is underneath. In Crush, ls is a built-in and it returns data in rows like a database. You can manipulate that database with SQL-like commands: ls | where {type=="directory"}.

You can still treat I/O as binary streams. But Crush also knows about CSV files, JSON, line-oriented files, and several other formats. So if you were trying to feed the output of ls to a program and need it in JSON form, that’s simple: ls | json:to ./listing.json.

Crush can also do math without resorting to trickery like older shells or strange syntax like bash. The shell supports closures that you can assign to a variable to make what amounts to a function. Another welcome feature is that Crush understands the idea of namespaces.

There are a few oddities like using % as a wildcard instead of * because the * is for multiplication. However, we like a lot of the features, including simplified remote execution and the ability to create custom types.

Crush is similar to Nu, but has some different goals with respect to programming languages. If you are still writing programs in traditional shells like bash, be sure to run a linter.

53 thoughts on “A Shell? A Programming Language? Relax! It’s Both!

  1. “Every time we publish a Linux hack that uses a shell script, someone will chime in about how awful it is to program shell scripts.”

    Consider it an incentive not to do long scripts where a program would do.

    1. Yes exactly who needs a simple shell script when instead you can supply an entire eclipse project and all its dependencies, why not throw in the binaries for the compiler and the linker for all the different platforms just to be sure.

          1. Power Shell is a left over from the time that Microsoft did think they did dominate the software world.

            On hindsight they, MSFT, could have delay their downfall by chosen Python3 instead of making up “Power Shell”

        1. as someone who writes nodejs regularly – please don’t use it for simple things. Nodejs has its own special dependency hell, and when it doesn’t, it’s still got to install half of npm to do simple tasks.

          1. As someone who also writes nodejs regularly, its a scripting language still. Its perfect for small or scaled things.
            Npm is much faster that copying node_modules.
            Ya ya, if u need express so what. Wait a few seconds and your there.

        2. “Just use AutoIt in Windows”

          Ok, When in Windows.

          “or deal with bash in Linux”

          Did you read the article at all?

          “OR… Use NodeJS for either!”

          What drugs are you on?

          I didn’t like Javascript when it was only used in web pages.

          Javascript started as a toy language to fill up empty space with animations when someone couldn’t come up with enough actual content for their pointless personal homepage. Then it was built up into something more when people realized they needed to do some actual real programming to run client side on a website.

          So the result is a language where a class is a function and a method is a lambda expression inside a function. That’s just ridiculous.

          And we still HAVE to use it if we want anything to run client side in a web browser because that is what web browsers support.

          But NodeJS? Why does that even exist? Why would someone want to take that steaming pile of manure that is Javascript and use it anywhere they didn’t have to?

          1. @cyberzerocool There are plenty of interpreted languages out there where a class is a class and a method is built like a method rather than a variable assignment. Just because a language isn’t complied doesn’t mean it has to be brain dead!

      1. Or use Perl. Or Python. Or Java without a development environment that enforces an over-complicated project structure.

        Many, many options between a shell script and Eclipse complexities.

    2. Python is not good as a replacement of shell. It lacks basic features to be the one. Like how would you write a singleliners in python when you have to manage indents? Just a basic script in python has to have a noisy condition to let it be useful as loadable and executable at the same time.

      You must have zero experience with CLI to propose python as shell replacement.

  2. Whatever, ordinary shells also have directory listing built in, with results available for pipelining. For example: echo *

    And it’s odd indeed that bash syntax is considered strange but using curly braces and == to represent a SQL condition is not???

    Wow! Closures! As if you couldn’t t control shell commands with backquotes! As if there were no variable substitutios. As if the find and xargs commands didn’t exist! And all this just to replace one odd syntax with another odd syntax, no actual features added.

      1. Using % as a wildcard operator is more “sane”??? Putting {} around SQL expressions is “less fragile”???

        What “errors” are you avoiding??? Just curious. How many bugs are there in your “shell” and how do we avoid them?

  3. “Full-blown” means basically “it will blow in your face”…

    Well-established scripting languages, like Python, can do system calls, read and write files or directories, and do all the things shell can do. Without workarounds like “% as a wildcard”. And they don’t use “SQL-like commands” because usually they speak SQL too. And JSON. And CSV. And any other format (including some, Ilve never heard before, probably)…

    Also ‘member when shell scripts used to rely on bugs in the commands making these bugs immortal? ‘Member when sysadmins spent more time fighting with these bugs in their scripts than actually being productive? Bash was great name for a shell back in the day, because after some work with it one wanted to bash his/hers head against a desk or a wall…

    1. Oh yes computers were perfect back then, bash was the only buggy program. Compilers generated perfect code and the hardware never failed. It would have been nirvana without that stupid bash program.

      1. Compilers generated perfect code – it did what programmer wrote, not what programmer wanted it to do. Besides that bash wasn’t the only buggy program, we also had sendmail which couldn’t send mail reliably (and acted like a dog chewing on the mail till envelopes and content got integrated into one sticky mess), X-Window thing that used all the resources of the hardware to show a clock on the screen (because it was written by nitwits),NFS (Networked File System), which didn’t understand networks, files, and systems, etc..

  4. Why not PowerShell? I’d say it’s more of a programming language than it is a shell, but it’s still a perfectly cromulent shell.

    Conceptually it’s the same, but calls return “objects” or in cases are object like, which is a lot more programmy languagey than everything being returned as a SQL row or a json object. Also you can leverage .Net (and Core on Linux apparently) from with in PowerShell to get into all kinds of fun trouble.

    I don’t care that it’s MS, I really like PowerShell.

    1. That was my first thought; PowerShell departs from the data interchange model that is essentially scraping each command’s text output. My only issue is that some properties come as an XML object that you have to search with XPath for the attribute you need. But at least I don’t have to worry about weird filenames, quoting or converting timestamps.

      1. How do you not have to “worry” about quoting in Powershell? It works just like shell:

        “When you enclose a string in double quotation marks (a double-quoted string), variable names that are preceded by a dollar sign ($) are replaced with the variable’s value before the string is passed to the command for processing.”

        “To prevent the substitution of a variable value in a double-quoted string, use the backtick character (`)(ASCII 96), which is the PowerShell escape character.”

        Quoting in Powershell is almost identical to quoting in bash. WTF are you talking about?

    2. It’s too verbose and has a really bad shell (i.e. nih, and I don’t use it), that said this tool is inspired by Powershell you can read more about why not in the Crush FAQ.. I’m uncertain this is the way to go mostly because I think Powershell scripts has the potential to be one of the worst things to encounter, I feel it can be compared to the Perl of Windows. Clueless people like myself write thousands of lines of it and the expect other people to just get it.

      That said sure there are nice hacks that can be made, as long as they are crystal clear.

    3. I like PowerShell too, and I use it on Windows, Mac, and Linux.

      > Conceptually it’s the same, but calls return “objects”

      You’ve awakened the anti-object-oriented oriented! While technically accurate, most of my PowerShell code consumes text and produces text; for example, I frequently `yadayada | grep yadayada` in PowerShell.

  5. Anyone complaining about shell scripts should be forced to use a z80 based machine with tape recorder to do their “work” for a while. I know for a fact that there are destroyers out there who has antennas designed with the help of a zx81. So your phone is a freaking insane super computer. The shell syntax is only weird because you don’t use it.

    1. You had tape? We would have killed for tape. We had to use punch cards that could only be punched by a chisel and a 12 lb hammer to make the 1s and 0s. (Of course, someone will reply — you had 1s? We had to program using only 0s!)

        1. Back in the day they had silicon computers, though the silicon was big lumps of rock stood on end. They ran off sunlight or starlight. The programming may have taken the form of accessory sticks and knotted cords, moved around and held in place by helpers.

          Evidence mounts that ancient cultures knew the solar system was heliocentric, and tracked all the major planets. They also knew what was not a planet… and what that might mean.

          It is not beyond the bounds of possibility that one of the greatest achievements of humankind was the deliberate survival of the Younger Dryas extinction event, by predicting a cometary collision and preparing for it. Of course the death toll was still massive, and it literally threw everyone back to the stone age.

  6. I really do like Al Williams’ stuff. I really do like Linux; ok, what I really like is Slackware. I really do like BASH; but my true love is C.

    But I sometimes wonder if HaD people are, at times, ‘channeling’ Brian Benchoff to stir it up a bit. A feature-limited shell in Rust?

  7. Whatevs. People do all kinds of stuff because it’s fun for them. I don’t see the point of a “new” language when PHP and Python make perfectly good shebang scripts. There’s literally no dearth of interpreters to do stuff already. This stuff often saddens me in that it makes me think really smart folks are just lost in abstraction sickness. Maybe there’s a good reason for reinventing the wheel over and over again, but honestly, it just looks like intellectual peacock plumage to me. The “why” is in the “look at me I am programmer.” Meh.

    1. I love it when people think that Unix programs are small, it’s like they are unaware of how big the standard libraries are, they seem to have no clue that printf() is really an enormous piece of code.

      I also love it when people think that shell scripts are “simple” things worthy of being called “duct tape” when in fact you can write entire applications in them.

      1. I’m AndyPanda, a comical cartoon character , look me up ….
        Yes, the libs are HUGE, but the code is page demanded
        let’s get over with the duct tape phrase, i didn’t invent that, it’s been around a while \
        anyone who wants to beat up UNIX will have to deal with ME

    2. I thought that wasn’t so much about size but function.

      Smallish programs that did one thing, sometimes a few things, but added up, sometimes through scripting, you could get a lot done.

      Cal.is a really useful utility, if you need to display a calendar. Look is great for checking tye spelling of a word. And so on.

      Even in the age of gui, many linux gui programs are just shells to run existing command line programs.

      This compares with Word, or Emacs, that try to do so.much in one package.

  8. Way to miss the entire point of the shell, that the piped Data has no structure as far as the shell and its tools are concerned. Tools like grep and sort and uniq and tr and cat, none of them care about the structure of your data and yet they can still operate on your data. There is no difference between a stream and a table and a command, they are all just streams of characters and they can all be operated on with the same functions.

  9. Personally, I have found Bash scripts rather a necessity over the years, as simple data-parsing and automating program/build-prep tasks is its primary use-case for me. The Expect shell (tcl) serves a similar function for remotely deploying installations, mirroring, and diagnostic taps.

    I don’t think I would want to debug a piped set of commands with complex regular-expressions in a language like Python. As Python requires many dependencies to even load, and some time to execute the full size interpreter. When I do see the junior team’s Python stats script pinning the cores at 100% in a sleep state… I remind myself it is often often not a busy-loop… but the Python aggressive back-off scheduler and fundamental lack of proper threading support (or “nice” process control)… screwing with the kernel scheduler in ways that is often unforgivable, and it is not the kids’ fault for not knowing why this matters yet.

    Small shell scripting languages are simple for a reason… at one time scripts could often load, run, and then exit faster than a compiled C version using system calls. I think Lua was one of the few languages that bridged these use-cases fairly well, but this is rare for other languages (even Perl).

Leave a Reply to Al WilliamsCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.