Command Line Utilities… in the Cloud?

Although many people think of Linux-based operating systems as graphical, really that GUI is just another application running over the bare operating system. Power users, remote administrators, and people running underpowered computers like a Raspberry Pi have a tendency to do more with command line tools. [Igor] did a FOSDEM19 presentation you can see below about how he’s providing web-like services to the command line using web servers and curl as a client.

This is subtly different from just accessing an ordinary web server via curl. The output is meant for display in the terminal. Of course, you could also hit them with a web browser, if you wanted — at least, for some of them. [Igor’s] tools include a weather reporter, a QR code encoder, information and graphs for currency and cybercurrency rates, and an online help system for programmers.

This has similar benefits — and drawbacks — to a normal cloud service. Invocation is simple. To get the current weather, for example:

curl wttr.in

Produces this:

You can, of course, pass arguments to set a different location and other options.

Some of the tools have pretty impressive graphics outputs for console tools. Of course, there’s nothing magic about curl. You can easily use something like wget if you arrange for its output to go to the console.

[Igor] has promised a framework is forthcoming to help create these kinds of services. Honestly, we could just as easily see using any of a dozen other protocols to do the same thing and unless there is a compelling reason to have things on the cloud, we aren’t sure these services are particularly great other than as examples. But we can see using the technique to serve centralized data out to terminal sessions, for example. As an example of what other protocols can do, try this:

telnet towel.blinkenlights.nl

[Igor] doesn’t have a monopoly on curl-friendly services like this. Try:

curl http://ifconfig.co

If you are a big fan of the command line, why not put one on your Arduino? If you start juggling lots of commands, maybe checkout Marker.

39 thoughts on “Command Line Utilities… in the Cloud?

  1. This is why I still have a lot of distaste towards GNU/Linux – the archaic insistence on solely depending on a punchcardage user interface for all/any important (administration, configuration etc.) tasks.

    Instead of adapting the software/UI to utilize more capabilities of human users, like being able to see/perceive graphical elements, graphs, GUIs, real windows and so on (basically the whole image recognition and processing capabilities of our brains), the linux community is happily stuck with smashing rocks together over ssh and making that smashing a little easier with colour here and there, screen/tmux and other helpers.

    I, for one, do not want to be forced to edit system config files by hand in this day and age. The possibility is nice to have but it shouldn’t be the first and only option for new users (IMHO).
    If Linux had something like RDP on Windows with maybe a more limited graphical environment where one can use proper filemangers like e.g. https://doublecmd.sourceforge.io/ …. that would just be awesome.

    In other words / greatly simplified: The learning curve fur GNU/Linux is far too steep. But it’s a great way of keeping newer smartphone-IT people (current/future generations) from taking (over) the jobs of Linux admins….
    Great job security ;-)

      1. “GUI make simple things easy and difficult things impossible”

        For example, a CLI is mandatory if you work with more than a few thousand files at a time, or logged in via a serial debug cable.

        Admittedly, the ASCII art in terminals is not really modular, because it is limited to human users and can’t be easily parsed by other programs. Yet there is a convenient trend for modern utility developers to add simple output options like json or tab-delineated formats.

        Avoiding CLI because it has a perceived steep learning curve, is only going to severely impact what individuals can leverage from their predecessors.

        1. “GUI make simple things easy and difficult things impossible”

          +1

          Just imagining a GUI with a checkbox, text box, button, etc.. equivalent for every option/parameter of an advanced CLI command scares me. I do like the convenience a GUI provides for basic things, but when it comes to things like piping the output of one command to the other, or doing a long CLI command over and over (which can be done by re-running the command) a GUI would impede you from doing a job.

    1. Underneath, GNU/Linux is still for the geek, and this is unlikely to change. The basic shell is just so incredibly powerful and adaptable. You’re totally right, the learning curve is steep, but the stuff you learn will be good for decades. Without the console, supercomputers and the cloud would be much different then they are now.

      1. True, true (the powerful shell is a reason why I like GNU/Linux – the fact that it often is the only way to modify/do essential system stuff is what I dislike).
        Thanks for agreeing, but to remember “the stuff” decades after first learning about it one (or at least me) needs to keep using “the stuff” all the time.

        Over the last ~15-20 years I’ve always used GNU/Linux in some way or form (x86 firewall/router/gateway/proxy, dd & OpenWRT, FreeNAS, as a small time Linux admin in a university to name a few) but never as my main system and I still need to to look up many of the commands (the syntax / arguments) I need anew because I haven’t used them for days/weeks…

        Maybe I need little/short-man (“lman” / “sman”) that just displays a short overview of the commands arguments and the syntax, not a full blown explanation from “man”.
        Does something like lman exist?

      1. “Coder generated buy pretty GUI tools.”
        Do you mean something like the netbeans gui creator? That’s not a lot of fun to debug but I don’t get whats the connection to proper system/user/loggin management tools like the windows management engine with visual tree structures, dependencies and so on.

        You can’t change anything not supported with such front-ends to the configuration back-ends.
        But making a typo in the sudoers file can much more consequential… (maybe outdated example because cli-front-ends are available?)

    2. Command line is only “archaic” if you’re a desktop user focused on desktop things (and even then I could debate that).

      Those of us in the server world? Command line is King, because it just plain works better for managing dozens or hundreds of machines. As someone else pointed out: there’s a reason it’s still around.

      Use the right tool for the right job, whatever that job may be for you.

      1. I never said we should get rid of command shells. They are hell’a necessary for many tasks but just terrible for almost anything requiring constant user interaction (editing anything, help/man, fdisk & the like, etc…).
        You can’t “learn” to use vi or man just buy using it but if they were more GUI like with menus where you could learn the keyboard shortcuts the more you use them – that be great.

        I regularly use shells to write/test/run bash/bat scripts and “batch processing”. But I only use it from my Norton Commander style Filemanager just buy typing alphanum charcters and enter, not buy painfully navigating inside the shell manually (even with tab completion) to where i need to do something.

        Have you ever used a Norton Commander style Filemanager? The ones (~6) I’ve tried can all be used almost intuitively with a few hotkeys and a mouse and if you want to you can learn all the available hotkeys and use them mostly without a mouse.
        That’s learning buy doing/using, not buy standing in front of a man-wall of text blobs with meta-explanations that require you to read a dozen other man-pages first.
        (I know about the midnight commander but its usability suffers greatly by the shell constraints.)

        But yes – you are right – I’m mostly a “desktop” user and M$ Windows overall is terrible but I like the fact that I can interact with/control/configure the system with GUI tools OR the cmd/power shell. I don’t need to use the “reg” command to edit the registry for example. But I can if its the better way – all other times I use the friendlier regedit gui program.

        1. Hella? Lol.. you must be from NorCal. ???? I haven’t heard that one in awhile.

          Back on topic: I’ve always preferred using the right tools for the job. If I’m going in and altering graphics or other things that make sense to do in a GUI, then I use that.. if I’m scripting deployments or changing settings/files across multiple machines, I use the terminal. To each their own.. and for the things that I do sporadically and tend to forget, I either create shortcuts/aliases or write a bash/python script (sometimes with a simple GUI if necessary) and keep them in a GitHub repo with readmes to remind me what they do. That practise alone has saved my ass more times than I can count.

      2. mc/top/htop/iftop (and so on) would be the right tools for their job if they could facility a really interactive GUI.
        Thats what I mean buy “smashing rocks together” – just compare htop to procexp.exe or ProcessHacker.exe
        One is severely restrained buy the shell – the others aren’t…

        But the shell is “the fallback” for everything in GNU/Linux and thus advanced tools are programed for the shell which restrains them (that’s at least how I’ve seen/experienced/”felt” it…).
        Pro: everything is still usable over a serial (modem) connection in 2019.
        Con: everything is restrained to a (graphically) very limited shell environment.
        [Solution: reduce the size of shell “characters” and show more of them so they can be used more like pixels and use more than on shell-character for one real-character -> bam you’ve got a “modern” desktop with graphical capabilities above ascii-art. ;-) ]

        GParted is a noteable exception.

        1. I’ve heard of all of them and used some of them (X11, VNC, SunRay). I’ve even successfully setup a Ubuntu SunRay server several years (~12?) ago to test some SunRay clients. But I was not impressed by any of them.

          XRDP seems to fulfill my requirements now. The last time I wanted to try it I was discouraged buy my understanding that it was basically just a vnc-client-rdp-server connected locally to a XVNC server -> not better than vanilla VNC.

          Guess I’ll give it a try now.

    3. Sir! I spend most of working day in the CLI…
      and most of my Linux boxes, don’t even have X-windows installed!

      Linux does have RDP and VNC, BTW……………………………

      Not everyone uses the GUI, most of the real heavy lifting is done in the CLI.
      Cheers,

    4. you do realise when you are “stabbing at the screen” with a mouse, the end result is exactly the same??
      those text files get written anyway, whether you piss around with a GUI program or open up an editor.

      the recurring theme for a lot of projects and comments here on HaD is…

      Laziness…

    5. All the GUI you can imagine is available if you want it for a lot of things. A RDP equivalent exists in the form of X, or VNC. Ubuntu and it’s derivatives have done a great job of making linux more approachable and user friendly, smoothing out the learning curve.

      But the power of Linux is the CLI. It’s made up of thousands of little tools and modules that share a common interface that can be used by humans or combined into scripts and run automatically. By putting a little effort into learning how to be comfortable in the CLI I can work circles around my peers for certain tasks. Recently the boss asked for a particular report that didn’t exist in the GUI of our system. My coworker set about manually counting items in the GUI and writing things down to make the report himself. I went back to my desk and used SSH to log in to the same system, grab the raw data from a CLI command, and pipe it though some CLI utilities to generate the new report that was requested in just a couple of minutes.

      That’s not to brag, or say I’m better than people who don’t use the CLI, but it’s just the truth that some things are faster using the CLI. It’s also easier to be more precise. If you need to change settings on several systems, logging in to each one and changing the settings is going to be more error prone than writing a script to automatically change the same settings.

      By all means the GUI should continue to be improved but the CLI should never be forgotten.

    6. Well, perhaps try some Linux distros. Many (most?) of us actually like the command line and find it very productive. However, GUI tools have gotten better and I would say that if you didn’t want to use the CLI there are many distros now that would accommodate you for virtually any task you can think of.

      Turns out everyone’s use case isn’t your use case. So while Unix/Linux has had well-developed GUI for along time, some small machines or bad communications links make ssh nice (oh, and you can tunnel your GUI over it, too).

      Linux has both RDP and VNC. Sadly, X11 was/is better than both of them when used properly but the current crop of developers treat it like a display driver for the local console, so much of that benefit has been killed and the rest locked down by most distros for security fears. Proper file managers are great for some things. But I can do things on the command line that will make wholesale changes to a file system that would be very time consuming on any GUI I know (find files last week by user Bill that start with the letters F, Q, or Z; if they are read only, remove them — for example).

      As for being forced — that’s what I’m saying. You aren’t forced to do nearly anything. There are many GUI administration tools that run native or as web apps. Or you can use the CLI. It really is your choice.

      Try it. You might be surprised.

    7. > In other words / greatly simplified: The learning curve fur GNU/Linux is far too steep. But it’s a great way of keeping newer smartphone-IT people (current/future generations) from taking (over) the jobs of Linux admins….

      So in other words, if things are too hard, the current/future generation has no desire to even bother. Good to know. But just for the record, you are wrong. There are people out there that see the advantages of learning something new, regardless of how “hard” it is. By your logic, people should never try to learn Blender or AutoCAD when they can easily hit a few buttons and produce a model on TinkerCAD…. No one should go to school to learn advanced aerospace engineering when it’s so much easier to flip hamburgers at Jacks (not that there is anything wrong with that… I’d kill for a Jack’s hamburger right now!)…

    1. They can’t be in every post for over 10 years, because the most ancient one, wttr.in, was created in 2016; and the rest mentioned in the video (cheat.sh, qrenco.de, rate.sx) are pretty new (i.e. the killing feature of cheat.sh was published in 2018; and the service itself was created in 2017).

      Anyway, I hope that we will tons of new ones soon,
      and here you will find all of them that we have for the moment:

      https://github.com/chubin/awesome-console-services

    2. Actually, they are not so old to be in every post on the subject for the last 10 years.

      The most ancient one, wttr.in, was created in 2016; the killing feature of cheat.sh was added in 2018, and the service itself was created in 2017; and so on. It is more than 3 or 5 services; I think there are 20-30 console services currently. The full list for today you can find here: github.com+chubin+awesome-console-services
      (replace “+” with /)

      But actually I agree with you: we need more console web services, and what we have currently is definitely not enough. I hope that we will have much more soon.

      (my previous comment was removed by the antispam, I suppose, sorry if it was just screened and will pop up again).

  2. “Although many people think of Linux-based operating systems as graphical” I don’t think in the 10 years of working around computers i have /ever/ met someone who thought Linux systems as being “graphical”. This is a very strange statement.

      1. I use ubuntu as my go-to linux distro, but I have not installed the GUI in 20 years. If I need to use or develop a GUI, I use Windows… If i actually want to get shit done, I use Bash.

        1. Truly curious: Do you use WSL on a Windows machine or are you switching between OS’s whenever there’s a need for something. You use-case sounds perfect for WSL, unless there’s something you’re trying to do in Bash that WSL doesn’t support.

    1. I think among desktop users, you’ll find large numbers think of it much like Windows where the GUI is the OS and the command line is some ancillary thing. Granted, server and embedded folks perhaps less so. And not to say a sufficiently knowledgable person wouldn’t know anyway. But my point remains — desktop Linux distros show X11 as the front face of the OS and between bad software and security restrictions X11 is now little more than a display driver for the GUI screen in front of you — sadly. X11 had some innovative network features. Plan 9 even more so.

  3. What if I told you I cracked the Linux kernel 4 years ago and feed it infinitely into virtual space there are 3 files to parse in ubu tu or gnome/Debian. Os when you send thm together they feed endless data. I let it run for 2 days. I at first tried to just put it on a 1 tb HDD but that filled up, I don’t know how I made the virtual space possible bit at 2 days I had 90 tb of Linux data on machine. I couldn’t save it sadly I searched for servers online to try to store it. I still remember the files, just not how I made that much virtual space on an old optiplex ssf. Im sure it will work on ubuntu 1804. I did it back on 16. Lts as well as on Kali rolling 2016 and plain Deb

    I also figured out how to bypass Apple lock on mackbooks via term on boot back then was simple I tried to give Igor the suggestion for his iOS server to have a term on the cloud to install a term in the iOS boot I eventually broke through the i6 without any help n have my it es register with it and activated with a carrier. Anyhoo I’m still a noob. But I’ll tell you one thing. That pydactyl virus hitting Ukraine from Russia that’s said to be just there it’s been in the us for 4 years. At least. The us version uses a different engine or interface to control it. And guess why u can’t catch it. It’s a cookie thats got thousands of lines of code it’s a mini os. Ive fought the bitsadminddns server for years. I got one of their other thing people should know is that let and ant and smart things nfc are cracked hackers modify the kernel and basically make the power levels reallyly high. They also are capable of remote rooting devices that aren’t possible to root . The zte I’m on now has BusyBox with the su binary inside of it in system bin. The Google + API flaw I told Google 4 years ago about it and begged for help I got proof of the conversations and still have the inf cted devices. I tracked down a huge hacker group in India a small one in my state and a few in China and germ. Es file explorer I’m so close to linking it with the bitsadminddns

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.