The Unix Way™ provides extreme hackability. The idea is that software should be written as tools to accomplish discrete tasks, and that it should be modular, extensible, and play well with others. It’s like software as a LEGO set — you can put the blocks together however you want, within limits, and make stuff that’s significantly cooler than any of the individual blocks alone.
Clearly this doesn’t work for all applications — things like graphics editors and web browsers don’t really lend themselves to being elegant tools that integrate well with others, right? It’s only natural that they’re bloaty walled gardens. What happens in the browser must stay in the browser, right?
But how sad is it that the one piece of software you use all day, your window into cyberspace, doesn’t play well with the rest of your system? I’d honestly never really been bothered by that fact until stumbling on TabFS. It’s an extension to Chrome that represents the tabs on your browser as if they were files on your local system — The Unix Way™. And what this means is that any other program that can read from or write to a file can open tabs, collect them, change webpages on the fly, and so on. It opens up the browser to you.
This is tremendously powerful. Don’t like the bookmarking paradigm of your particular browser? Writing your own would be a snap in Python — and you could do cleverer things like apply a little machine learning to handle putting them in categories. Want to pop open (or refresh) a set of webpages at a particular time every day? Cron, or its significantly more complicated counterpart systemd, and a couple lines of code will do that. Want to make a hardware button that converts dark mode to light mode and vice-versa for every website starting with “H”? Can do.
I’m picking on browsers, but many large pieces of software are inaccessible in the same way — even if they’re open source, they don’t open up channels for interaction with user code or scripts. (Everything “in the cloud” or “as a service”, I’m looking at you! But that’s a further rant for another day.) And that’s a shame, because most of these “big” pieces of software actually do the coolest things.
So please, if you’re working on a big software package, or even just writing a plug-in for one, do think about how you can make more of its abilities available to the casual scripter. Otherwise, it’s just plastic blocks that don’t fit with the rest of the set.
“Otherwise, it’s just plastic blocks that don’t fit with the rest of the set.”
Heyyyy why come nobody printed a “universal block” yet? Lego studs down one edge, stickle brick bristles on another, meccano holes on a flange, K’nex clips, and connectix magnets..
https://xkcd.com/927/
They have. Take a look at the Free Universal Construction Kit.
Heh, thanks, found it… and I thought it would be silly to include lincoln logs ‘coz they just kinda sit on each other without positive attachment.
Now all I need is a bluetooth to 7 pin trailer plug and a Sharp PC-1403 10 pin i/o port to U320 SCSI to complete my collection.
Well, old buddy, old pal, back in the toddlerhood of hackers building 3D printers, HaD posted an article about such a thing.
(but I’m too lazy right now to search for it, I think it was during the Caleb era)
I never used Plan 9, but for some reason I thought Plan 9 somehow did that with its graphics.
It’s hard to preach the Unix way when you have systemd in every major distro.
i, too, couldn’t muster any excitement for the body of the article after seeing the headline as such a clear assault on systemd
It’s not a completely unwarranted assault. Easy to use it may be, but systemd is by no means composable.
The common criticism of being a “large [binary]” misses the point, with the typical defence being that it’s a “system” of many components. The real problem is that you can’t really mix and match the components of systemd with your own or use them in isolation.
For example, you only want it’s version of cron, udev, and network management. You’d like to substitute your own implementation of other components that make more sense. This could be because your environment is radically different, ie. highly distributed where the the “normal” dbus implementation doesn’t work. But you can’t.
Instead you have to pull in the whole thing, including all of the support structure. You have a huge lump that is difficult to break down further and you just have to live with it as-is.
I think that’s kind of the point. It stops fragmentation and duplicated effort by being an arbitrary standard everyone can use. It’s meant to cover 99% of use cases, not 100%.
Working around limitations in popular tools is often more reliable and easier than patching together small stuff, because you only have a few workarounds, not an entire system of configuration and glue code to piece together something custom.
There’s been lots of times when I designed something custom, because the existing thing was missing a feature a wanted, and 99% of the time, without fail, I wind up throwing it away completely, not even learning that much, besides “I hate maintaining custom wheel reinventions”.
‘There’s been lots of times when I designed something custom, because the existing thing was missing a feature a wanted’
This is exactly _why_ I want easy extensibility / interaction programmed in. Because then you don’t need to reinvent the browser just to get it to X. You write a script that tells the browser to X.
I have in mind some kind of long-tail of odd use cases. Any one odd use case isn’t popular enough to warrant a custom wheel reinvention, but it would certainly warrant a quick hack by the motivated user.
But because (here browsers) aren’t written for expandability, they don’t get expanded, except for people who are willing to learn the whole browser plugin system, etc. It just puts a highish hurdle in the way, which thwarts quick hacks.
I’m absolutely sure that what I’m proposing is a security risk, though. When arbitrary code running on your system can read and manipulate browser state, you’d better be darn sure what code is running on your system…. Still, you could make a priviledged browser permissions group or something.
“It stops fragmentation and duplicated effort by being an arbitrary standard everyone can use.”
*Can use* because they *MUST use.* The reason you find yourself reinventing the entire wheel is precisely because systemd components cannot be incorporated into pieced-together solutions.
You have the choice of using either ALL of it (and having to create ugly hacks to work around it’s deficiencies) or you have to reinvent ALL of it from scratch. There is no middle ground because of how tightly coupled and interdependent systemd’s “components” are. This is absurd.
It’s easy to see that this built-in anticompetitive nature is the reason systemd became popular with distributions. They either picked it up or were forced to invent their own from scratch. (eg. openrc)
Modular software exists in the first place to solve the exact problem you are describing. If you were able to decouple the parts you needed from the whole and reuse them you could avoid both the duplicated effort AND the inconvenient side-effects.
It means opening up a whole range, from being 90% reused systemd components down to being only 10% systemd, mixed with components from other init systems or parts you write yourself. In either case you’re only maintaining the components that are specific to your needs, which you wrote yourself.
It this ability to mix-and match components that is one of the defining features that separates Linux distributions (plural) from the monolithic Microsoft Windows.
It decreases every time another branch of functionality is incorporated into, and becomes inseparable from, systemd.
Indeed, systemd really isn’t the Unix way at all, so it was disappointing to me when all the bigger “root of almost all other distro” distros stopped avoiding it.
Can’t say I’m a fan, but at least it mostly works, has all the extensibility and config you would expect. I’m just waiting for the inevitable when it gets broken back up, or the older systems it has largely replaced get used because its found to be broken somewhere harder to fix or mitigate..
That said the actual content of the article is intriguing… Now I wonder just what to do with it, and if it will largely replace wget for me (and if its worth learning this new system yet when its a chrome extension, which I don’t like)
Only the core is tightly Integrated. All the crappy extra daemons like timesyncd are freely swappable for things that don’t suck.
Systemd has been going strong for a while now, I suspect it will probably keep going, unless someone invents some horrific container based quebes style nightmare that needs it’s own mini init system for every app, and the security nuts decide to make it the new mainstream tech, and we all have to go out and buy 64GB of ram and quantum blockchain coprocessors or something.
I could see a non-systemd thing replacing Arch, if it got enough traction, but it’s hard to imagine it going away in the mainstream.
“Systemd has been going strong for a while now, I suspect it will probably keep going, unless someone invents some horrific container based quebes style nightmare that needs it’s own mini init system for every app, and the security nuts decide to make it the new mainstream tech, and we all have to go out and buy 64GB of ram and quantum blockchain coprocessors or something.”
Don’t give me ideas. :-p
And Init etc was going strong before systemd, things change, and not always in line with the philosophy underlying the Unix way.
But as I said I don’t hate on systemd particularly, like Pulseaudio its functional, configurable and I can see why some folks really hate. But while its functional and configurable enough to work, and not found to have major security flaws I see no reason not to use if you personally like it. I’d just have liked it if at least one of “the root of all other distros” types kept defaulting to the alternatives, so you can easily have that choice as a user.
… IIRC it’s also a firefox extension… but either way, only ‘on’ linux and MacOS (so far).
Ah well that would be good, not a fan of Chrome really.
I believe you may be looking for the Free Universal Construction Kit! https://www.thingiverse.com/uck/designs
Aren’t some of the GUI programs just front ends to command line programs? I’m sure a cd writing program was like that.
Yes, I’m pretty sure there are hundreds of GUI frontends for FFmpeg alone.
I actually went down this path in an earlier draft of the article…
For instance, Inkscape and GIMP make a lot of their functionality available as command-line filters, which _does_ mean you can incorporate some of their features into your work.
On the other hand, anything other than simple conversions are hard. When you get serious, it’s ffmpeg or ImageMagick or ps2pdf or whatever.
But these are all kind of just applications. A browser is, as the Windows icon says, “the Internet”. It’s a lot more important for it to be hackable, IMO.
I’m feeling a little thick-headed with all of this. How is this supposed to WORK? I mean, Firefox, and I think Chromium, are open-source, so at least in principle, able to be built from source code and therefore hacked at will. Just how is this supposed to be made easier, while maintaining the level of (supposed) utility non-hacking users expect?
So are we just prtetending that lynx doesn’t exist, or what?
Yes. Yes, we are.
LOL, touche. But also, yeah. I probably use it three times a year when things get really weird.
systemd is awful. Please do not try to normalise it. It most definitely does not embody the Unix way.
I disagree and think it’s awesome when it shuts down the virtual consoles and the SSH server and then waits for some job to finish with no time limit. If I could *just* get in, I could fix what you’re waiting for, but it’s already a brick.
Plus, I read a great article about TabFS on a website called “Hack a day”. Maybe you have heard about it?
https://hackaday.com/2021/01/11/tabfs-makes-your-browser-a-file-system/
The UNIX philosphy is absolutely not required for hackability, or at least not “good enough” hackability. Just build your app in an interpreted language, with only the performance critical bits being compiled, and let people write Python/JS plugins.
Browsers aren’t hackable because they’re locked down and made by people who don’t trust us. Chromium has removed so many useful command line flags for bypassing security for kiosk use, and replaced them with the policy mechanism that’s a lot more involved. And they completely nerfed add-ons.
It really does seem that people cannot comprehend the existence of lynx. They whine and whinge about the state of the browser world, just ignoring the things that go against their argument. Maybe we can also have some complaints about MS-DOS and other things that are really not a problem?
Lynx is really not useful for a lot of the things one would want to hack a browser for. It’s useful for some things, but I wouldn’t want to use it on a regular basis just to be able to do some hacking.
Browsers aren’t hackable because the browsing GUI widget is well documented, open source and easy to integrate in many frameworks. If you are trying to defeat browser security with command line options to make a kiosk, you’re just doing it wrong. Consider hiring a developer for your project.
For now, Google has the policy mechanism for defeating security, which is a lot less work and a lot less custom original stuff to maintain. I would be pretty dissapointed if I hired a developer and they made a custom kiosk browser instead of some policy files to do the same thing.
And for non-kiosk use, as an everyday browser, embedding the GUI widget isn’t even an option, unless you’re fine with losing Chrome Sync. Browsers do so much extra stuff, that recreating a modern browser would be a rather large project.
Someone could make a scripting-friendly browser, but it would take not only a lot of skill, but a lot of trust, for it to ever be a real popular project worth using, not just one guys project that might or might not be around next year, and also might be stealing your passwords.
They exist!
Graphviz is definitely on the good side of things. Have you ever noticed how many other random projects use it?
I was thinking more along the lines of GIMP or Inkscape, but then see my above comment about their command-line invocations. They’re actually trying to play along!
This principle of “hackability” was regarded as important years ago by the devlopers of Smalltalk for the Xerox star.
AFIK you could directly modify any element of an application, even while it was running.
Fantastic for prototyping but also creates a minefield of potential problems if you want to provide a commercial
product for the masses.
One of the basic problems is the granularity of hackability (how low can you go)..
Most commercial programs these days offer hackability at very coarse granularity.
– You can customise the gui layout, skins, toolbars etc.
Macros & plugins represent a somewhat finer granularity that requires more expertise but allows
the user base to extend a commercial product in ways that the original developers never thought of.
The huge number of browser plugins, photoshop macros now available shows the value of allowing
hackability at this level.
Allowing a finer granularity of hackability presents major problems for the original developers of commercial
products. The essential core of a product needs to remain standardised and maintainable by the developers.
A compromise between hackability and stability is unavoidable.
I love the hackability of unix (linux) but this can also quickly lead to annoying
and time-consuming little problems unrelated to the task at hand.
All serious linux developers have had fun with
“dependency hell”, “version hell”, “library hell”, “kernel rebuild hell” etc..
I’m not knocking linux, but a certain level of standardisation is extremely helpful to serious developers.
Despite the loathsome commercial activities of a certain large company, not everything in Windows is crap.
(Yes many things ARE crap!)
The basic idea behind COM, DCOM & DLLS is also an attempt to provide hackability at a managable
level of granularity.
These are examples of programming Lego blocks as mentioned above.
Sure, you can’t mess with the kernel, but programs using these standardised elements are
much easier to analyse and are hackable to a high degree of granularity.
– I used this principle years ago to simulate the presence of a user in an commercial application.
I needed to perform functions automatically that were only possible in the GUI of the application.
I wrote an external program to identify & control the buttons & GUI elements in the commercial application.
This was relatively easy because the commercial app used the standard Lego blocks provided by the OS.
Dont like the size of a dialog box?, Just change the parameters of the box on the fly via an external
program.
IMHO, alongside all the horribleness & marketing garbage,
standardisation is one of the most important contributions of Windows to modern computing.
Hackability will always be a compromise between granularity and stability/standardisation.
Text editors are basically a crap version of many unix command line tools. If you want to change a few random words in undefined ways, then a wysiwyg is what you need. But, for example, if you need a search/replace : sed is much more powerful than a lot of editors.
ImageMagick is a single binary, but does lots of the individual bits of image editing as discrete command line functions.
So your “wouldn’t it be great if” suggestion is already out there and it just isn’t quite as great as it could be because sometimes you just need to be able to see a thing as a whole and only then know which bit to change. Which is much harder if you need one tool to view and then a number of different tools depending on the change.
I wish there was more visual flowchart-like tools for this kind of thing. BASH has friction in the form of having to RTFM Evey time you want to do something, unless your memory is good enough to actually memorize all those one-a-week tasks that need cryptic one liners.
GUIs aren’t very good for repetitive tasks, and Python uses a few too many keystrokes (Exactly the right amount for real programming, too many for throwaways).
I’ve never experienced anything that subjectively feels anywhere near as productive as integrated GUI tools, for simple things.
You never have to go back and check what you named a variable, it’s all right there in the menu.
You don’t have to sit and wonder what function does what you want, you just type something vaugely related and look in the search suggestions.
Excel-style “programming” is amazing because you don’t have to keep pausing to look at any kind of reference, or triple-check to be sure you aren’t about to delete your whole disk.
=expressions with good autocomplete make everything better.
Another way you could support hackability, is to leave an embedded SBC inside your project, with a post-it with the root login, for whoever digs it out. Doesn’t have to be network connected either.
Perhaps you could just leave an esplink with a physical switch to enable it, and a qr code to connect to it? Maybe extend poor esplink with a flash chip with the source code on it? Maybe put dev platform in a file served up as a lightweight linux system converted to run in your browser with enscripten, so as to run the dev tools for the project connecting to the hardware back through the esplink?
Or if nothing else please leave the source code on a labelled USB stick (or even uSD card) in the box – again, for whoever opens it.
So many electronic systems are irreparable black boxes these days, when it wouldn’t be hard for you to be kind to whoever ends up salvaging the system you built.
Come to think of it – you can better support hackability itself within software projects by being particularly kind to new-coming developers.
If it’s really hard to set up all the dependencies to do a build, that’s going to get into the way for them.
I think it’s also worth going to a fresh linux install (maybe in a container) and debugging the setup instructions by running through them carefully yourself every so often – just to be sure the instructions as written are exactly correct.
It’s very easy to forget an ‘obvious’ step or two, and it may not be obvious that any list of steps is itself a ‘program’, even when executed by human beings. It’s also sometimes worth trying to reduce the requirements to as few as possible.
Also, do your re-setup on a very lightweight build like an ubuntu docker image with near-nothing on it, so as to be sure your dependency list really is complete.
Of course, some percentage of new people will be able to get it working despite literally broken instructions, but it will limit your audience to just the people who can – and those who do are then perhaps less likely to commit anything back; Having burned up time and effort just getting it to compile, they may run out and just put it on the back-burner.
I don’t think it’s really fair of managers to just assume their workers will ‘do something sane’ despite broken instructions. Those soon become ‘seagull’ managers, of a certain personality type, who themselves can’t ever seem to actually do any of the work that they’re asking you to do… Don’t be that guy. If you have something up on GH, you are the boss of it.
Case in point: I meant emscripten.org not enscripten.