The holy grail of computing is to have some way to distribute a program to any computer. This is one of those totally unachievable goals, but many have tried with varying degrees of success. People naturally think of Java, but even before that there was UCSD’s P-code and many other attempts to pull off the same trick. We were impressed, though, with Redbean 2.0 which uses a single executable file to run a webserver — or possibly other things — on six different operating systems. If the six operating systems were all flavors of Linux or Windows that wouldn’t be very interesting. But thanks to APE — the Actually Portable Executable — format, you can run under Windows, Linux, MacOS, OpenBSD, NetBSD, and FreeBSD.
This is quite a feat when you realize that most of these take wildly different file formats. There is one small problem: you can’t use much of anything on the host operating system. However, if you look at Redbean, you’ll see there is quite a lot you can do.
The APE loader takes advantage of some quirks. Under Windows, the executable is just that — an executable. Otherwise, there is a small shell script that the APE loader can use to map the executable into memory and do the ELF loading. You might say, “Wait! I don’t have the APE loader.” That’s OK. Each APE program has the 4K loader embedded in it that it will use if necessary.
Of course, it works because, at the bottom, the x86 instructions are the same across all the platforms. Since it is hard to call out to the underlying system, there is a libc, Cosmopolitan, that you can include. There are other modules, including a Unix module that provides things like file access, sockets, and other things you probably need to make working programs. The only place you might have trouble is Microsoft’s WSL which is sort of Linux under Windows. It “knows” how to load Windows executables, even under Linux so you have to disable that in some cases, and that’s explained on the APE page. You can also find some details about the executable format.
Redbean offers a lot of possibilities. You use zip to store assets like web pages in the actual executable. You have access to Lua, SQLite, TLS, and more. So it is possible to create a “web app” including things like JavaScript, run on any of the supported platforms, automatically launch a browser, and then use the browser as a GUI. Pretty slick. A .args file will prevent users from having to use any command line arguments. No installation and no picking the right version for your operating system.
Of course, you do need the right kind of processor. This isn’t going to ship out to your Raspberry Pi unless you run an emulator. But there are a lot of things we can think of that this would be perfect for. A web server on a USB stick, for example. Or a way to put an app on different Linux single-board computers. The APE loader is probably a good trick to have up your sleeve, too.
We had to wonder if you could integrate Tiddlywiki and have a portable notepad on a USB drive. Of course, if you want a portable web server, you could always just bring your own hardware.
Al, awesome article, loved it!
Redbean and Cosmopolitan https://github.com/jart/cosmopolitan is like sorcery! Its like alien technology. Have used both and I am so f**ing amazed. In fact so amazed that we are using this internally here. Redbean is higher level wizardry showcasing the power of cosmopolitan libc.It’s a zip file, it’s a executable – runs on many OSes – also I think it can be cross compiled to ARM but loosing the APE cross platform wizardry. Even the creator of TiddlyWiki, Jeremey Ruston sort of loves redbean: https://talk.tiddlywiki.org/t/custom-application-for-tiddlywiki-based-on-webdav/944/22
Imagine the time, space, bandwidth, electricity, brain explosion saved if this would become universal.
It is important to note that Redbean is very fast compared to other servers out there due to some clever programming. This video interview explains it: https://www.youtube.com/watch?v=1ZTRb-2DZGs
I have tried to compile NASA’s cFS with cosmopolitan and most of the stuff compiled except, for the OSAL part some of which I overcame with just writing the missing definitions for the libc functions which were present in cosmo, but it would have taken more time and beyond me to fix, but yeah promising. The fact that it got so far is an indication of how sufficiently mature it is already and evolving. This is due to the team consistently working towards supporting everything feasible.
GUI applications may not compile with this as they expect OS specific GUI libs. Don’t try to compile qemu – I’ll save you the time. Maybe if some of you know Fabrice Bellard, please point him to this new challenge of making computing and building software a better experience: https://github.com/jart/cosmopolitan Also the project needs more skulls, so hackers, you know what to do.
Justine and her crew are awesome people, friendly and very responsive, if there is a bug or a feature missing, they are highly responsive in fixing issues submitted to the their github!
Supporting more OSes is feasible, I have experimented a bit with running redbean and APE binaries on a few different OSes to gather info. So hackers here who love to fix such stuff, check out this: https://github.com/jart/cosmopolitan/issues/349
People like Justine Tunney and Fabrice Bellard have accomplished stuff similar to what Space X is to Space or what the Large Hadron Collider is to science! Much respect!
TiddlyWiki is pretty cool stuff! It caught my attention because of Joe Armstrong having used it to organize his thoughts when he was co authoring erlang.
The thing is, TiddlyWiki doesn’t need a server to be portable on a USB drive cos its a Quine, but then you’d have a lot of annoying copies on each save!
Since TiddlyWiki supports WebDav, it would be easy to implement that functionality in Lua in Readbean (it is sort of like two lines for the web dav thing) but the json api tiddler saving functionality would need work. Oh there is already one https://rpdillon.net/redbean-tiddlywiki-saver.html! So probably that way one has a portable self contained Wiki+Server on a USB usable on all OSes! (But windows and viruses…)
I use a custom https://gitlab.com/opennota/widdly (json api) and minbrowser https://minbrowser.org/ for a local desktop dedicated experience. I can access it from any device / phone in my network. My particular TiddlyWiki file with all bells and whistles from mermaid diagrams to latex (yes I have even embedded a KiCad interactive HTML BOM!) is sort of very heavy ie above 10MB so couldn’t fit nor reliable save on a NodeMCU – I’ll share that file in a repo for anyone interested.
I use TiddlyWiki in my org to communicate and document with my partner. Multiple instances of Widdly runs on server (the tiddlywiki file is compiled in) and changes are saved to a boltdb file (json) and proxied over Tor with key auth, so my partner can access it via onion and edit it securely, there is a bit of bash automation involved.
For the public version, after each edit via Tor a new wikifile is generated and stored on local IPFS and cloudflare DNS records are updated thus having the wiki live on web. Thus its very secure, no direct access to servers etc. The Tiddlywiki file has a full fledged embedded landing page too – technically that makes it a proper cms but the downside is that even with gzip http compression, it amounts to 2.4MB network wasted.
What caught my attention and love lately is Robbie Antenesse https://feather.wiki/ which like TiddlyWiki on diet – less than 70KB! This will fit a NodeMCU. I went even further down the rabbit hole – I put one one a STM Bluepill clone with rndis support – plug it in, open a browser: https://twitter.com/AMICABoard/status/1537815538709442560 had to fix a bit of the tools used for creating the images to add support for gzip compressed pages… I’ll make a product out of that.
Together I find that apart from GNU + Linux, NetBSD (BSD), KiCad, FreeCad, Blender, gcc, llvm, as well as Qemu that Cosmopolitan, TiddlyWiki, and FeatherWiki are the most useful pieces of software ever for me! When we are in the black, those are the projects that I want to support.
This comment was almost an entire HaD article in itself – in fact some might say it was at least above average for a HaD article.
Lol?
How about compiling your source code down to an abstract syntax tree and distributing that. On the end users machine they run a generic installer (specific to their platform) that inserts the opcodes required for their processor.
That would be just in time compilation, pretty much. JS/WASM do that already, except using minified JS as the “AST” format.
Why all platforms don’t just agree to support native portable JS apps, like electron with a system level shared browser, I have no idea.
Because JavaScript is not a system programming language and we need to stop using it as such.
But more serious, think about the ramifications of any security vulnerability on such system-level browser. The easier it is to run an arbitrary piece of software, the easier it becomes to exploit any vulnerability in the system.
> Why all platforms don’t just agree to support native portable JS apps, like electron with a system level shared browser, I have no idea.
Apple, for one, would hate that – they’re all about differentiation from the rest of the pack
When you only have one tool all the world looks like a browser/dependency hell.
The only reason to write a Javascript app is because you know no other programming languages.
Look up for a project called allvm and its relatives. As a side effect of allvm, one can compile to llvm bytecode, and then you can do the final linking on any target system… Did some experiments with it long time ago. https://publish.illinois.edu/allvm-project/
Isn’t that what .NET and Java do?
This was supposed to be in reply to anonymous.
delete your cookies for wordpress and hackaday.
WordPress JetPack strikes again! Don’t fret, nobody immune.
Java and .NET are JIT, although I’m pretty sure android does the “compile on install” thing. Honestly I’m surprised this isn’t done more, but I guess it wouldn’t really bw optimal for gigantic programs like, you know, web browsers that have like 100 GB of source code.
they both expect a rather substantial runtime environment to be present – whereas the concept per the article is a portable self-contained executable
Someone came up with a procedure to do TiddlyWiki in redbean.
https://rpdillon.net/redbean-tiddlywiki-saver.html
Of course this is only supported on Linux, Apple, and FreeBSD right now.
Thanks to you now it is also available for FeatherWiki:
https://codeberg.org/Alamantus/FeatherWiki/issues/58
Feel free to give it a spin and report if something is not working well.
Both .NET and Java required a runtime be installed. redbean is completely standalone – x86 code that runs on the aforementioned systems with no other dependencies.
would coding a java, .net or other JIT as the executable (together with a way to package the actual app of interest) into that same executable perhaps open up a way to make a load of existing java/.net/whatever-jit-you-have-used applications portable across all these OS’s?
“Redbean offers a lot of possibilities. ”
Like not worrying about which operating system to inject your virus?
B^)