With the latest release of Raspberry Pi OS (formerly Raspbian) the end of the X Window System has become reality, completing a years-long transition period. Although this change between display servers is not something which should be readily apparent to the casual user, the change from the client-server-based X11 protocol to the monolithic Wayland protocol has a number of implications. A major change is that with the display server and window manager no longer being separate units, features such as network transparency (e.g. remote X-sessions) are no longer a native feature, but have to be implemented separately by e.g. the Wayland compositor.
For Raspberry Pi the transition to Wayland was based on the perceived efficiency and security benefits of the monolithic architecture, with the 2021 release of Raspbian (based on Debian Bullseye) testing the waters using the hybrid X11 window manager/Wayland compositor Mutter. This allowed for switching between X11 and Wayland without committing. In 2023 Mutter was replaced with the Wayfire compositor with Wayland becoming the default on Raspberry Pi 4 and 5 platforms. Along the way it was found that the Wayfire project wasn’t developing in a way that would benefit Raspberry Pi OS, which led to what should now be the final Wayland compositor in the form of Labwc.
One advantage of Labwc is that it is more lightweight than Wayfire and Raspberry Pi has judged that this means that it should be the default across all Raspberry Pi systems. Compatibility with X11-based software is maintained with the XWayland library, so that users should ideally not notice any difference after switching to Labwc even on lower-end boards. Unless you’re one one of those people who use features such as (remote) X-sessions, nothing should feel markedly different.
In addition to this big change, the new Raspberry Pi OS release also improves touch screen support with the integrated Squeekboard virtual keyboard popping up when a touch screen is detected. Finally, the remote access Raspberry Pi Connect feature sees a few tweaks, which is the feature that effectively replaces remote X-sessions. Considering how glacially slow X desktop sessions can be, this is something which can be considered an improvement, but it would be nice if there was an alternative that didn’t rely on Raspberry Pi-provided services to work.
Does it mean it can finally run Tibia?
Urgh, this is just exhausting at this point. I guess I understand the need to switch away from X, but at least for me, the real change is that it’s really hard to find out how to do anything anymore. One of my projects involved auto-starting a few apps on a RPi, and I thought “great, an opportunity to learn how to use wayland”. But the thing is now if I search for how to do that all I get is a decade of great posts about how to do it in X, and almost nothing that’s relevant to wayland. And now I’ve just tried a search for how I might autostart apps in Labwc, and I’m honestly not sure if it’s even possible.
I’ll be honest that I don’t really understand what X (or wayland, or wayfire, or labwc) really is and how it really works, but it feels like, for people in my position there really isn’t a good guide to understand the basics. I think the documentation (for wayfire at least) is pretty sparse, and in the absence of a decade of forum posts, blogs, hacks, stackoverflows, and everything else it just feels like such a struggle to do even the simplest things with a raspberry pi. Anyway, that’s just an opinion, and I’m not sure what solution there would, if any
Totally agree. Lack of prominent public documentation about the new gen pi product will kill public support very quickly.
Since raspi is systemd based, start them using systemd services. Basically you write a service description file in /lib/systemd/system/mylittle.service, and enable it with “systemctl enable –now mylittle”
That does not work with GUI applications, of course. Those can be autostarted with X-based DEs using XDG (Freedesktop standard) autostart files in /etc/xdg/autostart and the user-specific version. Under Wayfire you had to configure a different file for this, and with Labwc it’s different again, of course.
biased and unhelpful comment follows
Using a pi for gui applications is the wrong tool for the job. If you need either the smallest footprint or power usage, pis are great, otherwise x86 is winning outright.
I’m sure the documentation / community understanding of how to do what you need in mainline x86 Debian or Ubuntu is easy to access.
Raspis aren’t the best tool for low power consumption. My notebook uses about 2-3W writing this comment. I never got a raspi 4 down that low. I don’t own an raspi 5 so I can report on this. And my CPU is much faster, so when I need compute power it is available.
I have exactly this issue, but my timeframes mean I just had to revert to X and pop it on the back burner. Finding good documentation for this issue is draining to say the least!
I use and love the network transparency of X11 and the client server model. I guess that means it’s time to ditch all things Raspberry Pi. Wayland also breaks things like Xdo and easily scriptable and cli control of gui elements.
doesn’t it get tiring? the constant bitching I mean. “this thing changed one thing I don’t like . therefore, I must discard all about it. that’ll surely teach them”.
it’s open source. you’re free to modify it. you’re free to NOT use the default they provide, and use an alternative that suits your taste.
A lot of distros/login managers let you switch between DEs that use the two at will on login… I’d be pretty surprised if this isn’t an option on raspi OS.
Agreed. Besides, X11 is 40 years old. It’s WELL BEYOND time to throw it in the bin and use something build for modern platforms.
That is a rather strange argument. So we are going to throw out Python in 7 years because it will be 40 years old then too?
The architecture of X11 is more tinker friendly. It is pretty easy to build your own window manager in very few lines of code for instance.
In general it seems that Linux is heading in the direction of what is good for corporations instead of makers/tinkerers.
Because it’s sitting on top of a pedestal that is only still standing due to the heroic effort of a very few people, and even they won’t be around for very much longer. You move people to a new ship while the existing one is still afloat, not after it’s sunk and everyone has drowned.
The vast majority of Linux developers have either been directly funded by “corporations”, or they have well-paying jobs that leave them enough spare time to do it. It’s been like that for almost the entire lifetime of the Linux kernel. You might get it for free, but only because someone else is paying for it.
Nobody is the sole arbiter of what “corporations” and “makers” want, certainly not you or I, and just because people make decisions that slightly inconvenience you specifically doesn’t mean that they’re somehow betraying this noble group you claim to be a part of.
Or time to turn off the voyager probes, because they are still working. /s
“Because it’s sitting on top of a pedestal that is only still standing due to the heroic effort of a very few people”
Yup. Very few people out there have the expertise to do this sort of work. That’s why choosing to break long beloved features and snarkily replying “it’s open source, implement it yourself” is kind of a dick move.
Seriously.. the wayland crowd could do so much better without even writing a single line of code by just not being dicks about things.
Age is not a criterium. You could argue the other way around, that if a package sets the standard for 40+ years, it already proves it can stand the test of time.
As long as articles like this one don’t get any further than “the user shouldn’t notice a difference, but feature so-and-so will be missing”, there’s no reason to switch. Switch when it BENEFITS the user!
Age is not a criterium. I’d argue the other way around anyway, if a software package is the default for 40+ years, it can’t be bad.
Windows has been around 40 years next year, just saying…
Won’t hear me saying Windows is bad. I think my last stroke of that silly evangelism was 35 years or something, when I had an Amiga and the Atari ST was bad. After that I grew out of it.
And Linux is based on Unix which is much, much older. Windows is the new fangled thing that was supposed to make Unix obsolete. I remember when Windows (NT) was new in the 90’s and they said it doesn’t need remote sessions or multiple simultaneous users like Unix since computers are so cheap everyone has one on their desktop. Funny thing that Windows supports both, now. So, yeah, age definitely isn’t a problem.
Forget X11 vs Wayland. We still haven’t settled which text editor is the best.
“cat” is the bestest, real programmers never make mistakes and never need to edit their code!
cat > hello_world.c#include<stdio.h>“
void main(int argc, char *argv[]){printf(“hello world!\n”);}^D`And it is the real reason everyone loves cats!
I really do hate when wordpress eats my pre-formated text
“If you don’t like where this ocean liner is going, you can jump off and swim wherever you like.”
Free to modify it as in speech, but not as in beer. “It’s open source” doesn’t mean you can flick a hidden config setting and get the behaviour you want. Sure you can make a small change to a small project and – assuming it actually compiles when you downloaded the source – you might get the solution you need for only a few hours of time (=$$).
But making a substantial change to the OS is going to require substantial investment of time to understand what needs changing and to make it all work – assuming on a larger project that it’s even feasible for a single person to keep up with it. That’s hundreds to thousands of dollars of time invested, and you don’t know you’ll get the solution you want.
And then, each time there’s an update, you have to re-patch it, which is not necessarily as simple as reapplying a diff. But even if it is, there’s certainly no auto-update, so you’re going to have to invest constant ongoing work to keep on top of security patches.
“It’s open source” doesn’t mean you can flick a hidden config setting and get the behaviour you want”
In the kernel it does, that is latterly what so many do when they rebuild the kernel. As for putting X11 back, so many options to do that, as simple as flicking a switch. On top of that these days with yocto / build root it’s even as simple a pulling some repo’s change some configs and you will have your own dream distro built by you built for you.
That didn’t seem to hinder systemd.
“doesn’t mean you can flick a hidden config setting and get the behaviour you want”
Only because that setting isn’t hidden. Yes, you can literally change one setting and the system will reconfigure to be identical to how it was previously.
Not only is there a menu on install to pick your DE/WE, you can also do this after-the-fact on a running system.
For me you’ve pretty much described why developers want to change to wayland. X is burdened with a lot of stuff that’s no longer relevant and it’s hard (or impossible) to add modern features at the appropriate level.
https://www.geekersdigest.com/how-to-switch-from-wayland-to-x11-in-raspberry-pi/
It’s quite easy to revert back to the trusted and stable X11 system.
it’s not open source. it’s raspberry pi. the graphics drivers are all deeply integrated with the firmware, which is closed source. it’s a real gosh darn nightmare, imo. for example, if i wanted to use the new version of raspios that uses wayland, i would have to give up all of the software i wrote based on the old firmware…because they broke all the APIs.
it’s vaguely possible to hack it but it’s not any easier than it is with closed source windows. it’s not open source.
XWayland is the API translation layer that it ships with to err not “give up all of the software” written for X11. Unless I’m missing something, I don’t see how the graphics firmware comes into it? (Which I agree with, it is annoying they are closed source binary blobs, but that’s not changed, and has got very little to do with Wayland/X11 unless you are writing wayland/x11?).
it just means you have a very limited capacity to mix and match or modify components because whatever you wind up with has to work with the closed source firmware. if you take advantage of the open source nature of some parts of the userland, you will still be bitten when the underlying firmware changes. you will have to chose between your hacks and whatever valuable updates raspberry has given you. it is generally possible to modify it, like hjf says. but if you do modify it, you have to contend with a zillion interlinked decisions all downstream of an undocumented, closed, and constantly-changing firmware.
on the pc, there are closed components but they are abstracted away to where it doesn’t hardly affect the open source components. but on raspi, the ‘open’ userland is a carefully-balanced hack closely integrated with the firmware. it is slowly getting better, but even that process of improvement is painful in a closed source kind of way.
oh sorry i misunderstood your question! the software i would have to give up is written to use the firmware, not to use X11. the firmware has changed to a better interface than the interface they used in 2020 when i wrote my software that uses it. but the change means throwing away all of my work. and because the firmware is closed and undocumented and tightly-integrated, i can’t mix-and-match.
in order to install wayland, i have to upgrade the firmware and give up my work. in order to keep my work, i have to use the 16bpp-only X11 they were distributing in 2020.
if i did decide to throw away my work, and upgrade raspios, and use wayland, and configure wayland how i want it to be configured…i would be in the same position in 2 years. they would upgrade the firmware again, and change the supported display server again, and those customizations would fail then too. but the challenge i face today is because i used an obsoleted firmware interface
This is such bs. There isn’t a graphic card nowdays for running normal desktop environments you can buy and it isn’t running its own proprietary firmware!
Oh, you may find a decades old one but guess what? The hardware is not opensource anyway so it is not opensource by your standard!
Oh but there is a truly opensource design somewhare…. which you can even build using opensource Yosys toolchain and run on your totally PROPRIETARY FPGA.
yeah but on the PC i benefit from the open source components. i have used xfree86 or xorg for almost 30 years, and i have even hacked its source on occasion and found that to be valuable (for example, i am using a hacked evdev input driver that i have carried with me through 3 laptops now). the closed source components are wrapped in stable interfaces.
on raspi, the closed source component keeps changing its interface and everything on top of it moves in sync to it. i can hack things but my hack will stop working in a year or two when they replace the entire stack that i had been depending on. upgrades are all-or-nothing because the userland is tightly integrated to the closed firmware. if you want to use a specific X server or wayland display server, you are locked to a specific obsolete version of the firmware.
On my PC, the proprietary graphics blobs are not necessary to simply boot up the computer. On the rPi, they are.
THAT is the bs if you ask me, libre purity spirals notwithstanding.
Wayland clients can be executed remotely quite trivially, so this is a nothing-burger.
And this isn’t Windows, almost nobody needs cli control of gui elements, even if they think they do.
When would Windows need “cli control of gui elements”?
I mean purely in the sense that practically anything you could do in a GUI in linux has a first-class CLI equivalent.
There’s lots of GUI-only Windows apps, though, and a lot of people end up using AutoHotkey or AutoIT or other tools to automate actions on them.
I’ve been slowly getting more comfortable with Linux over the last 2ish years, and I have absolutely no idea what you are talking about…
90% or more of the nitty gritty stuff either has no gui control, has a jank 3rd party gui last updated 7 years ago, has a learning curve bigger than the GUI tool AND scripting that tool, or is missing ONE critical thing you need and forces out to use the GUI for at least that step.
Ah, the “almost nobody needs”. Well, who needs Wayland then?
Don’t be disingenuous.
It’s trivial to show that lots of people use some sort of windowing system be it X or Wayland or MacOS or Windows.
Please, give me some obvious examples of where people would need to automate clicking a GUI button in an X or Wayland app where there isn’t a better and easier method. I’ll wait.
Group A has reasons to want to update, and dismisses Group B’s reluctance because A doesn’t care about the features B constantly uses.
Group B dismisses Group A’s desire to change, because the current system works for them, and the new system will require them starting from scratch again to get anything done.
Buth sides have valid complaints.
Don’t act like one side or the other is simply right.
I asked “who needs WAYLAND”, not “who needs a windowing system”.
Everyone who wants a GUI. At all.
Modern hardware doesn’t really work with X and X’s expectations about how hardware works map BADLY to modern hardware. On the M1 GPU it’s flat out impossible to make X11 work properly. It only works at all on nvidia and AMD because they both spend a lot of time hacking up their DDX drivers. Intel just flat out doesn’t give a shit anymore and as a result X11 is flat out broken on a lot of their older hardware.
In the very near future it’s going to just be straight-up impossible to run X11 on modern hardware.
Microsoft and apple have both moved on to more modern, better ways to render a desktop. It’s time the unix market did too. That’s why everyone who maintained xorg dropped it like a hot potato back in ~2015 and has been working full time on wayland ever since.
i keep hearing this and i am ignorant about whether it’s true yet? i hope someone can clue me in?
what i like about X11 remote execution is that it is thoughtless. i don’t have to anticipate that i will need it, i just am in an ssh session and i realize i’d like to view an image…i don’t have to start vnc, or scp over the image and view it locally, or put it in public_html or whatever. i can just run ‘xloadimage foo.png’ and X11 just magically works.
obviously, i hate the X11 slowness! a lot of round trip overhead. so if i know i am actually going to be doing work with some remote program, i have a script that starts up a headless X11 vnc server and connects to it on my laptop. and then everything i run under that is relatively fast.
so i know the overhead of starting up a vnc session is really pretty minimal…but i still like that i don’t have to for one-off impromptu tasks. i don’t have to anticipate that need.
my impression of wayland is that there are options but whenever i put a toe in, it seems like there are too many decisions to make, and each decision ties you to a display server which will surely become abandonware soon (like in this article, i guess). there’s no default path for remote execution, and the diversity of paths provides plenty of features but no stability over time??
in practice, does that matter? anyone have experience using wayland over a few years want to opine? do you have an easy time meeting your needs with a new display server or can you use the same one for years or what? do you have to periodically replace your whole userland of display clients or is there some steadiness over time?
https://gitlab.freedesktop.org/mstoeckl/waypipe
thanks! – that looks usable, maybe ‘essentially as good as’ ssh integration of forwarding. took a moment for me to put my finger on it, but i think you answered a different question than i asked.
this might seem super pedantic of me so i’m sorry but my question is, have you used waypipe? for how long? have you been forced to swap out the compositor? did that break your waypipe?
the thing i’m worried about is the user experience. for at least 5 years i know wayland has had all the features i want. the question in my mind is how many times have you had to mess with it or relearn it as it grows? like on x11 the worst i have is every new laptop i get, i seem to have to play the game of which one is fast: xterm, rxvt, wterm, etc (and i don’t know why that is). and i still have mixed feelings about the different x11vncserver options.
in my imagination, wayland brings that consternation closer to the center of your experience with the compositor having so much diversity with so many dependencies. but i don’t actually know and i want someone to say it ain’t so :)
@UT
That is a radical move but does it solve anything? I mean you would have to find other board with a X11 based distro hoping it will be equally mature to RPI and it’s OS. Why not just change operating system? I think Slackware still does that.
lol.
ydotool
waypipe
Fun fact: X isn’t network transparent either and hasn’t been for 30 years. Everyone (and I mean EVERYONE) is dependant on extensions (shared memory and DRI2) which are absolutely not network transparent.
See: https://youtu.be/RIctzAQOe44?t=1115
X11 is network capable but saying that it’s the same over the network as on the same machine is total nonsense. It’s flatly false.
And Wayland is network capable too now! It’s called Waypipe. https://gitlab.freedesktop.org/mstoeckl/waypipe
_
Dear Maya,
After reading two of your recent texts here I’d like to kindly ask you to be a bit more carful with words you choose. Neither ZFS is an “afterthought” in Linux nor local X sessions are “glacially” slow (remote are, sometimes). If you do not have numbers to support such claims, try as hard as you can avoiding them.
Thank you in advance,
s
Yeah that remark surprised me as well. “Glacially slow” is not my experience, and making such a subjective claim in a news item is bad journalism. Keep your opinions out of it. Want to give your opinion? Write a separate review or column or whatever.
Agreed. “Glacially slow” is not my experience, even when on a remote session. This is just bad journalism. Keep your subjective comments out of news items. Want to give your opinion? Write a review or a column or something. News articles should be factual and objective.
Considering that the “glacially slow” comment was specifically talking about remote X sessions only, I fail to see what your complaint is.
You say that as if you’ve never seen a remote X-session draw widgets slow enough that you can fetch a coffee before it’s done rendering a window refresh. Across a LAN it’s generally fine, and I have set up X-terminals in the past with no complaints, but remote X-sessions are a known pain point.
Maybe my words are in fact backed by credible evidence. Something for you to consider.
But you didn’t and haven’t cited any credible evidence, only replied to a comment with a personal anecdote. Measured and documented benchmark and statistics is credible evidence and there isn’t any to back up that claim in the article.
Why would I need to cite evidence for something that has been common knowledge since the 1980s? Even in this comment section you can find others who corroborate it, but feel free to do some googling yourself if you feel a quaint urge to defend X11 for something that’s simply a feature due to how the protocol works.
I like the breadth of topics you cover and don’t know much about the inner workings of X11.
However, I’m tempted to side with them because of the attitude in your responses and still being annoyed at how you brought religion into that one article.
You’d likely have gotten a lot more leeway had you framed the X11 comment as personal experience and opinion.
i really thought the claim regarding zfs smacked of FUD. a sensational gloss of a half truth, if you will.
but X11’s slowness over a network is extremely well-known. i’m not sure how much of it is caused by the default libX11 being overly synchronous and how much is intrinsic to the protocol but most X11 programs, when they start, have to endure an enormous number of round trips. they don’t just say “i want to make a window with xxx size and xxx name and xxx color depth”, they engage in a conversation for each microstep. they don’t move on to the next step of the handshake until after they’ve received an acknowledgement of the first step, even if the steps are all known ahead of time. it basically amplifies latency.
there are novel approaches like libxcb that might be able to make some improvement on that, but by and large if you actually use real X11 programs remotely, it’s not subtle. start up is often agonizingly slow and depending on which widget toolkit is in use, regular user interactions can be slow too. i know i gave as an example the ability to use ‘xloadimage’ remotely as an advantage, but it’s almost the only program i use that way, and even so i can tell you it is slow…in most cases, on my network, slower than ‘cat foo.png | ssh laptop “cat > foo.png; xloadimage foo.png”‘. much much slower than running xloadimage over vnc.
My limited experience with X11 remoting is that the experience is fantastic, even over a thin straw, if you’re only using the really simple (“ugly”) clients that are 1bpp (or almost) and are just drawing text and lines. And have enough memory for backing store. Absolutely no bitmaps, which is why almost all modern X11 (but also RDP and VNC, even over a high-bandwidth high-latency pipe) forwarding is painful.
The early 68k-based NCD X terms were just slow due to the 68k; the later MIPS-based ones were rather pleasant.
Indeed, it’s pretty performant if you stick to the basic widgets and shapes that X11 directly supports. Once you go beyond that and even just try to use applications that use e.g. the KDE or WxWidgets libraries, things can get painfully slow pretty fast.
I have seen it myself on remote X-sessions routed across a VPN (home office things), which worked great until those moments when a bigger refresh was needed and you could easily take that coffee break.
I believe that the wording of the paragraph when I was writing my comment didn’t make it clear (as it does now) that you compare performance over a network. If this context was there and I missed it, I am sorry.
Though I’d still argue that an X session without a hog like a web browser is as painful as you describe it.
Kind regards,
s
The ‘remote session’ context was there in the article from the beginning. No worries if you simply misread it. If I thought that X11 sessions were truly as horrible even over a LAN I’d never have bothered setting up X terminals (with PXE boot) for a school project. Depending on the applications you run and desktop environment you force the client to render the experience can range from ‘butter smooth’ to ‘kinda janky’ on LAN.
It’s when you take ‘kinda janky’ from the LAN to a session routed (via SSL/VPN) over the Internet that things degrade fast, and you get the session struggling to keep up with the drawing. That is what I was referring to. I’m sorry if I wasn’t abundantly clear in my wording :)
I guess it does not support XRDP then?
Does it support VNC Server?
I’m not using my RPis much via a GUI but I like to be able to use the GUI remotely when I need it.
There’s nothing X can do that Wayland can’t, you just sometimes need different tools to do it, e.g. wayvnc or KRdp. Or just use Raspberry Pi Connect which the article already mentions.
From what I understand you can install a VNC server (as usual), and depending on the Wayland compositor you may get some RDP options baked in too, but Labwc does not seem to support any RDP features.
Since it’s based on wlroots I think it should work with wayvnc for VNC access. Wayvnc works pretty well for sway at least (which is also wlroots based).
I miss screen savers in Wayland. I have no need for them, I just miss them. ;)
Labwc broke realvnc. Am I the only one ? Google search shows nothing about Labwc breaking realvnc. But then realvnc did not work off the bat for Wayland.
Does not break raspberry pi’s 4 and 5. Only effects 3, 3+, and 2’s
Because Wayland is SOOOO much more efficient than X it only runs on rPi 4 and 5 and anything before those has to get a reduced feature version of Wayland. What’s not to like about that steaming pile and who is pushing this since it sounds like something Microsoft would be wanting. ie a worst Linux experience.
Hmm. Was it written by Lennart Poettering?
Old hardware getting less active development is pretty normal in all sectors. Be glad that it’s Linux and receives any updates at all, 12 year old commercial offerings rarely do.
Pardon my ignorance, but aren’t GUIs just overlays to the CLI? All GUIs upon close inspection are decorative. You can buy a pre-decorated Christmas tree or a Christmas tree and boxes of ornaments and lights. A person can go to a field and chop down a tree and hand make ornaments and strings of lights. Others can blow glass, smelt metal, make paper, bake cookies …
From each according to their abilities, to each according to their wealth.
HaD gives me a sampler of all these activities – though dear HaD writers – you have come up short in the cookie recipe department this year.
For those saying X11 is trusted, stable; etc. etc. you do realize that what few devs working on it aren’t sure how long they’ll be able to continue keeping it that way and are basically keeping it on life-support at best.
But don’t take my word for it.
https://lwn.net/ml/fedora-devel/2673fbfa-4d5d-4b1a-8cfe-526ef78d8ef8@fedoraproject.org/
So what? Code doesn’t go ‘bad’ just because. That is why it trusted, stable. It works. I have code that I’ve not changed in many years (24×7 Scada application) and it just keeps working….
It’ll stop building though. Linux is pretty stable but it’s a moving target, the kernel is changing, its libraries are changing. Your scada application is not a good comparison.
yeah the upside, for better or for worse, is that people appear from nowhere with drive-by patches when core packages are broken.
I was disgusted to read about how inefficient and antiquated the X Window system was because it was built of 2 parts(client / server) and network protocol based and how great Wayland is because it is integrated to then hear Wayland is so inefficient it runs so poorly on anything less than a Raspberry Pi 4. Why is it still moving forward when one of its primary purposes, to be more efficient, has been proven wrong?
Don’t worry. XFree86 will save us!
just from my superficial awareness of both topics, i don’t think the problem with rpi3 and wayland is the fault of wayland. the raspberry pi developers made a number of unconscionably bad decisions in their rush to market more than a decade ago. i think it’s actually a fascinating story in software engineering mistakes.
they have a closed source firmware which they don’t genuinely have the expertise (or perhaps support from broadcom) to develop or maintain. and then on top of that they have an extremely thin layer of open source drivers in the kernel. the intent of those drivers was to expose absolutely as much as possible of the firmware features to the open source userland, but without documentation! and then there’s a userland which is a carefully balanced hack on top of that overly-general interface to the overly-special firmware.
the correct development pattern for kernel drivers is not to expose all of the features and warts of the underlying closed implemention, but rather to expose a standardized interface. that way, if you fix bugs or add features in the firmware, the standardized interface still works…nothing “above” that interface (open source userspace programs) needs to change. but the approach they used at raspberry pi guarantees that any tiny change in the firmware will have to be matched by a change in the userspace.
in some sense, it seems like a meaningless philosophical distinction but the reality of working with pi as an ‘open source hacker’ is really stark. it forces an insane dependency between microsteppings of firmware revision and the brittle hacks that form the userspace stack that rests on top of it. they need unique firmware and userspace hacks for the rpi3, and because the firmware is so closed and only uses non-standard interfaces, the community can’t contribute. they don’t leverage open source. we only get what they give us.
to their credit, the developers at raspi know this. they are trying to correct this mistake. one step at a time, they are switching to more standardized interfaces. but the transition itself provides its own pain. the defacto standard of “how it worked yesterday” is thrown away even as the true standard of “a stable, well-defined interface” appears on the stage.
i can’t imagine the hacker who developed ‘vchiq’ like 15 years ago had any idea the weight of that decision. i assume the mistake was made in the morning and the implementation in the afternoon and it seems like they made it to rpi3 before they even suspected the mistake might have downsides.
I use remote graphical X all the time and am completely happy with it. My entire business and thousands of customers are critically dependent upon it.
“Unless you’re one of those people who use features such as (remote) X-sessions”
/Raises hand./
The transition is comlpleted … my machine won’t boot now, but good to know …