When the PlayStation 3 first launched, one of its most lauded features was its ability to officially run full Linux distributions. This was of course famously and permanently borked by Sony with a software update after a few years, presumably since the console was priced too low to make a profit and Sony didn’t want to indirectly fund server farms made out of relatively inexpensive hardware. Of course a decision like this to keep Linux off a computer system is only going to embolden Linux users to put it on those same systems, and in that same vein this project turns a more modern Playstation 4 into a Kubernetes cluster with the help of the infamous OS.
The Playstation 4’s hardware is a little dated by modern desktop standards but it is still quite capable as a general-purpose computer provided you know the unofficial, unsupported methods of installing Psxitarch Linux on one. This is a distribution based on Arch and built specifically for the PS4, but to get it to run the docker images that [Zhekun Hu] wanted to use some tinkering with the kernel needed to be done. With some help from the Gentoo community a custom kernel was eventually compiled, and after spending some time in what [Zhekun Hu] describes as “Linux Kernel Options Hell” eventually a working configuration was found.
The current cluster is composed of two PS4s running this custom software and runs a number of services including Nginx, Calico, Prometheus, and Grafana. For those with unused PlayStation 4s laying around this might be an option to put them back to work, but it should also be a cautionary tale about the hassles of configuring a Linux kernel from scratch. It can still be done on almost any machine, though, as we saw recently using a 386 and a floppy disk.
Since the PS 4’s OS is based off of FreeBSD, wouldn’t it seem to be easier to get a general purpose version of FreeBSD running?
it would make sense to install FreeBSD or Linux on a common PC.
No because the problem you face is developing drivers and you need people for that. There are more people that know how to develop Linux drivers than there are for developing FreeBSD drivers. If the PS4 ran Linux and FreeBSD were more popular then you would be stuck with the same problem.
No one would have to write any drivers I think that is Justin’s point. Sony all ready wrote all the drivers you need for freebsd 9 (The PS4 runs a modified version of freebsd 9) you might even be able to reuse their kernel.
Most of these projects want absolutely nothing to do with Sony’s code besides what is absolutely necessary to boot. Keeping them clean lets them stay up.
i think a core difference between freebsd (BSD license) and linux (GPL 2.0) might be in play. there are exceptions (things dynamically loaded), but generally changes to the linux kernel have to be distributed to the customers, while BSD has no such requirement.
if they’d chosen linux, then their open source components would provide a great starting point for developers. for example, getting a custom linux kernel running on android and chromeos machines is easy because of all of the open source work that google has contributed. but BSD license doesn’t give you this leg up. all of the valuable work they’ve done that you’d like to leverage can be hidden from the public without recourse.
Docker is officially broken on FreeBSD, the whole point of this arrangement is to be a Kubernetes cluster. Since BSD ports can be closed source it really does not follow that anything is “easier” on BSD.
The honest reason was that I never used FreeBSD before and Linux Images were a lot more accessible. For context, a lot of the gaming stuff included in the original Arch image is still not uninstalled to this day.
FreeBSD… What about trying to port Mac OSX to it. :P
I don’t believe the PS4 OS is actually based on FreeBSD only using the filesystem code (I think!).
A presentation was made some years back about running Linux on the PS4, by Fail0verflow (formerly Team Twiizers, well-known for its GameCube and Wii hardware exploits.) The gist to running Linux on it was that the PS4 is an x86-based machine, but it’s not an IBM-compatible PC. x86 Linux expects a number of things to be present in hardware (such as a specific system timer) that are just not there and needed other elements to sub in for those missing elements on the PS4.
You can bet that if Linux needed heavy modifications to run, then FreeBSD will need those same modifications too. Since x86 Linux and x86 FreeBSD share this “IBM-compatible” trait, I don’t see FreeBSD being any easier to run than Linux. And that’s before talking about issues such as of non-existent drivers and lack of devs to make them…
Here is the presentation, by itself it’s about 40min on a caffeine high, then there’s a Q&A: https://youtube.com/watch?v=VpB49dhk2uQ
If I remember correctly, the ability to install software on the PS was only offered up to get around some BS tax on game or entertainment devices.
Sony and others continue to do this on cameras, crippling them to record less than 30 minutes at a time to avoid a European tax on “video” cameras. These idiotic legislators arbitrarily decided that video of under half an hour isn’t video. Meanwhile, users get screwed and miss recording critical events when their cameras shut off without warning for no other reason.
I think that 30 min limit is heat based. If you actively cool the CCD/CMOS chip, you can record for longer on some devices. There’s a YouTube video I can’t find right now that watercools the sensor. Maybe it was DIYPerks. There are other solutions for astrophotography that might also work. The motivations are different, but the goal is the same: cool the sensor.
https://skyandtelescope.org/astronomy-resources/beating-the-heat-in-astrophotography/
It’s a hard limit in the firmware to avoid EU camcorder taxes. I have a Nikon Coolpix S6200 that records excellent 720p video but it stops recording at 29 minutes. Pro Nikon cameras also have the 29 minute limit but there are hacked firmwares for several of them with limit removed. Would be nice if the hackers would turn their attention to the point and shoots.
I have another camera of similar size, different brand, with a 30 minute limit. Apparently Nikon chose to cut off at 29 minutes just in case some EU government busybody decided their law meant that the dividing line between camera and camcorder is 30 = camcorder, 29:59 = camera. A third point and shoot I have does not have the limit but its best video resolution is only 640×480.
I’m willing to bet it’s something the movie industry managed to get lobbied into law.
Pirate recording in cinemas on release day and whatnot.
Since they also managed to lobby some sort of storage tax with the excuse of Piracy.
It’s actually driven by the tax. Given that, because of the tax issue, the hardware will “never” be used for longer durations, the engineers saved additional money by reducing the cooling solution.
The better-engineered cameras acutally will run much longer than 30 minutes, reliably, with hacked firmwares. The cheaper-engineered cameras often hit their design thermal budget at almost exactly the maximum duration the original firmware would allow anyway.
In this case, causality is “tax avoidance, then reduced cost of cooling hardware”, not the other way around.
Maybe it’s both.
If you know that you are going to put a hard 30 minute limit in software then why engineer the hardware to be able to stay cool for > 30 min? Especially if that makes it any more expensive to produce. If a smaller heat sink saves a penny and you expect to sell a million units then that’s a million cents.
Officially the reason for Sony removing “Other OS” from the PS3 was someone figure out a way to leverage that for cheating in online multiplayer games. So of course rather than come up with a fix for their online service to block that, Sony took away “Other OS” from *everyone*.
Nobody had figured out a way to cheat. Sony was afraid someone might.
Sony was concerned that all these highly technical linux users might find a flaw in their hypervisor that separated linux from the games. So, they ripped out the linux feature.
Of course this pissed off all those highly technical linux users, who had exactly the skills required to find a flaw. Now sony had provided them with the motivation as well, and it wasn’t long before they found a way to run linux again. Sony also got sued for this as well.
And after looking up the result of the lawsuit: Sony settled it quickly for $4m instead of spending many times that on a drawn out lawsuit and everyone that pretended to have used OtherOS and sent in the paperwork got a whopping $10. The claimants apparently didn’t actually care enough to appeal the settlement and get a full refund.
Those of you astute enough to remember the above incident will probably remember where Sony attempt to fight back against the ‘hackers’, attempting to sue George Hotz in the US under DMCA and other dubious ‘computer hacking’ charges and even some in EU (where DMCA did/does not apply the same AIUI)
Really the only success they got is thanks to Hotz having an incompetent lawyer who couldn’t defend against Sony legal team was an agreement for Hotz to never hack on Sony products again. Some other people got scared off while others continue as before.
Still makes me angry to this day…
Sony wasn’t making money from the Consoles, they got money from the game sales. Also there was a way to run pirated games using the “Other OS” option so Sony pulled the plug. This pissed off the U.S. Airforce since they had built a server farm with PS3s.