On Tuesday, December 8th, Red Hat and CentOS announced the end of CentOS 8. To be specific, CentOS 8 will reach end of life at the end of 2021, 8 years ahead of schedule. To really understand what that means, and how we got here, it’s worth taking a trip down memory lane, and looking at how the history of Red Hat Enterprise Linux (RHEL), CentOS, and IBM are intertwined.
First, History
Red hat started way back in 1995, with the partnership between Bob Young and Marc Ewing. Ewing brought his nascent Linux distro, named Red Hat Linux after the fedora red lacrosse cap Ewing was known for wearing. Red Hat Linux quickly introduced a set of killer features, such as the Red Hat Package Manager (RPM), the Anaconda installer, and ELF binaries, to name a few. By 2003, Red Hat Linux was split into two separate distros, RHEL and Fedora Core. RHEL was the subscription-only distribution, while Fedora Core was the bleeding-edge distribution available for free. Just a note, I was running Fedora on my machines since before they dropped “Core” from the name.
The RHEL product, while open source, is only available for paid subscribers, or developers in non-production environments. Because it’s open source, there is nothing preventing a third party from removing the branding, and recompiling the packages for free. This is exactly what Gregory Kurtzer and the other founding members of CentOS did back in 2004. CentOS version 2 was the first such release, bringing an Enterprise Linux to the Community.
The next bit of history we have to cover was in 2009, when Lance Davis went missing from the project. This was a problem, because Davis held the project domain registrations, the donations fund, and some of the other intellectual property of the project. There was a possibility that this would kill the project, but Davis finally responded to a published open letter from the project, and agreed to hand over control. Crisis averted.
Becoming Respectable
In 2014, an important change was announced. Red Hat would officially sponsor CentOS. Several of the core CentOS team would go full time on the Red Hat payroll, Red Hat got a trio of seats on the governing board, and most importantly, the community was assured that “The CentOS Linux platform isn’t changing.” It wasn’t made entirely clear, but part of the deal included transferring the CentOS trademarks and IP to Red Hat for safe keeping.
This arrangement worked out well for years. CentOS 8 shipped September 2019, rapidly after RHEL 8, and CentOS Stream was announced and released as an early look at what was coming in the next minor update. CentOS Stream could be thought of as a polished beta channel.
IBM bought Red Hat in 2019. Part of the announcement was a promise that Red Hat would stay true to its Open Source roots, claiming that “Red Hat’s mission and unwavering commitment to open source will remain unchanged.” Even with such assurances, many users were concerned that the IBM acquisition would fundamentally alter the way Red Hat does businesses for the worse.
Where Are We Now
With this context in mind, the recent news is troubling. Many of us have deployed CentOS servers to production environments, trusting Red Hat’s promise that CentOS 8 would be supported through 2029. The Red Hat announcement means that those installs reach end of life after less than 2 years, with the only upgrade path being Centos Stream — AKA being a forced beta tester for RHEL. As you might imagine, the response from the community has not been positive. We feel betrayed and lied to. It’s unclear whether this decision was handed down from IBM, or was cooked up by Red Hat themselves. In any case, it’s become painfully clear that handing over control of CentOS was a terrible mistake.
For those of us who prefer RPM based systems, where does that leave us? Fedora is great, but the rapid development cadence is terrible for server deployment. OpenSUSE Leap is a rebuild of SUSE Linux Enterprise Server, much like CentOS, and is a viable option for new deployments.
In the comments on the CentOS announcement, Gregory Kurtzer asked for users and developers interested in a reboot to check in on one of his slack channels. Within 8 hours, over 250 of us showed up, wanting to make a CentOS replacement happen. I’ve talked directly with Gregory, and can confirm that a new community rebuild of RHEL is happening under the name Rocky Linux, in honor of one of another CentOS co-founder that has passed away. The plan is to support a direct transition path from CentOS to Rocky Linux.
Rocky Linux will be bug-for-bug compatible with RHEL. It will be CentOS as it was before the Red Hat takeover, just under a new name. I’m excited to see where the project goes, and hopeful to see an initial release in the following months. When it happens, we’ll be sure to tell you about it.
CentOS is dead… So long CentOS…
My servers are Ubuntu most of the time. Some are Debian. I grew up using Debian and Ubuntu, so it’s personal choice, but CentOS was well regarded by me. I respected it a lot, sometimes more than my Ubuntu LTS servers.
I hope Rocky Linux gets traction and fills the gap.
I tried centos in real environment’s and found them very lacking, from dodgy kenel upgrades to complete failure to boot due to packages not installing fully, then came zentyal which i had hoped would fix these issue only to add much worse problems like storing all the system settings in a sql DB then deploying that at boot time, this ment any fixes applied to the system would be reset at boot unless you patched the DB or the deployment system and if you needed sql in your environment you had to deploy two just to function as one is dedicated to the config system. Then zentyal went the way of clearos, paid app market where all the basic options were removed (mail, ftp, samba, sql) and had to be installed via online portal which would break if there was any failure to download. Since then we never deploy anything centos/zentyal/clearos based even if the client ask’s.
darkspr1te
That is one of the most annoying things about Linux – the sheer number of distros.
Although I loathe M$ operating systems, at least there was “just” one distro (well, at least when it started… then there were home editions, server editions, weekend editions…)
I consider it a feature, not a bug. :)
If “everybody” cooperated and worked on just one distro, imagine the progress that could be made!
https://xkcd.com/927/
> Ewing brought his nascent Linux distro, named Red Hat Linux after the fedora Ewing was known for wearing.
As far as I know, the name of the distribution comes from a Lacrossse hat that he misplaced at the time of naming the distro, so it was on his mind then. Fedora hat didn’t appear until much later, then the logo was changed from a red top hat to the shadowman that we all know and loved (until last year, when it was changed again).
This is also confirmed here: https://www.redhat.com/en/about/brand/standards/history
Where did you get the idea that it was a fedora?
Ah, right. It’s been a while since I read the story last.
> Where did you get the idea that it was a fedora?
Here’s a wild guess:
https://en.wikipedia.org/wiki/Red_Hat#Fedora_Project
Should have called it DollarOS or 50CentOS
Any references to IBM’s trademarked “CentOS” would be an invitation for a lawsuit.
call it ness-os, to represent that darn old lock ness monster asking for treefiddy!
I aint giving you no tree fiddy. Goddamn loch ness monster
Now if only systemd is ripped out if Rocky that would make my day.
It has to support all of RHEL’s bugs, so this is unlikely. :D
Perhaps you forgot, but there is another RHEL clone: Oracle Linux. They did the same thing as CentOS, but one of the major differences is their default substitution of UEK for the RHEL Kernel. UEK does break a lot of things however, so it’s not for everyone.
TBH, I’ve run Fedora Server 32 and CentOS 8, I’m finding Fedora Server 32 to be much more compatible and appropriate for server installs. The footprint is very small and it leaves the rest of the LV group empty, so you can deploy VMs, docker containers, or other filesystems. CentOS has always gobbled up the entire storage volume, having a bunch of unallocated VG is a welcomed change. I also find FS 32 to be very handy with Copilot installed, it’s like the headless counterpart to IPMI and makes getting new servers setup easy (I’m sorry, but since Networkmanager came about, setting up network interfaces has been a PITA for me).
NetworkManager is a learning curve true. However it is tied in to the OS at this point as other products rely on it. Initially it was horrendous but in CentOS 8 and RHEL 8 it is amazingly convenient. No more multiple ifcfg files for bridges, nics etc with settings for everything all over the place. nmcli con show shows just how much you can set up and configure in one single location now.
I’ve deployed Oracle Linux in production and it works for me and the web applications my team deploys. One reason I switched was that CentOS required too much work to make well known services like syslog to work in my environment. With Oracle Linux, services that I expect to be there, are just there from the get go.
IBM getting hold of RH was the embrace. System Derp was the extend. Now we see the extinguish. Can’t we fork from 6.10, get a modern kernel and XFS, and leave systemderp far, far behind?
The moral of this story is: yes you can, there’re no obstacles.
Laughable claim, since systemd is 8 years older than IBM’s purchase of Red Hat. Also systemd is nice actually, and I’ll never understand why some people argue that a pile of loosely-coupled shell scripts being run sequentially to start the system is better.
The reason might be because it actually works, doesn’t hog system resources or trash log files… Seriously, why some of you think it’s not torture is beyond me. Troubleshooting a bug on a system that’s running systemd is quite often an act of futility. I still likely have a hard drive image of the last version of Fedora I ran about 6 years ago before I jumped ship to Gentoo if you’d like to try to diagnose the mess systemd left for me. The problem started with postfix not connecting to mysql and by the time I was done I’d discovered that systemd had trashed every useful log(and IIRC a few config files it shouldn’t have had any hand in) I’d had and made it utterly impossible to recover from….and I tried for over 6 months to get the system operational again but it was FAR easier to move to Gentoo, and that’s saying a lot. And in the 6 years I’ve been using Gentoo I’ve had considerably fewer issues with my system and exactly NONE have involved openrc/init.
SystemD is the Microsoft way of thinking introduced to Linux.
So go figure it’s a overly complicated and obfuscated mess with rampant feature creep that makes even a bunch of hodgepodge scripts seem amazing and reliable.
Have you heard of the lord and savior OpenRC?
I will agree that I don’t like the feature creep of SystemD.
No, it’s the OSX way of thinking. systemd done [more] properly is launchd. We need to try to improve systemd, not just dump it. Dump the things that don’t work, keep the things that do. I suspect this is more nostalgia for SysV than anything else.
I have to force myself to believe that anyone claiming systemd is an init system is arguing in good faith. It’s much more than the framework that starts the system, it has become the system itself. Claiming “it’s better than a bunch of shell scripts” is disingenuous at best.
grumble. for one, it’s the unix way. every program should do one thing and loosely-coupled shell scripts should bind them together. that’s not for everything — i use android too, you know — but if you don’t like it, unix doesn’t have much to offer.
but the deeper fact of the matter is that systemd gets in the way of loosely-coupled scripts that have worked for decades. systemd doesn’t just do everything, it actively destroys everything that isn’t systemd. you can’t have any corner of your system that isn’t thoroughly described to systemd, or systemd will simply kill your processes. and there’s no easy “-f” sort of way to work around it, you can’t say “i know what i’m doing, simply don’t kill this process.” you have to bend to their way of doing things entirely.
and then the kicker, which really bummed me out when i discovered it, is that systemd is overdesigned and underdesigned at the same time. it’s so overdesigned that it’s hard for a new user to get up to speed with even the basic concepts, and then so underdesigned that even after you learn the multiple redundant different ways to accomplish what you desire, you find that it’s simply not possible to get there from here. there are ridiculous magic hidden configurations all over the place, decisions that have traditionally been exposed to the administrator in shell scripts are now buried in C code. you can’t readily see why it’s doing what it’s doing, and you can’t readily change it either.
to add injury to insult, poor decisions predominate. not only is it hard to change anything from the narrow vision of the systemd architects, but their taste is actively bad. i let debian install systemd on a laptop and i bent over backwards to accept the systemd way of doing things, only to discover some months later that it had filled the whole filesystem with a bloated logfile that never rotates. it doesn’t even “just work”, now you need to learn a dozen *new* details just to get logs to rotate on a system that sleeps overnight. in the old way of things, i could rotate them by hand, or i could use cron and logrotate (which are both ancient), or i could search for or invent a novel technique. but with systemd, you have to learn how they meant it to be done and if it’s done any other way then it will actively break everything.
the axiom that i’ve learned from overdesigned software in many contexts: people who will fix what isn’t broken won’t stop. once they’ve realized they can in one step force you to learn a replacement for dozens of utilities you’ve been using for 30 years, they will do it again. if you are so eager as to become an expert in systemd, LOOK OUT: it won’t last. by 2030 someone will replace systemd with “something that finally gets it right”. that one won’t last 5 years. this is the single most disturbing trend i see in software development on all fronts. and this disaster is only possible when there is a single ring to in the darkness bind them. the unix tradition of isolated tools allowed us to evolve over time, but now hyper-integrated tools result in constant revolutionary changes.
Can you go have a talk with kde about activities
From a fellow Linux user, condolences. It appears that money always corrupts. Thankfully, there are many distros, so money cannot corrupt all of them at the same time.
To bad. I bet there is some unhappy people with the cut off on version 8…. Back when, I used RedHat and CentOs as our work file server OS, they were rock solid once setup. One system ran for 7 years just doing it’s thing until I retired it. My recollection says it sure beat Windows NT and all the user licensing I had to buy/deal with, also rebooting every week or two to free up resources,,, Loved using RedHat and then CentOs for file/print serving. Even used it at home for my home file server. Again, just set there an ran and ran serving up files. Never a major issue with the OS I can recall. Only reason I finally got off of CentOs at home was AMD Ryzen came along. CentOs did not support the new hardware. So made the switch to latest Ubunutu. Now all my desktops, laptops, and server all run (K)Ubuntu LTS 20.04 . Good for 5 years.
CentOS Stream is not a public beta and is more directly tied to RHEL than CentOS was. I have already migrated servers to it already with little hassle. One thing that will be much better than CentOS is the lack of delay between when updated packages, security fixes, etc will be available. Instead of months behind (as CentOS was) they will be available almost immediately. While I am not happy about how this was announced and the fact that it wasn’t done before CentOS was release and before folks migrated from CentOS 7 to 8, I am still sticking to it over any alternative. Especially one “not yet formed”.
correction… While I am not happy about how this was announced and the fact that it wasn’t done before CentOS 8 was release and before folks migrated from CentOS 7 to 8, I am still sticking to it over any alternative….
Yes, agreed. Had this been announced before 8 was released, I would be much less frustrated.
Just switch to Slackware already
Companies I have worked for in the past have used CenTOS to build and test and then licensed RedHat in production. This will no longer be a viable path so folks will move to debian/ubuntu who also have fantastic support.
Why not? This seems to be exactly what you would do with CentOS Stream.
Yeah that could be the case. Problem with debian and Ubuntu they dont integrate with AD very well. Centos was smooth operating system. But redhat decided they wanted to cut off the dragons head thinking it will help them get customers. There is scientific linux as well. People wont stand for outragous pricing for open source code. I hope RockyLinux fills the gap quickly and that follows a dramatic bleed of redhat as a result of their greedy move with no regard for millions of users.
Red Hat introduced ELF binaries?? But what was the binaries like before on Linux?
Linux used a particularly feeble a.out implementation until 1.2 in 1995. https://en.wikipedia.org/wiki/A.out
Thanks ! #TIL
C code
C is sourcecode not binary
what about Scientific Linux?
They opted not to do a build of 8, and just use CentOS. The guys at Rocky are already reaching out to the Scientific Linux team.
Weren’t they part of CERN?
CERN and Fermilabs cooperated.
In the academic space, there is also Springdale Linux (formerly PUIAS Linux).
Happy hacking
Choice is good and all but I do wonder how many fractal flavours and sub-flavours of linux the world needs
Meh.. RedHat ceased to be cool all the way back when Mandrake was first released.
Thanks for the history – years of drama-queens and corporate incompetence. Now I have a 2d-party ‘opinion’ that I can point to for why this distro has never been suggested to any of my clients. I will continue to gently prod and point to vanilla Debian for the factory environment.
Yes, I am a cranky old geezer. My personal machines run Slackware without any of the drama. You kids get off of my dirt…
CenTOS is dead long live ROCKY!
https://www.theregister.com/2020/12/10/rocky_linux/
Come now, quit all the crying. There’s only one proper alternative: http://hannahmontana.sourceforge.net/
The song is very disappointing, where is the audio and MTV video?
In fact, we enjoy the privilege to send payment to anyone from any part