The best part about real-time OS (RTOS) availability in 2024 is that we developers are positively spoiled for choice, but as a corollary this also makes it a complete pain to determine what the optimal choice for a project is. Beyond simply opting for a safe choice like FreeRTOS for an MCU project and figuring out any implications later during the development process, it can pay off massively to invest some time up-front matching the project requirements with the features offered by these various RTOSes. A few years ago I wrote a primer on the various levels of ‘real-time’ and whether you may even just want to forego an RTOS at all and use a simple Big Loop™ & interrupt-based design.
With such design parameters in mind, we can then look more clearly at the available RTOS options available today, which is the focus of this article. Obviously it won’t be an exhaustive comparison, and especially projects like FreeRTOS have seen themselves customized to various degrees by manufacturers like ST Microelectronics and Espressif, among others. This also brings to the forefront less pleasant considerations, such as expected support levels, as illustrated by e.g. Microsoft’s Azure RTOS (formerly ThreadX) recently getting moved to the Eclipse Foundation as the Eclipse ThreadX open source project. On one hand this could make it a solid open-source licensed RTOS, or it could have been open sourced because Microsoft has moved on to something else and cleared out its cupboard.
Thus without further ado, let’s have a look at RTOSes in 2024 and which ones are worth considering, in my opinion.
Answering Some Questions
A crucial distinction when looking at operating systems for embedded systems is the kind of platform it is. If it’s something along the lines of an x86, Cortex-A ARM or similar, you’re likely looking at a desktop-like system, where a real-time OS such as VxWorks, QNX, a BSD or Linux (with or without real-time patches) is probably the best choice, if only due to hardware support concerns. For situations where hard real-time considerations are the most essential, an FPGA/CPLD-based solution might instead be worth it, but this is of course less flexible than an MCU-based solution.
If at this point an MCU-based solution seems the most sensible one, the next logical question is ‘which one RTOS?’. The answer to this is hidden somewhere in long lists of RTOSes, such as the one found over at Wikipedia, or the one over at the OSRTOS website. Assuming for a moment that we are looking only at open source RTOSes here that are seeing active development, we can narrow it down somewhat to the following list:
- Contiki-NG
- OpenERIKA
- FreeRTOS
- MicroC/OS (Micrium OS)
- Nano-RK (Atmel Firefly, MicaZ motes, MSP430; GPL or commercial)
- NuttX
- RIOT
- Rodos
- RT-Thread
- TI-RTOS
- TizenRT (NuttX fork by Samsung)
- Zephyr (formerly Rocket)
- ChibiOS/RT
- Apache Mynewt
Of note is that the popular Mbed project was abandoned in July of 2024, rendering the future of this RTOS highly uncertain. Even with that one taken out of the picture, we are still left with an impressive list. Is NuttX better than ThreadX? What about Zephyr versus RIOT or ChibiOS/RT? Merely reading the bullet points for the features gets one only so far. Perhaps the most important questions here pertain to issues such as:
- Build system requirements
- Demands on a specific compiler (version)
- Programming languages one can use with the OS
- Whether direct hardware access to peripherals is allowed or require going through a HAL of some description.
- Support availability when something inevitably doesn’t work as expected.
- Accessibility of the source code when reading through it (readability, documentation, etc.)
Baseline Expectations
The baseline for the build environment demands and supported features is set here at FreeRTOS. It runs on a wide range of (MCU) platforms, provides a number of schedulers, SMP support, happily compiles with just about any compiler toolchain and is C-based so can be used with any programming language that can cooperate with C APIs. Direct hardware access is the standard way for peripherals and the OS generally gets out of your way beyond scheduling and multi-tasking matters. This ‘stay out of the way’ approach persists with developer tools and configuration, which works as easily in Vim as in any other editor.
ThreadX
As of writing the documentation is a stack of Markdown files on GitHub which are clearly not converted yet from their Azure OS era, and ‘getting started’ refers to connecting to the Microsoft Azure cloud. Building the library is apparently done using CMake, Ninja and the standard ARM GCC toolchain for ARM targets, but where’s a sample project and what about other MCU platforms?
After confusing myself clicking through the ‘documentation’ for a while, I’m sure that I would not pick this RTOS as I am spending far too much time even figuring out the basics.
Contiki-NG
Documentation exists and doesn’t look too bad, but you’re pushed right into using a Docker image for development. Fortunately native installation instructions are provided for Linux and MacOS, but not Windows. It’s clear that this RTOS is focused on Internet of Things projects, while the ability to easily run the code as a native (Linux) process is nifty.
Feels like this RTOS could be nice for network-related projects.
OpenERIKA
Confusing website and the impression is that it’s ‘free’, but do not expect any support unless you’re willing to pay for it. Hard pass.
MicroC/OS
I’m sure that Silicon Labs had good intentions with their Micrium OS site, but it’s too hard to find anything on it, never mind how to get started with the thing, plus it seems locked to Silabs devices. Ditto for the Weston-Embedded website. Hard pass.
RIOT
Seems to use basic tools and the standard platform toolchains per the ‘getting started‘ documentation. Build system is GNU Make-based, which is very flexible and should integrate with other build systems with little issue. Quite a lot of documentation to dig through, and might be worth scratching an itch with that FreeRTOS doesn’t cover?
NuttX, Zephyr, ChibiOS/RT
RTOSes which have lots of bullet points are kinda fancy, but demanding the use of KConfig with NuttX, the insistence on setting up a special Python-based development with Zephyr and the seemingly hard requirement to use the special IDE with ChibiOS/RT all makes for problematic choices that will make developing with these either impossible — KConfig on Windows — or integrating with other build systems impossible to tedious.
While I’m not a Python hater, my experiences with Python-based build environments and tools are very negative, and I’d rather avoid such unnecessary dependencies in a development workflow. If you’re a Python fan, you might want to look more seriously at Zephyr.
With that said, the remaining RTOSes in the list are fairly small and can probably be skipped safely.
FreeRTOS
Back in my original 2021 article I covered getting started with FreeRTOS, which at the time was focused mostly on STM32 and similar ARM-based MCUs. Since then I have extensively expanded my use of FreeRTOS in the form of Espressif ESP32 development, both on the base ESP32 MCU and the ESP32-S3. This provided a range of interesting perspectives, also since I was porting significant amounts of cross-platform C and C++ code to these MCUs.
An important aspect of FreeRTOS is that it is commonly included in MCU SDKs, as is the case with Espressif’s ESP-IDF. It supports three different versions of FreeRTOS: the single-core ‘vanilla’ FreeRTOS, the Espressif (SMP) version and Amazon’s SMP FreeRTOS. Espressif’s version is optimized for the dual-core design of the ESP32 and ESP32-S3 and the default choice. What this demonstrates clearly is that the strength of FreeRTOS lies in its flexibility. You can slot in any custom scheduler, heap allocation algorithm, and pile on additions that are optimized for the target platform.
ESP-IDF provides partial POSIX compatibility, which uses FreeRTOS primitives internally. In order to port projects based on the cross-platform PoCo libraries, I added FreeRTOS support to these in the compatible NPoCo project. With this approach I can use virtually all of the features provided by the PoCo libraries also with (ESP-IDF) FreeRTOS, while allowing for the exact same code to compile on desktop platforms. The only platform-specific elements (e.g. start-up and peripheral use) are handled by compile-time preprocessor inclusions.
Drawing Conclusions
Although there are many ways to go about developing a project and advocating a particular approach is the best way to end up forever shunned by friends & colleagues, looking at a different approach can be enlightening. Over my own career I have gravitated strongly towards simplicity and reducing potential pain points. A big part of this is finding the optimal ways to do as little work as possible, which is where my own approach to MCU-based RTOSes comes from. I really don’t want to write more code than absolutely necessary, also because new code has new bugs.
As software is incredibly flexible, the real value in an (RT)OS lies in the scheduler, heap allocator and similar elements which provide the primitives on which other features can be built. While many RTOSes seem to go out of their way to (incompatibly) replicate the scope of the entire Linux kernel space & userspace in miniature, this to me at least seems restrictive. What I personally appreciate in FreeRTOS is that you can have as much or as little FreeRTOS in your code as you want, making it extremely hackable.
For others their priorities may be completely different, in which case any of the other RTOSes may work better, which is also perfectly fine. As long as the project is completed on time, within budget and no keyboards were thrown through the room, there are no wrong choices.
I like FreeRTOS so far: it provides all the features I want without a lot of extra bloat and it’s easy to use, since it only requires a few clicks to set up with the STM32 SDK and it’s already baked into ESP-IDF.
I haven’t needed any of the more advanced features of e.g. Zephyr and I’ve never used any of the Pic microcontrollers, so I don’t know what the landscape is over there.
I’m surprised that RTEMS didn’t get a mention here. It’s a BSD-licensed FOSS RTOS that’s supported on both ancient and modern platforms.
Agreed. That was the first entry I looked for.
I had exactly the same thought. Even if the last time I used it was in 1995 on a Philips 68000 variant! I was thrilled to find it still active after all these years when I went looking recently. Always found it interesting that the M originally stood for Missile.
Some comments from my own experience:
FreeRTOS: fine for what it does, but for many purposes you’ll need to combine it with a bunch of extra drivers and libraries and that can be quite a hassle.
NuttX: feels quite familiar when coming from Linux background. The build can be done on Windows, but is indeed easiest to setup on Linux or WSL. A decade back I was constantly hitting bugs with NuttX, but it feels a lot more stable nowadays.
Chibios: well designed hardware drivers. No need to use the IDE, build process is just regular makefiles. Paid license needed if GPL is not suitable for the project.
You overlook NuttX has the ridiculous “feature” of needing different drivers for MCP2515 used on different platforms, not to mention the horrible spaghetti of header files which makes almost impossible to fix dependencies in a reasonable manner
Basically, NuttX is a scam conducted by a gang associated with its main developer
Grr, sorry to say but you are completely wrong here, NuttX has only one MCP2515 driver that likes at drivers/can/mcp2515.c. Probably what you saw were the board specific bounding connecting arch specific SPI drivers to the generic MCP2515 driver. When you don’t understand the basic principles of the system it becomes hard to use. But as JPA said: NuttX is very easy for people coming from embedded Linux background!
I’ve used or evaluated most of these over they ears. They’re all fairly good. And getting one to work with a product, assuming the feature set matches your needs, boils down to investing the time into learning the OS and its tools.
ThreadX is great and when it went open source it was great news for some.
NuttX/Tizen is pretty straight forward and good for getting something up and going pretty quickly.
For closed source there are a lot of good RTOSes out there. Nucleus RTOS is still great. I somewhat miss working on the paid version of eCos. I don’t miss VxWorks, mostly because of the add on components are so bad and not worth the money. QNX is of course fantastic to develop on if money is no object on your project.
Must agree on ThreadX. The OS is available to selected STM32 chips via STM32MX tool, and we found it to be lightweight and memory efficient. I still prefer FreeRTOS just because I’ve used it long enough that I have muscle memory for its idioms and naming conventions.
I used VxWorks on a VME crate when working at CERN. It was decent, but I tried to avoid it. Since then, there seems to be lots that is better.
I’ve kept an eye on Lynx, they do a range of “aviation-certified” RTOS’, but I’ve no clue if they’re actually any good.
Do you know of any reliable sources of information on the various RTOS? FreeRTOS has books, SEL4 (as far as I know) doesn’t, CapROS barely has a website, and so feature comparisons are quite difficult.
I was the ‘platform’ person on a large optical node in 2000, running QNX Neutrino on IBM PPC 750 and Freescale MPC850. Only the main controller ran the PPC750, all the blades ran on the MPC850. To enjoy QNX, you really need to dive into the QNX mindset, but once you do, things like device drivers running in user space are just so awesome. I wrote a driver for a very complex 40Gb framer, and doing that same driver in NDIS or under Linux would have been hell. In QNX I had a resource manager framework in place in two weeks, and then after that it took about two days for each device feature/option.
The fact that a process on a blade in a node, could open a resource manager on another blade in another state over the optical ring, was magic.
Sounds a lot like plan9 / 9front
Here’s the GitHub page for uc/OS-III.
https://github.com/weston-embedded/uC-OS3
and here’s where to find the excellent free uc/OS-III book downloads.
https://weston-embedded.com/micrium-books
The uc/OS-III documentation is fantastic and the code is some of the cleanest you’ll find.
Yeah, not sure why they went so hard against it. It’s a really respectable RTOS with a lot of history.
Yes, I was surprised by the author’s impression. I got the book waaaay back when it came with a diskette and source code. You can’t get much better documentation than watching the design from scratch.
Thank you for the thoughtful discussion and for highlighting Micrium’s uC/OS. We are incredibly proud of what we achieved with Micrium: clean, reliable, and trusted software that has even been part of space missions (such as the Mars Curiosity rover’s analytical lab) (https://www.businesswire.com/news/home/20120914005987/en/Micrium-Chosen-for-Mars-Curiosity-Analytical-Lab)
Today, at Weston Embedded Solutions, we’re building on that legacy with our own fork of uC/OS, which we now offer as Cesium RTOS. Our focus is on evolving the software to meet today’s real-time requirements while preserving the quality and reliability that uC/OS users have long trusted. You can learn more about Cesium RTOS and our latest innovations here: https://weston-embedded.com/products/cesium
Didn’t see any mention of RTEMS. They’ve updated their site recently as well.
https://www.rtems.org/
Didn’t see any mention of RTEMS. They’ve updated their site recently as well.
https://www.rtems.org/
I have used ChibiOS in the past and it is normal Make-based build system. There are also quite a few CMake wrappers for it around. The ChibiStudio IDE is certainly not required – I have never used it myself. No idea where did author come to the notion that this is a hard requirement? The website doesn’t say anything of the sort.
I regularly use ChibiOS and never used the ChibiStudio IDE. It is definitely not required. It can help you set up the IO pin configuration for your board, but you can easily do that by defining them in an XML file instead.
One thing to note is that ChibiOS comes with an extensive HAL (Hardware Abstraction Layer) that really earns that name, in that the hardware of the supported microcontrollers is abstracted in a vendor-neutral way. So porting a project for example from a STM32 to a RP2040 becomes quite easy.
There is a free book-style documentation available which gives a good introduction:
https://www.chibios.org/dokuwiki/doku.php?id=chibios:documentation:books:rt:start
FPGA
Another one is RTE/MS. Besides having the most hard-core acronym (originally Real Time Executive for Missile Systems, and yes it started out as DoD rocket science.) Used in several sattelites and space probes. Hard (very deterministic) real time, continuing if low-key development, free mailing-list-level support with commercial support avail. Distributed and heterogenious–e.g. ARM and x86 processors/threads in the same system. C/C++/Ada. Slightly patched GNU toolchain. POSIX, FreeBSD drivers support. Documentation decent. Lots of different BSPs, including RPi (though limited HW features support, no put-it-in-hard-real-time mode for the BSP).
FreeRTOS : Simple workhorse, easy to work with, simple to setup and just works. Don’t mind writing the drivers at all, since everything works as expected.
OpenRTOS : Same as above, but i like it less.
ChibiOS : My brain thinks of it as FreeRTOS but with more complex networking options available. More experience required on my part to get a better taste of it though. It stays out of the way and lets me do whatever I want. iirc their GPIOs are called “pads” or something, but its quite run-of-the-mill otherwise.
Zephyr : Quite decent for IoT applications. If you want anything close to the hardware (using comparators, or other chip specific mixed signal hardware, or anything that’s not in their library) or anything timing crucial, you are in a world of hurt. 99% config, 1% actual programming. Good luck spending your days searching google for the one missing line of config you missed. 0 debuggability, compiler errors are complete nonsense. Provided sensor driver library is impressive at a glance but if you want anything that’s not the easiest and most obvious and straightforward thing, you will have to rewrite the driver, often completely from scratch. Its like arduino kind of libraries.
Still I will say, its good when you just want stuff to work and don’t care what’s happening under the hood. I won’t say that’s a good thing but I’ve had days at work where I just want stuff to work so I can go home. Zephyr is perfect for those days.
Also last thing, if your platform isn’t well supported by zephyr, DO NOT USE IT
/rant over. As an engineer, I believe I am free to attack poor engineering decisions, but never the people behind the decisions.
Zephyr’s device tree based. If you’re used to embedded Linux/u-boot/etc., it’s familiar. Heck, Zephyr’s device tree introduction is better than most embedded Linux tools’ documentation!
Did anyone tried out the QP Framework? Any thoughts about that?
Personally, I’d have added at least one more to that list. SEL4, definitely, as it provides certain guarantees, which never hurts and is definitely different.
CapROS, RTAI, and Xenomai definitely have unique qualities, but its not clear how useful they are or what state they’re in.
I really like RIOT. First as a user, and now as a contributor. It also has a first class community around it. Everyone is always very civil and helpful.
Personally I think RIOT has a more gentle learning curve, yet it checks all the boxes: small, fast, support lots of boards (including 8-bit MCUs), lots of drivers and packages, … I teach OSes to 2nd year Bachelor students, and RIOT is the one I’ve adopted for the labs. No regrets :-)
I used NuttX a little some 6 years ago. When it works, it’s not bad, and it is nice to have a POSIXian system API relative to the FreeRTOS which I now use almost exclusively.
The main problem I had is that I was using an STM32L476, which was then new, so support had to be added. That meant creating headers for all the SFRs, etc., and implementing all the drivers, because it doesn’t like to use the stuff provided by the silicon vendors.
If you’re using an ‘F4, then you’ll be better off.
Hi ziggurat29, that is true! It is better to use NuttX with a well supported MCU to avoid adding support to it. Nowadays STM32 F4 F7 and H7 are well supported (F0 G0 too). And there is a company adding support to STM32H5!
No eCos? But… eCos provides a platform for all the zany architectures like CalmRISC, FR-V, Hitachi H8, Motorola 68000, Matsushita AM3x, MIPS, NEC V850, Nios II, PowerPC, SPARC, and SuperH! ;)
What no VRTX?
https://en.wikipedia.org/wiki/Versatile_Real-Time_Executive
This is what we used for our real-time kernel back in the day for hydro/comm/substation RTU/IED automation projects. I wrote a vrtx API ‘layer’ to simulate the kernel calls in Linux and Windows for portability… Even though not real-time in those environments, we could easily port common code to those platforms for multi-threading use. Simple to use.
I haven’t messed with eCos in decades, but I enjoyed it when I did. Is it still actively developed?
Could Ultibo Core be considered as, or capable of being, a real time OS?
Multitasking OSs unnecessary overhead to an embedded controller app?
I like bare metal programming because it is always the fastest and uses the least resources. With proper design patterns of course. But if somehow we do need an RTOS for our solution we mostly use the one provided by the MCU supplier, so for STM it’s FreeRTOS and Azure, but for Nordic Zephyr, NXP has FreeRTOS I belief. Mostly because there IDE’s have added special debug and configuration tools made by the supplier specific for that RTOS.
But the “best” RTOS imho is RIOT for it’s event driven tickless architecture and decent network stack.
Zephyr only uses Python-written tools to build, configure, and set up the environment. There is no need to use Python while developing applications.
Are you familiar with Linux dts, kconfigs etc? Go with it. You want a custom driver but do not know how to write it? You are not restricted to writing it – use it aside as a library of your project – it works.
For me? Decent RTOS. If someone prizes Linux – should use Zephyr.
https://rtic.rs/ FTW!!
Which actually is using Cortex Mx’s NVIC (Nested Vector Interrupt Controller) as it was designed to be used i.e. as ~~ a hardware implementation of a scheduler for those really real-time (read ‘time bounded’) tasks in a (RT)OS.
Which is actually only second ‘RTOS’ I’ve seen to do that since 2011 which I find extremely sad since CortexM cores and super powerful in their execution & exceptions model yet very simple and documented only in dozen of clear pages (this is the vendor-agnostic part and often missing completely from vendors’ reference manuals and datasheets because the simple and short manual is provided by ARM itself).
Also, anyone looking at RTIC may also want to investigate https://embassy.dev
QNX is also (recently) free for non-commercial projects: https://blackberry.qnx.com/en/products/qnx-everywhere
TL;DR: In 2024, there are many RTOS to choose from: FreeRTOS.
Whenever RTOSes come up, I point at an important distinction for hardware engineers like me: vendor support. Much of the above discourse pertains to the strengths and specific features of schedulers, build systems, and capabilities I’m unlikely to use: I just don’t like digging into code that much.
As such, I call out a distinction between the following:
– Ecosystems
– RTOSes
– Both
– (Optional) Meta layers built on top
People that have seen my writings know I’m all-in on Zephyr, but it is because it is both an Ecosystem and an RTOS and the vendor support is growing rapidly; especially for the parts I normally target in my work for my employer. mbed was another example of “both”, they had a custom RTOS but also a set of drivers and hardware support. One argument against that as a true ‘ecosystem’ is that the driver support was mostly driven by arm, as opposed to community and vendor support.
Another good example is FreeRTOS, which is prevalent throughout embedded designs. However, that is an RTOS first. Many vendors have taken the core of it and implemented their own ecosystem on top of it. ESP-IDF is a great example of this. Modus toolbox is an example as well, though much less known. I’ll admit I’m not super knowledgable about many of the other listed RTOSes, though I recall Contiki had a port for some TI hardware I used in the past (so…ecosystem as well?)
Focusing again on my laziness as a hardware engineer, I think it’s important to also look at sensible defaults for all the things you predict you might want to do with a platform. If you’re doing super tight timing and there are solid examples within the RTOS for hardware that you think might fit your application, that’s a good target. If I am expected to do a porting layer or to bolt together different pieces and libraries without any reference material? That’s an non-starter for me.
Perhaps building on this idea, you could highlight ecosystems and vendor support in future articles. I called out meta layers at the top, because things like Arduino and Micropython are also ecosystems with their own code capabilitlies that help simplify getting started, but they’re also often built on top of ecosystems and RTOSes as well. For instance, Arduino is transitioning from mbed to Zephyr currently and MicroPython can also be built on top of Zephyr.
One final piece to this winding reply: momentum. I was looking at support for ecosystems when building some hardware back in 2021 and started looking at MyNewt and Zephyr as options. Ultimately I made a choice based on the pace of development, number of community contributors, vendor support (see above), and the available resources in terms of docs and blogs. I recommend people choosing their next RTOS do the same.
Minor correction – it’s Eclipse ThreadX with the Eclipse Foundation, NOT Apache ThreadX – https://threadx.io/
Thanks. Fixed.
EE here. RTOS’s on MCUs are pointless IMO. Multitasking while observing real time constraints is already pretty easy on bare metal, and theres less bloat and complexity.
It makes sense (but is hard to do) on PC hardware. The use case being when you have real time requirements, but you need the processing power of a PC.
yeah i feel that way too. and i’d take it a step further and say that if your PC has tight real-time constraints, you’d probably be happier off-loading the real-time component to a dedicated microcontroller (like an I/O controller that packages everything into DMA bundles).
but i think there are a few intermediate scales where it would make sense, where you are piling a lot of unrelated tasks into a microcontroller, and your overall product is cheaper to build and to develop if you can shove it all into one microcontroller with the handy abstractions of an rtos. but i’d still hand-roll it :)
Yeah, the distinction you’re making between “MCUs” and “PC hardware” would’ve made sense like, 10-15 years ago. Not really today.
The phrase “MCU” nowadays is so overly broad as to almost be useless – you’ve literally got microcontrollers that have multiple cores, are in the hundreds of MHz, have on-board networking capabilities (some with multiple interfaces!) can host USB, can have trivial access to literally hundreds of gigabytes via SD cards, etc.
Managing real-time constraints with multiple processors, network interfaces, USB, an SD card filesystem, etc. is not “pretty easy.”
Plus of course a lot of modern MCUs have garbage software support from the vendors, so really, the RTOS in a lot of ways is “yeah we figured out how to do all the tricks so you don’t need to.” It’s not unheard of to have an MCU you think will work for a task bare-metal and then find out the vendor’s libs poll on something because of some random bug, which means you need to dig through and rewrite the thing.