Most of the computers we interact with on a day-to-day basis use an operating system designed for flexibility. While these are great tools for getting work done or scrolling your favorite sites, they have a weakness when it comes to interacting quickly with a real-world environment. For these kinds of low-latency, high-reliability systems you may want to turn to something like freeRTOS which is optimized for this kind of application and which [Parikshit Pagare] has used to build his home automation system.
This build is based around an ESP32 for which freeRTOS, designed specifically for embedded systems, is uniquely suited. There are several channels built in capable of monitoring temperature, functioning as a smoke alarm, and sensing whether someone is at the front door. All of these are reported to a small OLED screen but are also updated on an Android app as well, which happens nearly instantaneously thanks to the real-time operating system. There are a number of user-controllable switches as well that are capable of turning lights or fans on and off.
For a home automation system, it’s one of the most low-cost and fully-featured we’ve seen and if you’re still having trouble coming across a Raspberry Pi as they sort out supply issues, something like this might make an excellent substitute at a fraction of the price. If you’re looking to expand even beyond this build, one of the gold standards for ESP32-based automation design is this build from [Marcus] which not only demonstrates how to build a system like this but goes into great detail on the ESPHome environment.
Nice project to study the evolution of operating systems!
But: Is a microcontroller running an operating system still a microcontroller? ;-)
You’d be surprised how weak a microcontroller can be, and still be able to run FreeRTOS!
No doubt some of these devices can run FreeRTOS, but to me a microcontroller is a device being free from that operating system overhang. it is a piece of silicon that can be fully understood and programmed down to the last bit, directly with no need for software that runs between my digitalWrite() command and the resulting GPIO behavior.
Is this just my point of view?
I mean, a device can do what you describe and also still be able to run FreeRTOS. If you can program it, anything that’s Turing-complete can do abstraction. For some people anything without a MMU is a micro, for other people, anything below a certain amount of FLOPS or IOPS… You can basically bare metal program anything if you want to though, right?
Yes, you can bare metal program anything, good argument. But let’s call only those chips microcontrollers that you can fully understand and program as a single person in no more than one lifetime?
Next step will be someone implementing containers in a microcontroller OS ;-)
Or Let’s call only those chips microcontrollers that have integrated memory, i/o, and cpu on a single ic, regardless of how that memory is written.
My experience is that once you start adding any amount of sophistication to a dumb “super loop” start dealing with problems like multi-threading, race conditions, inter-thread communications, etc., you end up writing a ton of code. Using an RTOS doesn’t actually add very much overhead once it’s compiled. FreeRTOS scales beautifully down to tiny MCUs up to giant multi-core processors.
Looks like our definition of “tiny” is very different ;-)
My favorite microcontroller is the 8 core parallax propeller. Helps with interrupts as things can run independently on each core.
@ Norbert.
If anyone laughs and points then it’s tiny.
I agree that a super-loop invites problems like race conditions and inter-thread communication. As your software grows in complexity, you end up needing to re-invent the solutions to things which an RTOS already has solved. Where I work we use FreeRTOS on a Cortex M0 running at ~50Mhz for a network-connected device. There are about 100 tasks running concurrently and the binary takes up ~3MB. It truly amazes me how much that little MCU can do. An RTOS really lets you squeeze a ton of functionality into a tiny MCU. Some say we should use Linux instead, but really you can’t beat the price of the hardware and the little amount of heat that it generates.
FreeRTOS is mainly a scheduler for multi-threading, far from a classic OS wich provides abstraction for all resources. You *can* use driver to ease access to hardware components, but you equally can program hardware on the bare metal.
I did this on a ATMEGA128p4 with 128KB Flash, and 10K or so RAM. FreeRTOS ate <5KB Flash, and a few 100 bytes of extra RAM. In idle it takes around 5% CPU. This „overhead“ is more than worth the benefits for me.
The times where the world of MCUs ended with he 8051 is long gone.
Also the times where you programmed manly in assembler only. I never needed a line of assembler, even not for bitbanging and interrupt handlers. The only assembler came for context switches within FreeRTOS.
If you ignore the add-on libraries and use just its core functionality, FreeRTOS is effectively just a wrapper around a couple interrupt handlers with some stack management code on top.
It can be fully understood… And I’d say the hardest part of grasping what it’s doing is mentally filtering out all the architecture-specific ifdefs :)
Have you ever used RTEMS? It is a lot more than “a wrapper around a couple interrupt handlers with some stack management code on top” but still gives you deterministic and bounded behavior.
> no need for software that runs between my digitalWrite() command
As long as you discount the Arduino bootloader and the libraries that do all sorts of stuff behind your back, including interrupts that interfere with your GPIO timing.
With FreeRTOS and other small embedded executives the application has direct hardware access. Though there may be board support packages that provide I/O services, they are not part of the kernel. I have used FreeRTOS on Cortex-M (STM32 and SmartFusion2), and MicroBlaze soft cores. I use the bare metal drivers unmodified. What the executive provides is multiple threads with priority preemptive scheduling, timers, and thread synchronization primitives. I find that superloops and interrupt handlers alone are less responsive and harder to get asynchronous events from multiple sources behaving.
I have recently been using Kelvin Lawson’s Atomthreads (available on GitHub) on a small MicroBlaze soft core in a Xilinx FPGA and actually reduced the overall code size – the memory footprint was larger because of the separate runtime stacks, but the complexity of the software was reduced.
Not all microcontroller applications will benefit from an RTOS, but most non-trivial embedded applications that I have been involved with over the past 40 years either did or would have.
You are mixing GPOS with RTOS. An RTOS (or most of them) does not provide an abstraction between your code and the hardware. But an RTOS allows you to have multiple tasks and encapsulat resources to avoid race conditions.
Both add some sort of delay between the digitalWrite() and the actual execution of that command. I don’t care what that kind of code mix is named ;-)
While it may be common to not provide device class abstractions to increase portability, over at RTEMS, we try to have them. This means the BSP and drivers plug into the bottom of a framework and the application code can remain unchanged across hardware platforms. We do this for at least serial devices, network (BSD or lwIP), block devices, non-volatile storage, and i2c/SPI access.
We also have the view that you should not have to cobble together a bunch of third party addons and ports. The main source tree has ~18 architectures and ~200 BSP variants.
Norbert would you call a 32-bit CPU with integrated RAM and peripherals still be a microcontroller?
https://www.microchip.com/en-us/products/microcontrollers-and-microprocessors/32-bit-mpus/sip-and-som/system-in-package
The rp2040 microcontroller runs a python repl. That is getting pretty close to an operating system since it can do all the functions of one. I agree with you that the terms are getting murky because of how they can and do overlap.
The core of FreeRTOS is about 5 c files and in the range of a thousand lines. You can understand how it works completely if you are motivated in less than a week. It does not distance you from the bare metal feeling.
FreeRTOS is only about scheduling parallel tasks and does not do anything with GPIO. It is possible to use FreeRTOS and direct GPIO access. On the other hand digitalWrite() is an unneeded abstraction. If you think about portability it is not even portable because the pin index parameter is still platform dependent. On Arduino the digitalWrite() function is about 100 times slower than just writing a 1 to the register.
We’ve all used a screwdriver to pry open paint cans, poke a hole in dry walls, dig a hole in the ground, etc. Do we still call it a screwdriver? 😊
Only for the moment
BASICs and FORTHs provided on-board OSs,
Until declared ‘No good. And must replace by c/c++ code by industries .’?
But c/c++ computer programmers failed with:
1 buggy
2 malware vulnerable
3 software modules longer than one page of code in violation of Boeing hardware engineers’ software standards?
Unmaintainable?
At a horrible cost to world economies?
From MCU coding to OS creation: “I wanted a task scheduler, but ‘Abracadabra’ now I’m an operating system magician!”
Nice. Yep, a lot of projects don’t require a full blown OS like Linux. Most don’t even require Real-time where Python or Arduino style programming will do just fine. Once in awhile you’ll run into a project that requires an RTOS.
Having used both FreeRTOS and RTEMS I’ll prefer the latter.
Thanks for the RTEMS mention :)
Yup, RTEMS is a thing still.[1][2]
1. RTEMS (Real-Time Executive for Multiprocessor Systems)
https://en.wikipedia.org/wiki/RTEMS
2. RTEMS Real Time Operating System (RTOS)
https://www.rtems.org/
Still a thing indeed! Quite active as a project and currently everywhere from the Sun to Jupiter. You’ve likely seen projects like JUICE, DART, and Parker Solar Probe that use it in the news recently. The commercial projects tend not to get as much press. :)
One thing I’m proud of is that ESA has done a space pre-qualification and made it publicly available. They have worked to ensure that as much as possible of this is submitted to the RTEMS Project.
https://rtems-qual.io.esa.int/
If you are using Arduino with an ESP32 you are using an RTOS. Weather you know it or not.
Not really, wireless stack takes precedence and can interrupt any arduino process any moment, so no real time at all there.
Which only gives you a proof that you are running (Free)RTOS on behind. Task scheduling and running is then just upon the user app and RTOS.
You are missing the point, freertos runs on ESP32 chips whether it looks like it or not when you are using Arduino or esp-idf. The main task is just run as a freertos task as is the wireless stack. That doesn’t make it not real time, how would running wireless functions all of a sudden make it not real time?
Anything you can do with a rtos on a controller could be better done with a structured fsm with proper data structures, dma and interrupts.
Even better, with regular polls instead of interrupts. Interrupts are very hard to reason about if you need hard real-time.
Interrupts aren’t hard to reason about at all. If you need something to happen right when the condition occurs then you need interrupts.
There is a reason why polling is looked down on in microcontroller programming but especially when using an RTOS, it is inefficient and can be slow and prevent other parts of the program from running whilst it polls. You can get much more efficient time management by using interrupts, everything executes close to when it needs to.
Real time just means faster timing, you will never get truly real time when using a processor unless it only does that one specific thing and polls constantly. Using an RTOS doesn’t really add any more latency or overhead compared with polling everything but it does generally mean more efficient time usage and can mean faster response times.
Define “better”.
Hand coding and hand optimizing your own timers, schedulers, loops, semaphores and FSMs certainly has higher performance(lower latency) and requires less RAM/ROM. But it does add more to the development time for larger projects. I’ve used both and also combinations of them and I certainly prefer the elegance of hand coded FSM, but often use FreeRTOS for speeding up development.
An threading library like FreeRTOS basically turns every thread function into an implicit FSM. Every wait function (delay, wait for event, wait for semaphore etc.) is an unnamed waiting state.
A lot of projects use an expensive PI rather than the esp32 to do something an esp32 would do easily, which has always surprised me..
The Esp32 has more power, and more memory, than a PC in the early 90’s, and RTOS is a lot better than DOS! It easily handles quite a few threads – my current main project has 15 – and can handle a surprising large project…
Yeah people tend to use SBCs for things microcontrollers should be used for, I think it is just because they are familiar with a pi. There is also quite a big link between using a pi in place of an MCU and programming in python so I think part of the problem may be that people can’t program in C or C++ and are familiar with using a pi, even though some MCUs can be programmed in micro python.
It does annoy me seeing a very basic project using a pi 4, it is a total waste of resources and it uses much more power as well.
The ESP32 line will get more impressive soon with the ESP32P4, a dual core 400 MHz RISC-V processor with AI/vector extensions, no wireless though.
While this is very cool, and I’m definitley going to be going through the code as it’s given me a few ideas I also think the build by Marcus that’s linked in the article is a much better way to go. I can’t see this leaving much room for expansion, particularly cameras. Also don’t think I’d want to replace any off the shelf smoke alarms for a homebrew, but would be alright alongside one.
In fact I’m just putting a similar HA build to Marcus right now, and on the assumption that you have an old android phone in a drawer it’s basically a free upgrade from this one.
Only a couple of drawbacks to using an old phone instead of an unobtainable pi:
1/ Getting a home assistant server setup on android is an absolute ball ache, requiring quite a bit of compiling random modules such as an older termux ffmpeg version, but hey the techie masochist in me kinda enjoyed it.
2/ You only get home assistant core, so no addons. This is no problem for me, everything I’m doing uses esphome
3/ Ideally for safety you don’t leave the phone permenantly charging. I’ve gone down the rooting the phone road to limiting the battery charge (still testing this), or you could automate a relay connected to the charger so as to maintain the battery at say 50-60% if that’s easier.
If you are concerned about the battery then you could just find a way to remove it and still have the phone work, maybe then connect it to a more suitable UPS that handles the battery properly.
I would call an ESP32 a lot of things. “Tiny Microcontroller” is not among those things. The ESP32 is a very beefy uC. If you called the “Arduino Chip” ATmega328 tiny, I wouldn’t complain. And there are Chips specifically named ATtiny…
Great to see that there is someone else who shares my view of what a tiny microcontroller is!
The main advantage of an RTOS is support of modular design but that support is only needed for complex systems. Home automation can be done with Arduino libraries that run as a single task on ESP32. If there are a large variety of sensors, then additional ESP32 tasks can be created within Arduino code since it uses the Espressive libraries. So I don’t see the need for an RTOS in typical Maker projects. That said, the porting of freeRTOS to ESP32 is impressive. It might be essential for special use-cases.
ESP32 generally runs freertos anyway, this isn’t a new port or anything. They have supported it right from the start, if you use ESP32 with Arduino or ESP-IDF then you are already using freeRTOS, it just hides it away if using Arduino but the setup and loop function are just run as their own task within freeRTOS with the wireless communication handling running as another task.
So all that is to say, this isn’t new and if you have ever used an ESP32 you have most likely used freeRTOS on it without realising.
I see no revolution in this. The autor has just used tasks and passes data over queues. Simply search for “ESP32 arduino multitasking” to learn more.
What is new or newsworthy about this? ESP32 has supported freeRTOS and mainly run on freeRTOS right from the start, anyone who has used the Arduino port for ESP32 or the ESP-IDF has used freeRTOS. The setup and main functions in Arduino are just run as a freeRTOS task already with the WiFi and Bluetooth handled in a separate task.
Loads of MCUs support freeRTOS including the rp2040 and STM32 MCUs, so what is new or newsworthy about an ESP32 running freeRTOS when they have for years, if not from the start. FreeRTOS is one of the only practical ways to use both cores in the ESP32 or rp2040.
Agreed. There’s nothing new to mention. Well, maybe one thing; I would even discourage people from following such project, because messing Arduino framework with making own tasks if they are not sure they can run into race conditions for example is better to be aware of.
This project only uses the Arduino IDE with FreeRTOS, what are you talking about.