The concept of Continuous Integration (CI) is a powerful tool in software development, and it’s not every day we get a look at how someone integrated automated hardware testing into their system. [Michael Orenstein] brought to our attention the Hardware CI Arena, a framework for doing exactly that across a variety of host OSes and microcontroller architectures.
Here’s the reason it exists: while in theory every OS and piece of hardware implements things like USB communications and device discovery in the same way, in practice that is not always the case. For individual projects, the edge cases (or even occasional bugs) are not much of a problem. But when one is developing a software product that aims to work seamlessly across different hardware options, such things get in the way. To provide a reliable experience, one must find and address edge cases.
The Hardware CI Arena (GitHub repository) was created to allow automated testing to be done across a variety of common OS and hardware configurations. It does this by allowing software-controlled interactions to a bank of actual, physical hardware options. It’s purpose-built for a specific need, but the level of detail and frank discussion of the issues involved is an interesting look at what it took to get this kind of thing up and running.
The value of automatic hardware testing with custom rigs is familiar ground to anyone who develops hardware, but tying that idea into a testing and CI framework for a software product expands the idea in a useful way. When it comes to identifying problems, earlier is always better.
This is brilliant, we need more of these which support other form factors, but so far I like it. Happy that it’s open sourced as well. Would buy one immediately if it were on Tindie.
This project is perfect to do development and testing on top of a framework. However if the framework need to be debugged. More exposed IO port and ability to remap them become necessary.
When I develop framework for CH552, I build a jig to remap a IO of CH552 to a CH559 with a CH446 matrix. And a Raspberry Pi or computer automatically compile, upload and test the IO of the sketches with python scripts.
https://github.com/DeqingSun/CH552-Automatic-Test-Jig
Really nice!
Just a tip for others building test jigs: Black solder mask looks great, but it is harder to trace faults on the PCB. It’s fine if it is just yourself trying to fix your project, but when it is midnight in the factory and you need to fix a broken down test jig, normal green is easier on the eyes. (Having a spare PCB to swap out is even better, but I digress.)
Nice!
We do something similar (but not as polished from an hw pov) at Balena to test OS’s across tons of SBC. It’s called Autokit (and it’s opensource)
https://www.youtube.com/watch?v=Gjp2hgIXhHQ
Some projects use software simulators to run integration tests in their CI pipeline
Belay lets you run python code on your PC and RP2040 split transparently https://github.com/BrianPugh/belay They use the wokwi https://docs.wokwi.com/parts/wokwi-pi-pico to execute their integration tests https://github.com/BrianPugh/belay/tree/main/tests/integration