You may know your way around the registers of that favorite microcontroller, but at some point you’ll also need to wield some ninja-level math skills to manage arrays of data on a small device. [Scott Daniels] has some help for you in this arena. He explains how to manage statistical calculations on your collected data without eating up all the RAM. The library which he made available is targeted for the Arduino. But the concepts, which he explains quite well, should be easy to port to your preferred hardware.

The situation he outlines in the beginning his post is data collected from a sensor, but acted upon by the collection device (as opposed to a data logger where you dump the saved numbers and use a computer for the heavy lifting). This can take the form of a touch sensor, which are known for having a lot of noise when looking at individual readings. But since [Scott] is using the Mean and Standard Deviation to keep running totals of collected data over time it is also very useful for applications like building your own home heating thermostat.

I haven’t grabbed the code, but the description in the linked page seems to show the author making every sort of numerical stability and loss of precision mistake possible. Even wikipedia has a good tutorial Kunth’s algorithms for keeping these calculations stable and precise.

It seems that the integer version is using “sqrt”. That is the float implementation for sqrt. Probably should have implemented own isqrt function.

The first rule of efficient programming for AVR is to ignore that inefficient software clusterfuck called arduino and all its affiliates. Making efficient libraries for arduino is like mounting a V8 on your dolly to pull a lexus. It’ll be a nice video on youtube but anyone who uses it doesn’t get how things work.

At least someone gets it. I was a little mystefied as to why the author was trading off 200 bytes of SRAM for a floating library when that same space could have easily be used to buffer his 10 values for calculating his average and mean. It took a little fiddling but once you wrap your head around the pointer acrobatics of a Round Robin table, the rest of the work is just figuring out what kind of precision you want. And I did mine in assembly…

Do I detect sarcasm or are you just very proud? If it’s the former, I think we should be celebrating the fact that some young people actually care about the low-level details. If it’s the latter, you’re definitely proud of something alright.

Let me see if I can put it another way. Arduino now is like what VB was ten years ago. When you start jumping through more and more hoops to accomplish something that a “lower” language can easily do, it’s time to rethink the strategy.

Cool. Post your library somewhere!

OK, I’m hesitant to post this because I’m usually wrong, but in the calculation of the mean value (really a running average) it seems to be that in short order you would overflow your 16 bit integer in pretty quickly, especially if you’re averaging a set of numbers that is already large-ish.

It might, it might not. The multiplier he uses will start “compressing” the stored values after “N” samples have been added together. So if they are small enough, and N is small enough too, the stored value for calculating the mean won’t overflow. Assuming an 8 bit ADC reading, a 32 bit long can safely use over 16 million samples. For a 10-bit reading, it is still “safe” up to 4 million samples or so.

The sum to calculate the standard deviation is more likely to have problems: it could overflow at 1008 observations with10-bit readings.

That’s in the integer library, BTW. The float version has a substantially different set of issues.