At some point when developing embedded applications, you’re going to want to store unique values in non-volatile memory, values that can’t be fixed at compilation time. Many microcontrollers have a small amount of EEPROM memory for this very purpose, but it’s usually rather limited if it’s provided at all. Even if you do have a bit of space on an EEPROM at your disposal, actually formatting your values into the memory and dealing with the pesky problem of wear leveling (necessary for parameters that need to change often) can be a bit of a hassle.
Lucky for us, [Marcelo Barros] decided to share his own implementation, Kved (key/value database) which uses the flash memory instead for such storage. Kved implements a dictionary type data structure, using numeric keys and values, supporting a few integer types. Using the library should be straightforward enough, as [Marcelo] says, all you need are a pair of spare flash sectors and the ability to port the flash the sector read, write, and erase functions. There are plenty of examples of such code available for practically any microcontroller out there, so that should be no barrier. For those who want to play with it right now, the repo currently has ports for the STM32L433RC and STM32F411CE, as well as a simulated version you can compile and run on your computer.
From an implementation perspective, the write algorithm uses a COW (Copy On Write) method. Changed values are invalidated by over-writing the storage location with all-zeros, and re-writing the changed value to a new location, cycling through the unused locations until the sector is full. Data-integrity mechanisms are implemented, preventing corruption of the data structure due to power fail situations, so incorrectly written values will be corrected on start-up and not affect the integrity of the configuration.
When looking around, we found a similar project, Embedis, over on hackaday.IO, as well as this article on the subject of embedded filesystems from a little while back.
Look for libnklabs on github, another variation of this kind of thing. It saves the serialized version of a structured RAM database in flash. We use it for configuration and calibration data for embedded systems. There are CLI commands to print the database in text or query any part of it. I’ve been working on a schema compiler for it, then it will be like protobufs for microcontrollers.
In Automotive, we use something called FEE, Flash Eeprom emulation.
Similar in principle, but maybe more advanced. Has its weaknesses of course.
Typically operated with a specific type of flash memory which has higher erase cycle life than standard flash.
Also have a look at https://os.mbed.com/blog/entry/littlefs-high-integrity-embedded-fs/
Also esp-idf has ‘nvs’, which is similar.
Yay, someone who can spell “key/value.”
I don’t know why you’d want a key-value store, because it would only store key values.
Thanks to Marcelo not just for the code, but also for taking the time to write a clear readme explaining how it works.
Excellent Job!