Kved: An Embeddable Key/Value Datastore

June 14, 2022 by Dave Rowntree 7 Comments

At some point when developing embedded applications, you’re going to want to store unique values in non-volatile memory, values that can’t be fixed at compilation time. Many microcontrollers have a small amount of EEPROM memory for this very purpose, but it’s usually rather limited if it’s provided at all. Even if you do have a bit of space on an EEPROM at your disposal, actually formatting your values into the memory and dealing with the pesky problem of wear leveling (necessary for parameters that need to change often) can be a bit of a hassle.

Lucky for us, [Marcelo Barros] decided to share his own implementation, Kved (key/value database) which uses the flash memory instead for such storage. Kved implements a dictionary type data structure, using numeric keys and values, supporting a few integer types. Using the library should be straightforward enough, as [Marcelo] says, all you need are a pair of spare flash sectors and the ability to port the flash the sector read, write, and erase functions. There are plenty of examples of such code available for practically any microcontroller out there, so that should be no barrier. For those who want to play with it right now, the repo currently has ports for the STM32L433RC and STM32F411CE, as well as a simulated version you can compile and run on your computer.

From an implementation perspective, the write algorithm uses a COW (Copy On Write) method. Changed values are invalidated by over-writing the storage location with all-zeros, and re-writing the changed value to a new location, cycling through the unused locations until the sector is full. Data-integrity mechanisms are implemented, preventing corruption of the data structure due to power fail situations, so incorrectly written values will be corrected on start-up and not affect the integrity of the configuration.

When looking around, we found a similar project, Embedis, over on hackaday.IO, as well as this article on the subject of embedded filesystems from a little while back.

Using Lookup Tables To Make The Impossible Possible

January 10, 2020 by Dan Maloney 60 Comments

Embarrassing confession time: I never learned my multiplication tables in grade school. Sure, I had the easy tables like the twos and the fives down, but if asked what 4 x 7 or 8 x 6 was, I’d draw a blank. As you can imagine, that made me a less than stellar math student, and I was especially handicapped on time-limited tests with lots of long multiplication problems. The standard algorithm is much faster when you’ve committed those tables to memory, as I discovered to my great woe.

I was reminded of this painful memory as I watched Charles Lohr’s 2019 Supercon talk on the usefulness and flexibility of lookup tables, or LUTs, and their ability to ease or even completely avoid computationally intensive operations. Of course most LUT implementations address problems somewhat more complex than multiplication tables, but they don’t have to. As Charles points out, even the tables of sines and logarithms that used to populate page after page in reference books have been ported to silicon, where looking up the correct answer based on user input is far easier than deriving the answer computationally.

Yes, this is a Minecraft server all thanks to LUTs.

One of the most interesting examples of how LUTs can achieve the seemingly impossible lies in an old project where Charles attempted to build a Minecraft server on an ATMega168. Sending chunks (the data representations of a portion of the game world) to clients is the essential job of a Minecraft server, and on normal machines that involves using data compression. Rather than trying to implement zlib on an 8-bit microcontroller, he turned to a LUT that just feeds the raw bytes to the client, without the server having the slightest idea what any of it means. A similar technique is used by some power inverters, which synthesize sine wave output by feeding one full cycle of values to a DAC from a byte array. It’s brute force, but it works.

Another fascinating and unexpected realization is that LUTs don’t necessarily have to be software. Some can be implemented in completely mechanical systems. Charles used the example of cams on a shaft; in a car’s engine, these represent the code needed to open and close valves at the right time for each cylinder. More complicated examples are the cams and gears once found in fire control computers for naval guns, or the programming cards used for Jacquard looms. He even tips his hat to the Wintergatan marble machine, with its large programming drum and pegs acting as a hardware LUT.

I found Charles’ talk wide-ranging and fascinating. Originally I thought it would be an FPGA-heavy talk, but he didn’t actually get to the FPGA-specific stuff until the very end. That worked out fine, though — just hearing about all the cool problems a LUT can solve was worth the price of admission.

And for the curious, yes, I did eventually end up memorizing the multiplication tables. Oddly, it only clicked for me after I started playing with numbers and seeing their relationships using my first calculator, which ironically enough probably used LUTs to calculate results.

Continue reading “Using Lookup Tables To Make The Impossible Possible” →

Hackers Want Cambridge Dictionary To Change Their Definition

June 26, 2018 by Jenny List 83 Comments

Maybe it’s the silly season of high summer, or maybe a PR bunny at a cybersecurity company has simply hit the jackpot with a story syndicated by the Press Association, but the non-tech media has been earnestly talking about a call upon the Cambridge Dictionary to remove the word “illegal” from their definition of “Hacker”. The weighty tome from the famous British university lists the word as either “a person who is skilled in the use of computer systems, often one who illegally obtains access to private computer systems:” in its learners dictionary, or as “someone who illegally uses a computer to access information stored on another computer system or to spread a computer virus” in its academic dictionary. The cybersecurity company in question argues that hackers in fact do a lot of the work that improves cybersecurity and are thus all-round Good Eggs, and not those nasty computer crooks we hear so much about in the papers.

We’re right behind them on the point about illegality, because while there are those who adopt the hacker sobriquet that wear hats of all colours including black, for us being a hacker is about having the curiosity to tinker with anything presented to us, whatever it is. It’s a word that originated among railway modelers (Internet Archived version), hardly a community that’s known for its criminal tendencies!

Popular Usage Informs Definition

It is however futile to attempt to influence a dictionary in this way. There are two types of lexicography: Prescriptive and Descriptive. With prescriptive lexicography, the dictionary instructs what something must mean or how it should be spelled, while descriptive lexicography tells you how something is used in the real world based on extensive usage research. Thus venerable lexicographers such as Samuel Johnson or Noah Webster told you a particular way to use your English, while their modern equivalents lead you towards current usage with plenty of examples.

It’s something that can cause significant discontent among some dictionary users as we can see from our consternation over the word “hacker”. The administration team at all dictionaries will be familiar with the constant stream of letters of complaint from people outraged that their pet piece of language is not reflected in the volume they regard as an authority. But while modern lexicographers admit that they sometimes walk in an uneasy balance between the two approaches, they are at heart scientists with a rigorous approach to evidence-based research, and are very proud of their efforts.

Big Data Makes for Big Dictionaries

Lexicographic research comes from huge corpora, databases of tens or hundreds of millions of words of written English, from which they can extract the subtlest of language trends to see where a word is going. These can be interesting and engrossing tools for anyone, not just linguists, so we’d urge you to have a go for yourself.

Sadly for us the corpus evidence shows the definition for “Hacker” has very firmly trended toward the tabloid newspaper meaning that associates cybercriminality. All we can do is subvert that trend by doing our best to own the word as we would prefer it to be used, re-appropriating it. At least the other weighty tome from a well-known British university has a secondary sense that we do agree with: “An enthusiastic and skilful computer programmer or user“.

Disclosure: Jenny List used to work in the dictionary business.

We’re Using The Word Firmware Wrong

September 29, 2017 by Mike Szczys 123 Comments

I had an interesting discussion the other day about code written for an embedded system. I was speaking with Voja Antonic about ‘firmware’. The conversation continued forward but I noticed that he was calling it ‘software’. We later discussed it and Voja told me he thought only the parts of the code directly interacting with the microcontroller were firmware; the rest falls under the more generic term of software. It really had me wondering where firmware stops being firmware and is merely software?

The topic has remained on my mind and I finally got around to doing some dictionary searches. I’m surprised that I’ve been using the word differently and I think most of the people I’ve heard use it are doing the same — at least as far as dictionary definitions are concerned. My go to sources are generally Merriam-Webster and Oxford English dictionaries and both indicate that firmware is a type of software that is indelible:

Permanent software programmed into a read-only memory.

–Oxford English Dictonary

computer programs contained permanently in a hardware device (such as a read-only memory)

–Merriam-Webster Dictonary

According to this definition, I have never written a single bit of firmware. Everything I have written has been embedded software. But surely this is a term that must change with the times as technology progress so I kept digging.

Continue reading “We’re Using The Word Firmware Wrong” →

Hackaday Dictionary: Servo Motors

March 24, 2016 by Richard Baguley 29 Comments

How do you make things move? You add in a motor that converts electrical energy into motion. That’s a simple idea, but how do you know where the motor is? That’s where the servo motor comes in. By adding a sensor and a controller to the mechanism, these motors can figure out how far they have rotated and maintain that setting without any need for external control.

A disassembled servo motor showing the controller, motor, rotary encoder and gears. By oomlout - SERV-05-ST_TEARDOWN_03, CC BY-SA 2.0 — A disassembled servo motor showing the controller, motor, rotary encoder and gears. By oomlout, CC BY-SA 2.0

What is a Servo Motor?

These neat devices can be large or small, but they all share the same basic characteristics: a motor connected to a gearing mechanism and an encoder that detects the movement and speed of the motor. This combination means that the controlling device doesn’t need to know anything about the motor itself: the controller on the servo motor handles the process of feeding the appropriate power to the motor until it reaches the requested position. This makes it much easier to build things with servomotors, as the designer has already done all the hard work for you.

The first place that most people encounter a servo motor is in the small hobby servos that are used in remote control vehicles. Manufactured by companies like Hitec and Futaba, these drive a gear or arm that transfers the rotation of the motor to perform tasks like turning a wheel to steer a car, moving a control surface on an RC plane, or any task that requires a small range of motion at high precision. The gearing in the servomotor offers more torque than connecting the shaft directly to the motor. Most hobby servos of this type are restricted to a certain range of motion (usually 180 degrees) because the position encoder is a simple potentiometer connected to the output shaft.

A selection of different sized servo motors. By Osamu Iwasaki

Servomotors usually have three connection wires: a power line, a ground line and a signal line. The signal line is fed a pulse width modulation (PWM) signal that determines the angle that the servomotor moves to. As the name suggests, the length of the pulse (or the width, if you look at it on an oscilloscope) is the thing that controls the angle that the servo moves to: a short pulse (1 millisecond) sets it to the zero angle, while a long pulse of 2 milliseconds sets it to the maximum angle. A pulse length between these two limits signals the servomotor to move to the corresponding angle: 1.5 ms would set it to 90 degrees.

It is important to note that servomotors and stepper motors are not the same thing. Both are used for positioning, but steppers usually run without feedback. Instead, steppers turn (as the name suggest) in discrete steps. To figure out where a stepper motor is requires a limit switch, then driving the stepper until this is triggered. Then if you keep count out the number of steps that it’s traveled, you know where it is. That’s why devices like inkjet or 3D printers will move to their limits when they start up, so the controller can detect the far limit of the mechanism being driven, and calculate the current position from that.

How Do You Use A Servomotor?

Because the designers of servomotors have done most of the hard work for you, servomotors are very easy to use. To drive them, you just need to feed them power (usually 5V) and feed the PWM signal to the servomotor. You can drive them directly from an Arduino or similar microcontroller using a library that converts an angle into a PWM signal on one of the output pins.

Each servomotor requires a dedicated output pin if they are being driven this way, though, so if you are driving a lot of servomotors, a dedicated controller makes more sense. Devices such as the Adafruit Servo Shield and the Pololu Maestro allow you to control multiple servos from a single output pin on the microcontroller: the microcontroller sends a signal to the device addressing each servo in turn, and the device converts this into the PWM signals for each. If you need to drive a lot of servos, the SD84 can control up to 84 servos at once from a single USB port.

(Headline image bots: µBob and Hexapod4.)

Hack A Day’s Dictionary Of Questionable Utility™

May 15, 2010 by Phil Burgess 43 Comments

We frequently receive inquiries from eager readers asking how they can best get started in electronics and computer projects. Countless great books have been written on these subjects, and of course now there’s our answers.hackaday.com site. But there’s a difference between being “book smart” and being “street smart.” What are the terms that you really need to know to get ahead in this field? We’ve collected a few of our favorites here.

Have any terms or definitions to add? Leave a note in the comments!

Continue reading “Hack A Day’s Dictionary Of Questionable Utility™” →