messagepack

2 Articles

The Seedy World Of Message Serialization

June 10, 2020 by 26 Comments

Look, I’ve been there too. First the project just prints debug information for a human in nice descriptive strings that are easy to understand. Then some tool needs to log a sensor value so the simple debug messages gain structure. Now your debug messages {{look like : this}}. This is great until a second sensor is added that uses floats instead of ints. Now there are sprinklings of even more magic characters between the curly braces. A couple days later and things are starting to look Turing complete. At some point you look up and realize, “I need a messaging serialization strategy”. Well you’ve come to the right place! Continue reading “The Seedy World Of Message Serialization” →

MessagePack Is A More Efficient JSON

March 12, 2020 by 50 Comments

It is an age-old problem, that of having some data you want to store somewhere, and later bring it back. How do you format the data? Custom file formats are not that hard, but if you use an existing format you can probably steal code from a library to help you. Common choices include XML or the simpler JSON. However, neither of these are very concise. That’s where MessagePack comes in.

For example, consider this simple JSON stanza:

{"compact":true, "schema":0}

This is easy to understand and weighs in at 27 bytes. Using MessagePack, you’d signal some special binary fields by using bytes >80 hex. Here’s the same thing using the MessagePack format:

 
0x82 0xA7 c o m p a c t 0xC3 0xA6 s c h e m a 0x00

Of course, the spaces are there for readability; they would not be in the actual data stream which is now 18 bytes. The 0x82 indicates a two-byte map. The 0xA7 introduces a 7-byte string. The “true” part of the map is the 0xC3. Then there’s a six-byte string (0xA6). Finally, there’s a zero byte indicating a zero.

Continue reading “MessagePack Is A More Efficient JSON” →