We’re back! This week Jonathan chats with Mattias Wadman and Michael Farber about JQ! It’s more than just a JSON parser, JQ is a whole scripting language! Tune in to find out more about it.
- https://jqlang.github.io/jq/manual/
- https://play.jqlang.org/
- https://github.com/wader/jqjq
- https://github.com/wader/fq
- https://github.com/01mf02/jaq
- https://github.com/01mf02/jq-lang-spec/
Did you know you can watch the live recording of the show right on our YouTube Channel? Have someone you’d like us to interview? Let us know, or contact the guest and have them contact us! Take a look at the schedule here.
Direct Download in DRM-free MP3.
If you’d rather read along, here’s the transcript for this week’s episode.
Theme music: “Newer Wave” Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
They might want to work on that initialism.. Do some A/B testing.
By initialism you mean how to get started learning jq?
Presumably that’s because the initials have another use, which is very bad.
https://en.m.wikipedia.org/wiki/Jewish_question#Contemporary_use
In a previous role in a big tech company’s network infra team, we had a few good use cases for JQ:
Often network devices are able to spit out metrics in JSON. Our monitoring systems had a JQ module to be able to transform the data to extract the parts we and transform the data. This is useful for example creating new aggregated metrics to export to a time series database with quick one-liners.
We had an internal tool roughly similar to AWS Step Functions. One of the ways of transforming data between stages of the workflow was to use JQ.
Some gotchas!
JQ in the “pre 1.7” days didn’t deal well with large numbers. There was a 5 year gap before 1.7 was released, but it did mean JQ was “changing” some big numbers without us being aware. Became a big issue when transforming data that contains IDs as big numbers! This “gotcha” would be an interesting discussion in itself.
$ echo '{"number":288230376151711744}' | jq '.number' 288230376151711740Yes large number support has been confusing and also the JSON spec (rfc8259) does not say much about numbers more than recommending doubles (binary64), that is what JavaScript uses. To make it even more confusing jq for a while, probably accidentally, used decimal64 instead of binary64 when one did arithmetic operations, was fixed in 1.7.1.
Thank you got getting a release out! We used some JQ Cython bindings at the time, and it was super annoying to have to run our own builds because an upstream patch hadn’t been released yet.
Thanks! was the work of lots of ppl coming together. Ideally jq would support big int even for operations but i’m not sure if that will happen, gojq already do and jaq will in a future release.