Parametric Press Unravels The JPEG Format

February 14, 2023 by Dave Rowntree 5 Comments

This is the first we’ve heard of Parametric Press — a digital magazine with some deep dives into a variety of subjects (such as particle physics, “big data” and such) that have interactive elements or simulations of various types embedded within each story.

The first one that sprung up in our news feed is a piece by [Omar Shehata] on the humble JPEG image format. In it, he explains the how and why of the JPEG encoding process, allowing the reader to play with the various concepts along the way, in real time, within the browser.

RGB colour-space subsampling doesn’t affect each component to the same degree due to the human eye cone cell response. Also, the chroma components are much less affected than the luminance.

For those not familiar with the format, the first step (which is actually optional) to JPEG encoding is to transform the image from the RGB color space, into a YCbCr (luminance, chrominance) color space. Since the human eye is far more sensitive to luminance (brightness) differences than it is to Cb (chroma relative blueness) and Cr (chroma relative redness) differences, these latter two components can be subsampled by only storing a single value for each, in every 2×2 pixel matrix. JPEG allows other matrix sizes, but 2×2 is the most common.

This sets the scene for the clever bit, that comes next and allows more of that harder-to-perceive chroma information to be discarded. It’s fun to play with the chroma sub-sampling slider and see how the different colours are not equally affected, due to the relative sensitivities of the human eye cone cells.

Next, the three YCbCr components are treated independently to a discrete cosine transform and quantization. This transforms each 8×8 pixel block into 64 discrete spatial frequencies. The JPEG compression level (which you can change) affects how many of the upper-frequency components get discarded, and thus how much of the fine spatial detail gets discarded. This is the main source of JPEG image quality loss. Finally, the compressed blocks are delta encoded, where each subsequent block is coded as the difference from the previous one. Like chroma subsampling, this doesn’t offer any compression on its own but allows the subsequent run-length encoding to be more effective, giving more (lossless) compression. Finally, the whole lot is then Huffman compressed with a unique table stored in the JPEG header. So want to play with JPEGs some more? here’s the GitHub source.

If all of this theoretical stuff is a bit useless to you, perhaps you just want to decode some JPEGs, then here is a speedy library for just that.

GhostSCAD: Marrying OpenSCAD And Golang

February 12, 2023 by Dave Rowntree 18 Comments

It’s been at least a couple of months since we’ve seen a different 3D modeling language project, so here’s [Lukasz Janyst] with GhostSCAD: a take on creating OpenSCAD models, using the Go language as the front end, bringing all the delights this modern modular language has to offer (and a few of its own idiosyncrasies.) As [Lukasz] says in the blog, from a programmer’s viewpoint, openSCAD has a number of failings that make it not necessarily hard, just kinda annoying to work with, due to the way the geometry tree works. The OpenSCAD way of working ends up with the programmer requiring knowledge of the internal workings of sub-modules, in order to work at the top level (assembly) which is not an ideal situation from a code reuse perspective.

A programmer would describe this problem as “abstraction leakage” and it doesn’t make modular, reusable coding easy to do without a lot of extra work. [Lukasz] says regarding the example GhostSCAD project, that some parts were modeled in a way that knowledge was needed of some mounting points of sub-modules, but those sub-modules had no way to expose this information to the outside world. GhostSCAD enables the programmer to define parts that expose specific parameters to the world that can be queried, for example, to produce a joining part, or an exploded assembly diagram. These properties can be interpreted without the querying module having any knowledge of the internal structure of the thing it’s working with. GhostSCAD provides a Java3D-like API for defining the geometry tree, which may be familiar to some.

Continue reading “GhostSCAD: Marrying OpenSCAD And Golang” →

XML Is A Quarter Century Old

February 11, 2023 by Jenny List 58 Comments

For those of us who have spent entire careers working with structured data, it comes as something of a surprise to be reminded that XML is now 25 years old. You probably missed the XML standard on the 10th of February 1998, but it’s almost certain that XML has touched your life in many ways even if you remain unaware of it.

The idea of one strictly compliant universal markup language to rule them all was extremely interesting in an era when the Internet was becoming the standard means to interchange information and when the walled gardens dating back to the mini- and mainframe era were being replaced with open standards-based interchange. In the electronic publishing industry, it allowed encyclopedia and dictionary-sized data sets to be defined to a standard format and easily exchanged. At a much smaller level, it promised a standard way to structure more mundane transactions. Acronyms and initialisms such as WAP, SOAP, and XHTML were designed to revolutionize the Web of the 21st century, but chances are that those are familiar only to the more grizzled developers.

In practice the one-size-fits-all approach of XML left it unwieldy, giving the likes of JSON and HTML4 the opening to be the standards we used. That’s not to say XML isn’t hiding in plain sight though, it’s the container for the SVG graphics format. Go on — tell us where else XML can be found, in the comments!

So, XML. When used to standardise large structured datasets it can sometimes be enough to bring the most hardened of developers to tears, but it remains far better than what went before. When hammered to fit into lightweight protocols though, it’s a pain in the backside and is best forgotten. It’s 25 years old, and here to stay!

Header: [Jh20], GFDL v1.2.

Create Your RTL Simulations With KiCAD

February 6, 2023 by Dave Rowntree 4 Comments

[Bob Alexander] is in the process of designing a homebrew discrete TTL CPU, and wanted a way to enter schematics for digital simulations via a Verilog RTL flow. Since KiCAD is pretty good at handling hierarchical schematics, why not use that? [Bob] created a KiCAD plugin, KiCadVerilog allowing one to instantiate and wire up the circuits under consideration, and then throw the resulting Verilog file at your logic simulator of choice.

KiCadVerilog doesn’t do all the hard work though, as it only provides the structure and the wiring of the circuit. The actual guts of each TTL instance needs to be provided, and a reference to it is manually added to the schematic object fields. That’s a one-time deal, as you can re-use the component library once generated. Since TTL logic has been around for a little while, locating a suitable Verilog library for this is easy. Here’s ice-chips-verilog by [TimRudy] on GitHub for starters. It’s intended as a collection for Icestudio (which is also worth a look). Still, the Verilog code for many TTL series devices is presented ready for the taking, complete with individual test benches in case you need them.

Check out the project GitHub page for the module source code, and some more documentation about the design process.

We’ve seen many RTL hacks over the years, here’s an interesting way to generate a PCB layout with discrete logic, direct from the RTL.

Hacking The Python For Loop

February 5, 2023 by Al Williams 33 Comments

In the early days of C, you’d occasionally see someone — probably a former Pascal programmer — write something like this:

#define BEGIN {
#define END }

This would usually initiate complaints about abusing the preprocessor and generally being anti-C. Surely no modern language would permit such things, right? Perhaps not. Consider [Tushar Sadhwani] who wanted to create a classic C-style for loop inside of Python. He did it, and the journey is perhaps more interesting than the result.

First, you can’t just transport straight C for loops into Python. There has to be some concession to Python syntax. The initial attempt was clever but not clever enough. However, the disassembly of the Python code was telling. The second attempt, however, was particularly interesting.

That attempt used an odd feature to examine the interpreter’s tree structure for the code and then modify it. This is sort of like a very painful C preprocessor but more powerful. That version works although it is pretty convoluted.

Ironically, [Tushar] then set up a third attempt after seeing code that tries to replace Python indentation with braces using a codec. In Python-speak, a codec lets you convert different text encodings. However, you can do other things than text encoding conversion. This is closest in spirit to the C preprocessor method. You can wade through the source code ahead of processing and make whichever changes you see fit.

Is any of this really useful? Probably not as it is. But you never know when you might need to do something exotic and one of these techniques could save the day. You probably couldn’t get away with some of this on MicroPython, of course. Your mileage may vary depending on where you find your Python running — like the Web.

Screenshot of a terminal showing the HELP command in action - outputting descriptions of other commands

Let’s Make SCPI More Helpful

February 2, 2023 by Arya Voronova 9 Comments

The SCPI (Standards Command for Programmable Instruments) protocol is exceptionally popular in lab and workspace tools, letting you configure and fetch data from oscilloscopes and lab scales alike in a standardized way. However, when interfacing with a SCPI device, you need to use a programming guide document if you want to know the commands for any of the inevitably extended features; essentially, SCPI isn’t as human-friendly as you might want. [MisterHW] argues that SCPI could use more discoverability by proposing a HELP? command.

This proposal is so intuitive, it makes you wonder why it isn’t in the base spec. It adds a built-in command that provides information on other commands. Internally, the description is just an extra string parameter that you add to your command definition code, and you can use it to describe the parameter types and ranges it takes. The output is both human-readable and machine-parseable, and as it’s stored within your code, it’s way quicker to update the description string than it is to re-release programming guides. Which are themselves prone to being outdated as-is, so decreasing reliance on them is a win-win.

The proposal makes a lot of sense, and [MisterHW] is willing to back it up with a pull request to the most popular SCPI library, libscpi. Whenever the pull request finally goes through, you will have the option to easily add the HELP? command support to whatever SCPI-connected device you might have brewing.

While the old devices will eventually fade, SCPI is not about to die out – hackers keep building devices with SCPI as the communication protocol, as the spec is quite powerful. For instance, here’s this fancy temperature logger, or this Source Measurement Unit – both of them use SCPI for hacker-to-device data transfer, and it’s likely to be libscpi under the hood. Ever wondered what SCPI is all about? Check out our overview!

Showing KiCanvas board viewer component inside a browser window, with a board being displayed and toggleable layers

KiCanvas Helps Teach And Share KiCad Projects In Browsers

January 31, 2023 by Arya Voronova 27 Comments

KiCad is undeniably the hacker favourite when it comes to PCB design, and we’ve built a large amount of infrastructure around it – plugins, integrations, exporters, viewers, and much more. Now, [Stargirl Flowers] is working on what we could call a web viewer for KiCad files – though calling the KiCanvas project a “KiCad viewer” would be an understatement, given everything it aims to let you do. It will help you do exciting things like copy-pasting circuits between KiCad and browser windows, embed circuits into your blog and show component properties/part numbers interactively, and of course, it will work as a standalone online viewer for KiCad files!

Continue reading “KiCanvas Helps Teach And Share KiCad Projects In Browsers” →