Parametric Press Unravels The JPEG Format

February 14, 2023 by Dave Rowntree 5 Comments

This is the first we’ve heard of Parametric Press — a digital magazine with some deep dives into a variety of subjects (such as particle physics, “big data” and such) that have interactive elements or simulations of various types embedded within each story.

The first one that sprung up in our news feed is a piece by [Omar Shehata] on the humble JPEG image format. In it, he explains the how and why of the JPEG encoding process, allowing the reader to play with the various concepts along the way, in real time, within the browser.

RGB colour-space subsampling doesn’t affect each component to the same degree due to the human eye cone cell response. Also, the chroma components are much less affected than the luminance.

For those not familiar with the format, the first step (which is actually optional) to JPEG encoding is to transform the image from the RGB color space, into a YCbCr (luminance, chrominance) color space. Since the human eye is far more sensitive to luminance (brightness) differences than it is to Cb (chroma relative blueness) and Cr (chroma relative redness) differences, these latter two components can be subsampled by only storing a single value for each, in every 2×2 pixel matrix. JPEG allows other matrix sizes, but 2×2 is the most common.

This sets the scene for the clever bit, that comes next and allows more of that harder-to-perceive chroma information to be discarded. It’s fun to play with the chroma sub-sampling slider and see how the different colours are not equally affected, due to the relative sensitivities of the human eye cone cells.

Next, the three YCbCr components are treated independently to a discrete cosine transform and quantization. This transforms each 8×8 pixel block into 64 discrete spatial frequencies. The JPEG compression level (which you can change) affects how many of the upper-frequency components get discarded, and thus how much of the fine spatial detail gets discarded. This is the main source of JPEG image quality loss. Finally, the compressed blocks are delta encoded, where each subsequent block is coded as the difference from the previous one. Like chroma subsampling, this doesn’t offer any compression on its own but allows the subsequent run-length encoding to be more effective, giving more (lossless) compression. Finally, the whole lot is then Huffman compressed with a unique table stored in the JPEG header. So want to play with JPEGs some more? here’s the GitHub source.

If all of this theoretical stuff is a bit useless to you, perhaps you just want to decode some JPEGs, then here is a speedy library for just that.

New Arduino JPEG Library Focuses On Speed

August 26, 2020 by Lewin Day 10 Comments

Working with graphics on microcontrollers has always meant focusing on making the most of limited resources. Particularly in the 8-bit era, all manner of tricks were used to get low-performance chips to achieve feats beyond their lowly station. However, these days, we’re blessed with 32-bit workhorses with clock speeds in the tens, or even hundreds, of MHz and many kilobytes of RAM to match. It’s these higher performance chips [Larry] had in mind when writing his JPEGDEC library.

As [Larry] discusses in a blog post on the topic, JPEG libraries already exist for the Arduino platform. However, many of these are aimed at 8-bit platforms with tiny amounts of RAM. While it’s possible to decode JPEGs piece by piece with some intelligent code under these conditions, it’s possible to go much faster when you’ve got a little more headroom. [Larry] does a great job of explaining the variety of optimizations he’s developed in the two decades since writing his first JPEG decoder back in 1994. From eliminating unnecessary marker checks to ignoring unneeded data for scaled-down output, it all adds up to get the job done faster. The library targets the Cortex-M0+, or any chip with a minimum of 20K of RAM, as its bare minimum to operate. Faster chips with higher clock rates naturally do better, and [Larry] provides benchmark decoding times for various common hardware using the library.

We’ve featured [Larry]’s GIF decoder for the Arduino platform before, again a useful library that’s optimised for good performance. If you’ve got your own neat tricks for image processing on microcontrollers, you know how to call!

File Compression By Steganography

October 19, 2019 by Sharon Lin 29 Comments

In a world with finite storage and an infinite need for more storage space, data compression becomes a very necessary problem. Several algorithms for data compression may be more familiar – Huffman coding, LZW compression – and some a bit more arcane.

[Labunsky] decided to put to use his knowledge of steganography to create a wholly unique form of file compression, perhaps one that may gain greater notoriety among other information theorists.

Steganography refers to the method of concealing messages or files within another file, coming from the Greek words steganos for “covered or concealed” and graphe for “writing”. The practice has been around for ages, from writing in invisible ink to storing messages in moon cakes. The methods used range from hiding messages in images to evade censorship to hiding viruses in files to cause mayhem.

The developer explains that since every file is just a bit sequence, observing files leads to the realization that a majority of bits will be equal on the same places. Rather than storing all of the bits of a file, making modifications to the hard drive at certain locations can save storage space. What is important to avoid, however, is lossy file compression that can wreak havoc on quality during the compression stage.

The compression technique they ended up implementing is based on the F5 algorithm that embeds binary data into JPEG files to reduce total space in the memory. The compression uses libjpeg for JPEG decoding and encoding, pcre for POSIX regular expressions support, and tinydir for platform-independent filesystem traversal. One of the major modifications was to save computation resources by disabling a password-based permutative straddling that uniformly spreads data among multiple files.

One caveat – changing even one bit of the compressed file could lead to total corruption of all of the data stored, so use with caution!

Interactive Demo Shows The Power Of Fourier Transforms

April 10, 2019 by Dan Maloney 14 Comments

When it comes to mathematics, the average person can probably get through most of life well enough with just basic algebra. Some simple statistical concepts would be helpful, and a little calculus couldn’t hurt. But that leaves out a lot of interesting mathematical concepts that really do have applications in everyday life and are just plain fascinating in their own right.

Chief among these concepts is the Fourier transform, which is the key to understanding everything from how JPEGs work to how we can stream audio and video over the Internet. To help get your mind around the concept, [Jez Swanson] has this interactive Fourier transform visualizer that really drives home the important points. This is high-level stuff; it just covers the basic concepts of a Fourier transform, how they work, and what they’re good for in everyday life. There are no equations, just engaging animations that show how any function can be decomposed into a set of sine waves. One shows the approximation of a square wave with a slider to control to vary the number of component sine waves; a button lets you hear the resulting sound getting harsher as it approaches a true square wave. There’s also a great bit on epicycles and SVGs, and one of the best introductions to encoding images as JPEGs that we’ve seen. The best part: all the code behind the demos is available on GitHub.

In terms of making Fourier transform concepts accessible, we’d put [Jez]’s work right up there with such devices as the original Michelson harmonic analyzer, or even its more recent plywood reproduction. Plus the interactive demos were a lot of fun to play with.

[via the Adafruit blog]

JPEG Decoding, Arduino Style

July 12, 2017 by Al Williams 32 Comments

When you think of image processing, you probably don’t think of the Arduino. [Jan Gromes] did, though. Using a camera and an Arduino Mega, [Jan] was able to decode input from an Arduino-connected camera into raw image data. We aren’t sure about [Jan’s] use case, but we can think of lots of reasons you might want to know what is hiding inside a compressed JPEG from the camera.

The Mega is key, because–as you might expect–you need plenty of memory to deal with photos. There is also an SD card for auxiliary storage. The camera code is straightforward and saves the image to the SD card. The interesting part is the decoding.

Continue reading “JPEG Decoding, Arduino Style” →

A Year Long Time Lapse Camera

December 22, 2014 by Brian Benchoff 19 Comments

All [val3tra] wanted was an RF-accessible camera. A camera that would take pictures, save them to an SD card, and occasionally send them over an RF link to a computer. This project has grown out of control, and now it has become an open-source camera that’s able to take year-long time-lapse movies.

The build started as a low power camera using an eBay JPEG camera modified for 3.3V. That’s only 640×480, but each frame averages only 48kb – small enough to store a few thousand pictures on a FAT16 formatted SD card. A $4 RF module, an ATMega, and an RTC make up the rest of the build that has a power draw of about 100 Joules per hour. A D-cell has about 60,000 Joules, and a pessimistic estimate of a battery of four in series, two in parallel gives a run time of 200 days.

This build was then improved, bringing the total battery consumption down to about 3.5-4 Joules per frame, or at one frame every 10 minutes, about 24 Joules an hour. That’s impressive, and getting this camera to run longer than a dozen or so months raises some interesting challenges. The self-discharge of the battery must be taken into account, and environmental concerns – especially when leaving this camera to run in a Moscow winter, seen in the video below – are significant.

If you don’t want to go equipment-lite you could seal your DSLR, Pi, and some serious batteries in a weatherproof enclosure.

Continue reading “A Year Long Time Lapse Camera” →

Towards More Interesting Instant Cameras

December 18, 2014 by Brian Benchoff 22 Comments

When [Ch00f] was getting jeans rung up at Nordstroms, he noticed how fast thermal receipt printers can put an image on a piece of paper. This observation isn’t unique to the circles [Ch00f] frequents – there are a few small receipt paper printers out there that connect to the Internet, iPhones, and a whole bunch of other Kickstarter-friendly keyword devices.

Nevertheless, a device that can make a hard copy of an image quickly and cheaply isn’t something you just stop thinking about. After rolling the concept around in his head for a few years, [Ch00f] finally came up with the perfect build – a camera.

The hardware for the build is based around an STM32F4 Discovery board. It’s a bit overpowered for this sort of application, and this is one of [Ch00f]’s first adventures in ARM-land. The rest of the hardware consists of a thermal receipt printer and a JPEG camera, the latter of which replaced a cellphone CMOS camera module that was lost in a move.

A custom camera requires a custom enclosure, and for this [Ch00f] made something remarkable. The entire enclosure is CNC milled out of a beautiful piece of figured walnut. The end result looks far too good for a prototype, but it does polish up nicely with a bit of linseed oil.

Now [Ch00f] has an instant camera that takes the idea of a Polaroid and turns it into something that produces a print for tenths of a cent. There’s a time-lapse function – just a zip tie on the shutter button – filters with the help of highlighters, and the ability to record movies in flipbook format.

It’s a great project, and also something that will make for a great crowdfunding campaign. [Ch00f] has already started work on this. He already has a sleek, modern-looking website that requires far too much scrolling than should be necessary – the first step to a winning Kickstarter. [Ch00f] also learned a lot about ARMs, DMA, dithering, gamma correction, and the JPEG format, but that’s not going to get anyone to open up their wallet. You know what will? A slick video. You’ll find that below.

Continue reading “Towards More Interesting Instant Cameras” →