Does Library Bloat Make Your Smartphone App Look Fat?

While earlier smartphones seemed to manage well enough with individual applications that only weighed in at a few megabytes, a perusal of the modern smartphone software store uncovers some positively monstrous file sizes. The fact that we’ve become accustomed to mobile applications requiring 100+ MB downloads on what’s often a metered Internet connection in only a few short years is pretty crazy if you stop to think about it.

Seeing reports that the Nest app for iOS tipped the scales at nearly 250 MB, [Alexandre Colucci] decided to investigate. On his blog he not only documents the process of taking the application apart piece by piece to find out just what’s eating up all that space, but lists some potential fixes which could shave a bit off the top. Even if you aren’t planning a spelunking expedition into your pocket supercomputer’s particular variant of the Netflix app, the methodology and tools he uses here are fascinating in their own right and might be something worth adding to your software bag of tricks.

By passing the application’s files through a disk usage visualizer called GrandPerspective, [Alexandre] immediately identified some rather large blocks of content. The bundled Apple Watch version of the app takes up 23 MB, video and audio used to walk the user through the device setup weigh in at 22 MB, and localization files for various languages consumes a surprising 33 MB. But the biggest single contributor to the application’s heft is the assorted libraries and frameworks which total up to an incredible 67 MB.

Of course the question is, how much of it is really necessary? It’s hard to be sure from an outsider’s perspective, but [Alexandre] notes that a few of the libraries used seem to be redundant or obsolete. In some cases this could be the result of old code still lurking in the project, but the four different libraries used for user tracking probably aren’t in there by accident. It also stands to reason that the instructional videos could be offloaded to something like YouTube, so that only users who need to view them have to expend their bandwidth on it.

Getting a little deeper into things, [Alexandre] notes that some of the localization images appear to be redundant. As a specific example, he points to the images of the Nest itself displaying Fahrenheit and Celsius temperatures. While logically this should only be two image files, there are actually eight copies of the Celsius image, each filed away as language-specific. These redundant localization images could easily be stripped out, but with gains measured in only a few hundred kilobytes, it probably wasn’t considered worth the effort during development.

In the end there’s really not as much bloat as we might like to believe. There were some redundant files, maybe a few questionable library inclusions, and the Apple Watch version of the app could surely be separated out. All together, it might get you a savings of 30 – 40%, but still not enough to bring it down under 100 MB.

All signs point to the fact that modern smartphone software development is just a lot more burdensome than us hackers might like. Save for projects looking to put control back into the hand’s of the users, it looks like mobile operating systems aren’t going to be slimming down anytime soon.

A Real Time Data Compression Technique

With more and more embedded systems being connected, sending state information from one machine to another has become more common. However, sending large packets of data around on the network can be bad both for bandwidth consumption and for power usage. Sure, if you are talking between two PCs connected with a gigabit LAN and powered from the wall, just shoot that 100 Kbyte packet across the network 10 times a second. But if you want to be more efficient, you may find this trick useful.

As a thought experiment, I’m going to posit a system that has a database of state information that has 1,000 items in it. It looks like an array of RECORDs:

typedef struct
{
  short topic;
  int data;
} RECORD;

It doesn’t really matter what the topics and the data are. It doesn’t really matter if your state information looks like this at all, really. This is just an example. Given that it is state information, we are going to make an important assumption, though. Most of the data doesn’t change frequently. What most and frequently mean could be debated, of course. But the idea is that if I’m sending data every half second or whatever, that a large amount isn’t going to change between one send and the next.

Continue reading “A Real Time Data Compression Technique”

Stretching My Skills: How (and Why) I Made My Own Compression Sleeves

Have you ever noticed how “one size fits all” often means “one size poorly fits all”? This became especially clear to me when I started using a compression sleeve on my arm. Like any hacker, this seemed like something I could fix, so I gave it a shot. Boy, did I learn a lot in the process.

A little over a year ago, I started dropping things. If I was holding something in my left hand, chances were good that it would suddenly be on the ground. This phenomenon was soon accompanied by pain and numbness, particularly after banging on a keyboard all day.

At best, my pinky and ring fingers were tired all the time and felt half dead. At worst, pain radiated from my armpit all the way to my fingertips. It felt like my arm had been electrocuted. Long story short, I saw a neurologist or two, and several co-pays later I had a diagnosis: cubital tunnel syndrome.

Continue reading “Stretching My Skills: How (and Why) I Made My Own Compression Sleeves”

Mechanisms: Mechanical Seals

On the face of it, keeping fluids contained seems like a simple job. Your fridge alone probably has a dozen or more trivial examples of liquids being successfully kept where they belong, whether it’s the plastic lid on last night’s leftovers or the top on the jug of milk. But deeper down in the bowels of the fridge, like inside the compressor or where the water line for the icemaker is attached, are more complex and interesting mechanisms for keeping fluids contained. That’s the job of seals, the next topic in our series on mechanisms.

Continue reading “Mechanisms: Mechanical Seals”

Character Generation in 144 Bytes

[Jaromir Sukuba]  has an awesome BrainF*ck interpreter project going. He’s handling the entire language in less than 1 kB of code. Sounds like a great entry in the 1 kB Challenge. The only problem is the user interface. The original design used a 4 line character based LCD. The HD44780 controller in these LCDs have their own character table ROM, which takes up more than 1 kB of space alone.

[Jaromir] could have submitted the BrainF*ck interpreter without the LCD, and probably would have done well in the contest. That wasn’t quite enough for him though. He knew he could get character based output going within the rules of the contest. The solution was a bit of creative compression.

bf-cs-3Rather than a pixel-by-pixel representation of the characters, [Jaromir] created a palette of 16 single byte vectors of commonly used patterns. Characters are created by combining these vectors. Each character is 4 x 8 pixels, so 4 vectors are used per character. The hard part was picking commonly used bit patterns for the vectors.

The first iteration was quite promising – the text was generally readable, but a few characters were pretty bad. [Jaromir] kept at it, reducing and optimizing his vector pallet twice more. The final design is pretty darn good. Each character uses 16 bits of storage (four 4-bit vector lookup values). The vector pallet itself uses 16 bytes. That means 64 characters only eat up 144 Bytes of flash.
This is exactly the kind of hack we were hoping to see in the 1 kB challenge. A bit of creative thinking finds a way around a seemingly impossible barrier. The best part of all is that [Jaromir] has documented his work, so now anyone can use it in the 1 kB challenge and beyond.

1kb-thumb

 

If you have a cool project in mind, there is still plenty of time to enter the 1 kB Challenge! Deadline is January 5, so check it out and fire up your assemblers!

 

[CNLohr] Reverses Vive, Valve Engineers Play Along

[CNLohr] needs no introduction around these parts. He’s pulled off a few really epic hacks. Recently, he’s set his sights on writing a simple, easy to extend library to work with the HTC Vive VR controller equipment, and in particular the Watchman controller.

There’s been a lot of previous work on the device, so [Charles] wasn’t starting from scratch, and he live-streamed his work, allowing others to play along. In the process, two engineers who actually worked on the hardware in question, [Alan Yates] and [Ben Jackson], stopped by and gave some oblique hints and “warmer-cooler” guidance. A much-condensed version is up on YouTube (and embedded below). In the links, you’ll find code and the live streams in their original glory, if you want to see what went down blow by blow. Code and more docs are in this Gist.

Continue reading “[CNLohr] Reverses Vive, Valve Engineers Play Along”

Squoze Your Data

I have a confession to make. I enjoy the challenge of squeezing software into a tiny space or trying to cut a few more cycles out of a loop. It is like an intricate puzzle. Today, of course, there isn’t nearly as much call for that as there used to be. Today even a “small” microcontroller has a ton of memory and resources.

Even so, there’s still a few cases where you need to squeeze those last few bytes out of memory. Maybe you are trying to maximize memory available for some purpose. Maybe you are anticipating mass production and you are using the smallest microcontroller you can find. Or maybe you’re doing the 1 kB Challenge and just want some advice.

One way to find techniques to maximize resources is to look at what people did “in the old days.” Digital Equipment computers once had a special character set called Squoze (or sometimes DEC Radix-50). This technique can be useful when you need to get a lot of strings into memory. The good news is that you can reliably get 3 characters into 2 bytes (or, as DEC did, 6 characters into 4 bytes). The bad news is that you have to pick a limited character set that you can use. However, that’s not always a big problem.

Continue reading “Squoze Your Data”