While earlier smartphones seemed to manage well enough with individual applications that only weighed in at a few megabytes, a perusal of the modern smartphone software store uncovers some positively monstrous file sizes. The fact that we’ve become accustomed to mobile applications requiring 100+ MB downloads on what’s often a metered Internet connection in only a few short years is pretty crazy if you stop to think about it.
Seeing reports that the Nest app for iOS tipped the scales at nearly 250 MB, [Alexandre Colucci] decided to investigate. On his blog he not only documents the process of taking the application apart piece by piece to find out just what’s eating up all that space, but lists some potential fixes which could shave a bit off the top. Even if you aren’t planning a spelunking expedition into your pocket supercomputer’s particular variant of the Netflix app, the methodology and tools he uses here are fascinating in their own right and might be something worth adding to your software bag of tricks.
By passing the application’s files through a disk usage visualizer called GrandPerspective, [Alexandre] immediately identified some rather large blocks of content. The bundled Apple Watch version of the app takes up 23 MB, video and audio used to walk the user through the device setup weigh in at 22 MB, and localization files for various languages consumes a surprising 33 MB. But the biggest single contributor to the application’s heft is the assorted libraries and frameworks which total up to an incredible 67 MB.
Of course the question is, how much of it is really necessary? It’s hard to be sure from an outsider’s perspective, but [Alexandre] notes that a few of the libraries used seem to be redundant or obsolete. In some cases this could be the result of old code still lurking in the project, but the four different libraries used for user tracking probably aren’t in there by accident. It also stands to reason that the instructional videos could be offloaded to something like YouTube, so that only users who need to view them have to expend their bandwidth on it.
Getting a little deeper into things, [Alexandre] notes that some of the localization images appear to be redundant. As a specific example, he points to the images of the Nest itself displaying Fahrenheit and Celsius temperatures. While logically this should only be two image files, there are actually eight copies of the Celsius image, each filed away as language-specific. These redundant localization images could easily be stripped out, but with gains measured in only a few hundred kilobytes, it probably wasn’t considered worth the effort during development.
In the end there’s really not as much bloat as we might like to believe. There were some redundant files, maybe a few questionable library inclusions, and the Apple Watch version of the app could surely be separated out. All together, it might get you a savings of 30 – 40%, but still not enough to bring it down under 100 MB.
All signs point to the fact that modern smartphone software development is just a lot more burdensome than us hackers might like. Save for projects looking to put control back into the hand’s of the users, it looks like mobile operating systems aren’t going to be slimming down anytime soon.
This article reminded me of the other article about everything slowing down and bloating etc, couple month ago here on hackaday.
https://hackaday.com/2018/09/22/one-mans-disenchantment-with-the-world-of-software/
if a correctly implemented app would give you a 30-40% space savings, now expand that to all the apps in your phone. A 30% increase in free space in the phone is not negligible, and possibly said apps would also run smoothier ( and with fewer bugs, one can hope ) .
I for one wish that Android software, or Google’s store, would report the size correctly. I pull down an app that claims to need 32 megs, and I loose 300 on the actual device without the app downloading anything on its own.
I’d be satisfied with “can move to SD”. Surprisingly many can’t (Google especially except for “Google Books”).
I’d be satisfied with Android not measuring the files on my SD card as belonging to the same filesystem as on the phone’s memory, so I could actually see how much space I have available.
Unfortunately that’s a Linux limitation – the filesystem is a mess and asking “how much free space overall do I have” doesn’t have a simple answer because different devices are lumped into one tree with links and virtual files going every which way making it nearly impossible to tell what there actually is.
Huh? df will happily tell you how much free space each filesystem has.
Yes, but where exactly is it?
When I ask my phone how much space my photos or music or downloaded files take, it won’t tell a difference between what’s on the phone’s storage and on the SD card because they’re sort-of-merged. The system manager app tells me I have 1 GB free, yet Google refuses to install updates because I’ve “ran out of space”. None of the software can tell what’s actually there and what isn’t.
I suspect that there really IS “as much bloat as we might like to believe” hidden in all those dependencies built into just about any modern code on just about any hardware platform. Consider the following rant: http://tonsky.me/blog/disenchantment/. Note that ‘bloat’ isn’t just about size – it’s also about poor programming practices that make everything run slower, sometimes by orders of magnitude. Then there’s this: https://medium.com/s/silicon-satire/i-peeked-into-my-node-modules-directory-and-you-wont-believe-what-happened-next-b89f63d21558 . It’s specifically about Javascript, but I’m pretty sure the same failure to sweep out the cruft applies to a lot of code written in a lot of other languages.
” It also stands to reason that the instructional videos could be offloaded to something like YouTube, so that only users who need to view them have to expend their bandwidth on it.”
Like I said somewhere else, a lot of what’s on Youtube was placed there by companies for advertising, or some other benefit to the company. Youtube basically is a free dumping ground for all things video.
Mobile app developer here.
I am in the luxury position that I work for a finance company as a mobile developer, directly (i.e. No contractor, no freelance) I have full control over what I include in the app. Even in this relaxed situation there are management decisions on what to include as a feature in the app for our customers. Some of these features require me to include the SDKs and or Libraries of our contractors (3rd party services) that we include in our app.
Our core app, without any additional libraries would be around 6 MB including any required system frameworks.
One really pesky example is that we are legally bound to check the identity of people whose money we handle. This check of identity is a one-off action during registration. No user will need to get their identity checked over again as long as they keep their accout. We have included the services of a 3rd party provider that does this modern video-call identity check. Unfortunately, where human interaction happens, quality goess down. All tested identity-checking-services are quite crappy on the uptime and availability. Starting next year we have the prepaid-sim-card obligatory registration starting and probably all services we currenly use will be exhausted as even present and active prepaid sim cards will need to be re-registered.
Since these identity-checking-services are so notoriously unreliable in terms of availability we have now included three (3!) of them in our app. Each of them requesting to include their own libraries. Each sizing wetween 8 and 12 MB. Of these SDK’s in turn two include regular WebRTC for the video communication stream, in different versions(!). Both of the SDK’s have WebRTC hardcoded(!) internally in the library (not referencing), on top they did not even bother to change the namespaces.
Additionally to these three identity-checking-services we also include a 20+ MB OCR library that allows the user to scan their identity card or passport so we can have their preliminary registration without the need that they enter their basic data manually. All these libraries are included for first time sign-ups only. Not for regular use.
All of these are added as management decisions. All 3rd party service companies offer their services only with their precompiled crappy SDK. None are open to discussion to not include common parts. Most of them don’t even have proper testing in place and we report them bugs all the time or reject the inclusion of new versions until they fix their libraries.
I have 4 versions of an HTTP library in my app just because 3rd part libraries need to use it (in their own version).
I have meanwhile 3 versions of the QR-Code scanning library ZXing included (of which one lies completely dormant !) just because the 3rd party libraries need to be monolithic and have to be included as a complete set.
Talk to the SDK developers about library bloat, not to us App developers.
And why can’t these checks be done inhouse? Even “All of these are added as management decisions.” can be countered with an “increased security” never mind cost savings by NOT using a third-party. Especially three of them.
Actually legal requirements for an identity check demand that the check is done by a 3rd party. Some much much bigger companies do this in-house but the legal requirements for that are way beyond what we can handle.
We can still talk to you app developers. If the bloated libraries/applciations are only used once, why can´t YOUR app use them in a way that after that one-time use they are removed from the phone ? Doesn´t help much as for the need to download a big app, but at least it will not lie unused and wasting space in the device.
Mobile app development does not work that way. You can remove files you have created after installation (ie. photos) but your app cannot remove parts of itself. Your app does not have access to the file system at that level. Even if it did have, signature checks would fail and your app would stop working.
I stand to be corrected, but this is how I know things are working. There are new mechanisms being developed that might work on new mobile phones that will only load parts of the app on the fly as they are needed. But this functionality will not be backwards compatible to your existing phone.
Hence cloud, cloud, cloud. Always connected really does alter things, much like always on altered personal computing.
Do you ever go anywhere where you don’t have network connectivity? I do, and not just because I want to go somewhere to “disconnect.” Networks impose a real and non-concealable cost.
@ David Brown.
No doubt. Some of those are even great vacation spots.
However it doesn’t take Nostradamus to see where things are headed, and if one lowers the bar from “needs to stream Netflix” a lot more phones are reachable in the context of “phone apps”. Doubly so if the heaviest stuff never leaves home.
Install the registration as a separate app?
My thought exactly.
+1.
Or the registration as additional files, even if they need to be downloaded after the main app install. If one app can create and delete files, the necessary ways to be able to run code from those files must exist..
Please note that I am not saying this is easy. Just a better / more correct way to do things. Just installing a bunch of crap because it is needed once or because it comes along with some necessary library is not a good practice. We know the marketing dept. may ask for it, but we also know they are morons.
Also with the option to do the registration on a PC with a webcam with standard WebRTC… no app install at all.
As far as that goes 81% of mobile users have WebRTC, built into the browser with zero additional software installed, there is no reason not to do the registration with a web app for both desktop/mobile and be done with it. https://caniuse.com/#search=webrtc
So how convoluted do you want to make this process? “Here, use our app, it’s easy! Whoops, you actually have to download a separate app too before you can use ours. Hold on, these are synchronizing with one another so we can share data. Okay, that didn’t work for some reason, maybe they put out a new version or something. Try another third party app?”
(user uninstalls everything)
This. Also, I’m not a developer (don’t know the details), but I have installed extensions for some Android apps. Can’t it be done for this? “You installed our app. Already registered? Good, just sign in. No? Please tap here to install this 300MB extension too. You can remove it later.”
(sigh!)
I fear that most commenters here didn’t understand what Kollege Müller explained.
As a developer you often are not FREE to do the right thing. And you cannot grab a pump gun and run up to 6th floor just to make those who could allow you to do it right ALLOW you to do it right.
Sometimes (in my experience as an external contractor – not a “freelancer”, I don’t do cheap) there are exactly two options: Do as you are told to do or don’t. And get a new job.
It’s not like “reducing the size of the app” gets the highest priority. Usually “it has to work” is quite a bit further up the list. “It has to work the second time, too” is there, too. “It has to work in unknown conditions” is there, too. There’s a lot of “it has to work”, long, long, LOOOOOOOONNNNNGGGG before anything even mentions “size of the app”.
Yup. It’s not a sound business decision to mangers. Why spend developer time (aka money) to change things that won’t improve the bottom line? If there’s no incentive then it won’t happen. Think e-commerce – bounce rates are correlated to page speed so there’s incentive to optimize it.
We really understood it correctly. Almost everybody here may already been in a situation of having to do what is demanded of them with little input on the situation.
But even if you are just told to dig a trench, you can either do it correctly, or just dig a very bad trench.Or do it using the wrong tools.
When the culture of just doing things the easiest way instead of the better/correct way permeates from above, from upper management and down, it surely is hard for a little cog the middle that knows how it must be done. But it is even more important then for that little cog to strive to do its job the correct way. Sometimes we do it the “bad” way , and it´s ok, everybody needs money. But what was done is still a bad/wrong/incorrect job. “Following orders” can´t be an excuse only sometimes.
Developing and testing most of these libraries is often a huge effort on it’s own, bigger than the app itself in most situations. You can’t expect a company to reinvent the wheel for every feature they want to implement.
I do agree that libraries should be modular to include only the features you need (Though this can also be a challenge for library developers).
FYI,
my (sigh!) was one of sympathy for Klaus.
All of that sounds like a problem with immutable dependancies. In a sanely package managed environment, things could just use the latest version, and if it breaks, you downgrade. Preferrably without downgrading the version that other packages use.
The whole NixOS “Exactly this package version and nothing else” idea is just AppImage with extra steps.
when i write code i avoid solving problems with the #include statement. unless of course its a library that i wrote. i didn’t learn to code so i could put legos together.
That belongs on a t-shirt: “I didn’t learn to code so I could put Legos together!”
Maybe in the form of an XKCD? ;-)
The Legos help me build things I couldn’t reasonably do in my own time. I want to make a game, not fiddle with hardware interfaces.
If I have seen further than others, it is by building on the shoulders of giant Lego sculptures.
LEGO
(always capitalized, never pluralized)
What shape is your version of the wheel then?
One fix might be to require app developers to code at least one Atari 2600 game a year – that might force them to remember to pay attention to code size and efficiency.
B^)
I wish I would get that much time from my employer to spend on my hobbyist projects..
Write a Doom clone in debug :)
Reminds me of the “Flight Simulator” Easter egg that was in a version of Excel.
“In the end there’s really not as much bloat as we might like to believe”
umm, 4 libraries for user tracking IS definitely bloat and that’s before dependencies that have their own dependencies. I think that a major problem is that software development is now treated like lego building with each company making their own little part of the code that has its own dependencies (quite often the same snippets of code at different versions) and because everyone wants their piece of the pie none of it can be changed to be more efficient. On the other hand, it does encourage the end user to be a bit more cautious with the things they download. I choose the minimum number of apps possible to make my phone do what it needs to do and that’s it. Between lineage OS and f-droid my use cases are all taken care of, remember this article is about the nest app just think about the kind of people who need to use the nest app.
I had to buy a new phone yesterday because 8 GB was not enough to even install updates anymore. Oh DOS, where art thou …
Who needs DOS ? I’m still using a Nokia N9 :)
It’s a single core ARM oled device running Maemo. It’s as fast as my 4 core spare Android phone and the battery lasts a lot longer.
Because it’s real Linux under the hood the N9 is easy to navigate after turning on developer mode. The audio is so good I use it to record my u-tube videos and SCP the file to my FreeBSD workstation for post processing.
The Nokia N9 is dead … long live the Nokia N9!
I’ll just drop this here.
http://designingmlearning.com/resources/ZenOfPalm.pdf
Mobile applications don’t need to be feature rich.
I think a lot of this bloat would go away if wireless networking was much more expensive so less focus would be put into including APIs of diffrent online services.
A corollary to this would be an increase in personal information stored on the device and not on distant servers.
That’s what drives me nuts. I made an app that’s around 4 megabytes, and it does thinks like read twitter/facebook/weather services/etc, and individual apps that do those things are roughly 50 megabytes.
I lost interest in Android programming when my “Hello World” program was 1 Mb in size.
So what’s the ultimate solution?
I ended up culling out a bunch of apps due to the bloat… I don’t need a 120mb banking app when I can just get the same data from the website using the browser. But I’d like to have the
Or worse. The website has more functionality than the app.
A lot of banks are now using the App for the 2FA on the website. Good luck using the website for anything useful…
200mb for a thermostat? What a time to be alive!
LOL!
The problem is that all those web APIs and backends you need to talk to expect the inclusion of their own bloated SDK libraries. Instead, one could just use those APIs over, say, JSON-RPC or something like that and not use dedicated libraries for every single call.
I agree, part of the problem, if it is a problem, most apps are built in a ‘click and create’ environment with a small amount of hand written code.
If Apple were to stop making up new languages and just used C/C++ and Android dropped Java then the world would be so much faster. Yes I know you can use native code on these systems but you always have to include some managed code in your app and so have the hit of uncontrolled library inclusion.