The smartphone represents one of the most significant shifts in our world. In less than thirteen years, we went from some people owning a dumb phone to the majority of the planet having a smartphone (~83.7% as of 2022, according to Statista). There are very few things that a larger percentage of people on this planet have. Not clean water, not housing, not even food.
How does a smartphone work? Most people have no idea; they are insanely complicated devices. However, you can break them down into eight submodules, each of which is merely complex. What makes them work is that each of these components can be made small, at massive economies of scale, and are tightly integrated, allowing easy assembly.
So without further ado, the fundamental eight building blocks of the modern cellphone are: the application processor, the baseband processor, a SIM card, the RF processor, sensors, a display, cameras & lenses, and power management. Let’s have a look at them all, and how they fit together.
Application Processor
When you think computer or, by extension, smartphone, you think of a single cohesive unit powering what your smartphone does. However, even putting aside the universal multicore processors, the processing inside your phone is not a single voice but a cacophony of voices chattering away at each other. The application processor (AP) is what we think of as this traditional brain. It features a large multi-core CPU, almost always ARM-based.
These APs might come as part of an extensive system on a chip (SoC). The SoC can include memory, the GPU, digital signal processors, image processing units, the modem (which we will get to later), and other minor things like video decoders and AI engines.
But back to the CPU. It runs your phone’s operating system and is primarily the ring leader in your phone’s decisions. These days, it often has big powerful cores that consume much more power and small cores that consume much less energy. This is usually done by making the smaller cores single dispatch, in-order, while the larger cores are massive multiscalar out-of-order cores. We could spend quite a long time breaking down the various parts of a modern processor, but we think you might learn better by making your own or fixing a broken simulation.
Baseband Processor
The baseband processor (BP) is an entirely separate processor that the AP can talk to, often referred to as a modem. Even the iPhone, which has its own custom AP, uses a Qualcomm or Intel BP. Generally, Wi-Fi, Bluetooth, and other radio-type communications are handled by the AP and the BP focuses entirely on cellular communications. Even the specs for the ancient 3G protocol are long and dense, with each revision adding intricacy on top. Complex power amplifiers, multiband-multimode transceivers, and other techniques add to a complex circuit.
Baseband processors have their own firmware and RAM, and the codebases are opaque manufacturer-specific binaries. There’s a good talk on hacking an old 3G USB modem. In fact, some of the Intel modems in some iPhone models had x86 processors in them. [Comsecuris] detailed a buffer overflow they found in many iPhone modems that gives some helpful information on the internal workings.
SIM Card
We think of them as just being storage, but the small piece of carrier-branded plastic with the handfull of gold contacts has a small chip inside it. It’s all according to spec, but the interactions between AP, BP, and SIM cards has grown quite complex. SIM cards come into the security limelight, as they are updated over the air with little control from the user. They can talk to the BP without the AP knowing. Can you imagine your phone calling or texting someone, and the only way you would know would be by auditing your phone records? Your phone already does so. We’re slowly moving away from SIM and towards eSIM, but that’s a ways off for most of us.
Sensors
Sensors make your phone much more aware of its surroundings. Cameras, GPS, microphones, distance sensors, proximity sensors, gyroscopes, accelerometers, altimeters, magnetometers, LIDAR, fingerprint sensors, radar sensors, ambient light level sensors, and pressure sensors are all common additions to adorn our smartphones.
The amount of data streaming in can be truly astounding. Many of these sensors have their own microprocessor to handle the load and present a more cohesive view to the AP. In fact, some phones have coprocessors to handle these different sensors in more power-efficient ways. So we have the AP talking to a coprocessor, talking to microprocessors, and reading sensors. All these sensors go a long way towards making a smartphone much more useful in day-to-day life. You can call, get directions, rotate for portrait mode, or use your phone for physics experiments.
Display
Smartphone screens are a marvel. They have a resolution that matches or beats many household TVs while being high quality and affordable. Used phone screens are often used in projects here on Hackaday as they’re easy to find and then drive. We see advancements here with OLED displays and microLED displays. As the screen is often the front of your phone, having it heal itself would be pretty handy. There’s nothing exceptional about the displays in smartphones as they’re just miniaturized versions of larger panels.
Cameras
Modern cameras on smartphones are crazy.
As an example, modern smartphone lenses are crazy. Here's the design used on the iPhone 7; every single lens element is highly aspherical with a very large number of correction coefficients; this lens completely breaks the existing poly optics implementation from the 2016 paper. pic.twitter.com/cYr9qsa5Bj
— Yining Karl Li (@yiningkarlli) February 27, 2022
Lenses and sensors are unique in many ways. Unlike displays, where you can shrink the process, light and lenses get weird when they’re small. So, how do you capture more light in a smaller area? An incredible (though long) resource on lens design is here that details how we do that. The lens design is a complex art with a long history of iteration and improvement since the early 1800s. There are several notable inventions listed in the link above. We could do a whole article just about camera lenses. In particular, the long exposures of old cameras had just as much to do with lens design as it did film.
The lens and sensor are going crazy, but the Image Signal Processor (ISP) that processes the photos is going crazy as well, in some cases blatantly lying. Like a traditional Signal Processor, the ISP takes an image and processes it, applying some transformation. However, while some filters are done on the AP — think the Snapchat filters — the actual reading of the sensor and forming a picture are done by the ISP.
Power Management
Phones are mobile and expected to last a long time on battery. Like other battery-powered products, there’s a battery management system (BMS) that supports the battery. However, there are some pretty high demands placed on it. It has to charge wirelessly and at incredibly high rates and often. On the flip side, efficiency is considered in all the other parts that we’ve discussed. There are power modes on every module. Every component is scrutinized against the budget, both monetarily and against the power budget.
Put it All Together and Throw it Away!
In our modern society, we replace our smartphones frequently, and our world seems to have no end to our hunger for them. So with more and more discarded smartphones on hand each year, packed with features and processors, why aren’t we hacking on them more? They make a perfect companion for your 3D printer.
There have been some attempts to make smartphones modular, allowing you to swap out the baseband processor, the display, or the camera. However, the concept seems to have stalled. Particularly the application processor, baseband processor, and sensor unit are so tightly connected in many modern phones, that swapping out any of the component parts would be like redesigning the whole thing from scratch. Smartphones are valuable because of their scale and tight integration, and it’s just hard to beat that.
Still, it’s incredible how far we’ve come in smartphones in general. We at Hackaday can’t wait to see what the next decade holds.
Bring back easily replaceable batteries, make it easy to load the operating system of one’s choosing, and hardware switches to disable the modem, mics. and cameras.
I think you’re looking for the PinePhone (pro).
I’ve considered getting one Last I’ve looked, there was still a bit of work to be done software-wise. Not sure I want to dump over $400 USD on something with only a 30-day warranty.
Yeah it is a bit of a hobbyist device, I was only half-serious. From what I hear the core functionality as a phone (sms, calling) is actually quite solid in a number of the distros ATM, so if you you’re comfortable installing a distro on a pi it would be usable as a phone.
Hardware decoding, camera functionality, suspend functions and efficiently running apps in the background required more work to be polished last I checked, so using it a daily driver smartphone is a little more of a stretch.
The warranty wouldn’t be an issue to me, but I can understand the worry.
My next phone will likely be a pixel w/ lineageOS (unfortunately my current phone doesn’t have a build), I am too spoiled with modern phone performance to go Pinephone myself. Hopefully the EU law will have forced a shift towards removable batteries by then.
while you are at it, also add a handle a decent powerplug and a screen that’s 10 times larger, that might help too. This way we don’t have to use these large fonts any more and we can fit more onto a webpage without having to worry that nobody can’t read it. And ohh… a decent keyboard because touchscreen interfaces make me feel like an idiot, where I sometimes need to hover or swipe in certain mystery locations in order to make a button or additional functionality to appear. Or sometimes just press and hold, but not too long, not not that!
Back in the days when buttons had names and software came with manuals things were made to be recognizable, but then somebody thought it was a good idea to replace the text settings by an icon (good idea if combined with text underneath, but not really a space saver, but it looks nice).
Back then menus were menus and were located in bars on the top of the screen, they were called pull-down menus, because you pulled the items down from a list. But now you have an almost empty bar (if you are lucky) that holds 3 mysterious lines. Took a few days before I realized you could press on that, yes I know I am an idiot, how could I not know that 3 lines mean “menu”, because everybody knows from the moment they are born that three lines of 6 pixels wide represent a “menu”! This is present in the top right corner of the page… well mostly because sometimes there are no additional settings… or there are but they are hidden.
If you don’t have a bar, you can swipe from top to bottom anyway and see all sorts of functions appears, cool, great, intuitive, NO, because if you do not know that how would you get the idea to just do that. Hmmm… I wonder what happens if I draw a circle on the top left center of the screen. If nobody tells you that, how would you know what to do?
Touchscreen interfaces… please don’t talk about cold weather, where my cold fingers aren’t registered by the touchscreen and I can’t even unlock the darn thing. And browsing a website on a phone, is fun… scrolling all day long (which surprisingly works very well) until you decide to go back a page and then oops… one page too far back… how do I go forward?!? Ahhh never mind. I’ll use my PC when I get home.
I miss the times of manuals supplied with software, I miss my rotary phone, don’t miss the dial up connection, but the days of pull-down menus were great. Now with push-up menu’s and things called taskbars life just isn’t the same.
It’s unfortunate that there are online translators from Sheep to English, it wasn’t worth it to read your simping for Corporations that are forcing closed off hardware/software that spies on it’s user onto us.
I bet you also think bluetooth earbuds are an “upgrade” over good wired headphones and that a headphone jack is “obsolete” just because that’s the excuse that Apple’s marketing gave lol.
https://pbs.twimg.com/media/EcgsWpQXYAAfF3R?format=jpg&name=medium
IKR? WTH is this ‘Wereweeb’ prating on about? Railing against something other than what the previous poster stated.
Ima call it a ‘misunderstanding’ & move on…
I feel a need for an ADA compliant English or other language text only mode for operating systems and file browsing. All text can be spoken by a voice-assistant and thus blind enabled. Pretty icons are for enhancing ADHD and are a distraction to many others.
I can have the overflow list of links on my toolbar which is a vertical list have room to write a time-day-title for a certain radio stream. Details! With extra-wide screens why do we need the bottom or the top used for inefficient short tags describing things along with the pretty pictures. A detailed list running down the left side would suit many. No tiles. Netscape of old and Firefox till about 5 years ago supported text only mode, then it went away.
Those three-stripe things are called a “hamburger menu”, and are meant to look like what the menus at a fast-food restaurant do when viewed from a distance (you don’t see the text, you just see a line).
_But_, unless you’re used to tapping on one of those, indeed, you might not even recognise it as being a button. I think that’s more UI design failing to tell you “this is a button”.
Back in the early 90s, we learned to look for blue underlined text… and by the time I got connected in the mid 90s, imagemaps were a thing as were changing the colours of hyperlinks, so you also learned to hover the mouse over areas and watch for the arrow changing to a pointing hand.
Hovering your finger over a link on a touchscreen does not work due to technical limitations.
Before the world-wide-wait… buttons usually were pretty obvious. They might’ve used a word surrounded by brackets to “mark” them as being active. Perhaps different colours. That was in the early days of text UIs.
GUIs of the day at minimum drew a box around the text. By the mid 90s, this box was filled with a solid colour, and a highlight/shadow was added to the borders so it “looked” 3-dimensional. It was obvious it was a button, even without a “hand” icon.
UI design has forgotten this, and so yeah, we’ve got a completely flat interface. No clues as to what responds to clicks (or taps), and what doesn’t. That hamburger menu button might be more obviously a button if it were bordered with a 90’s style fake-3D border or a differently-coloured box (optionally with drop-shadow), but by making it “blend in”, it just looks like a (not-so) “pretty picture”.
These are mistakes that have been known about for a long time: e.g. see http://hallofshame.gp.co.at/shame.htm — that’s not the original home for that site but I think its authors would have a field day with today’s application UIs.
Modern UI design seems to be more about “looking good” without being functional. Time to bring back some retro to our interfaces methinks.
The term “Hamburger menu” brings back a memory. Back when “icons” first became popular, I stopped at a fastfoodburgerjoint and on the wall behind the counter was a homemade chart of the ‘icons’ they used along with an English definition. Apparently some employee put it on the wall out of self defense. Thought that was hilarious.
Whilst they’re at it… put a little U.fl connector on the board to connect the antenna with and maybe make provision for a U.fl→SMA pigtail for an external antenna.
Not everybody lives a block away from a cell tower.
The one and only phone I’ve used that did something like this was the KitePhone… which is unobtainium despite being an excellent device.
Very yes. How many bad movies have you seen where someone’s out in the boondocks holding their phone to the sky looking for bars?
Anyone who’s worked with radios knows the importance of the antenna. But sadly, most with a phone don’t know it is a radio — just that it is wireless.
Application -> General Computation
Baseband -> Wireless
Clarity feedback. Industry vs layperson comprehensibility. Thanks for the article!
How do they figure the 83.7%? From sales? I’d say it’s more likely that 2 billion folks are averagely on their 3rd smartphone.
Good article. Thanks.
Amazing what’s in them, and more importantly affordable to the masses.
https://youtu.be/fCS8jGc3log
what’s sad though is DRM-boot, Locked bootloaders. Old GPL viollating kernels, or worse userspace proprietary drivers. Closed firmware blobs.
Not to speak of modems sharing ram with the main SOC.
This makes old phones almost impossible to re-use. If you can’t toggle a gpio, how are you gonna re-use it?
So right to repair is crucial, but bypassing secure boor/ ability to unlock bootloaders is even more crucial.
In addition, right to repair won’t save our landfill)s from phones that are fine in hardware, but “broken” in software …
You forgot to discuss the RF processor. I presume this is part of the baseband processor?
Most of these components are available as USB devices, and USB 3 clearly has the bandwidth to handle the screen and WiFi. So you could build a modular phone around USB 3, in principle. I haven’t looked into the details but I expect this approach might strain the power budget.
In the late 90s, ubiquitous data and a versatile mobile device seemed like a science fiction utopia. I really lusted after a Nokia 9000 communicator. That thing was awesome. Clamshell, real antenna a hardware keyboard. Now, 20 years later, we have the locked-down cesspool that is the modern smartphone. Disposable hardware, packed to gills with surveillance, that benefiits everyone except the user of the phone. Advertising and spyware abound. We dreamed of so much better in the 1990s, where did we go wrong?