Recently, I stumbled upon a cool write-up by [DHowett], about reprogramming a Framework laptop’s Embedded Controller (EC). He shows us how to reuse the Caps Lock LED, instead making it indicate the F1-F12 key layer state – also known as “Fn lock”, AKA, “Does your F1 key currently work as F1, or does it regulate volume”. He walks us through adding custom code to your laptop’s EC firmware and integrate it properly into the various routines the EC runs.
The EC that the Framework uses is a MEC1521 chip from Microchip, and earlier this year, they open-sourced the firmware for it. Now, there’s a repository of microcontroller code that you can compile yourself, and flash your Framework laptop’s motherboard with. In a comment section of HackerNews, a Framework representative has speculated that you could add GPIOs to a Framework motherboard through EC firmware hacking.
Wait… Microcontroller code? GPIOs? This brings us to the question – what is the EC, really? To start with, it’s just a microcontroller. You can find an EC in every x86 computer, including laptops, managing your computer’s lower-level functions like power management, keyboard, touchpad, battery and a slew of other things. In Apple land, you might know them as SMC, but their function is the same.
Why have we not been reprogramming our ECs all this time? That’s a warranted question, too, and I will tell you all about it.
What’s The EC’s Job?
The EC controls a whole bunch of devices in your laptop. Not devices connected to USB, LVDS/eDP or PCIe, because those would fall within the purview of the chipset. Instead, these are devices like power switches, the charger chip, and various current monitors, since these have to work correctly even when the chipset and CPU are powered off. But of course, it’s not just power management – there’s a whole lot of things in a laptop you need GPIOs for.
Generally, anything that you’d control with a digitalWrite or monitor using a digitalRead, measure through an ADC, or talk to using I2C – these are things handled by the EC. Thus, the EC reads battery state and charger voltages, drives the fans with PWM and takes temperature measurements from various sensors. The laptop keyboard is a key matrix, and the EC scans that matrix and processes key presses, forwarding the key events to the chipset that your OS then reads. Whether your touchpad is PS/2 or I2C, the EC handles it and exposes it to the OS, too.
Your laptop’s power button is connected directly to the EC. As a result, your EC is the first thing to get powered on; and if a broken laptop of yours has no reaction to the power button, it means the EC can’t do its power management job for whatever reason. In fact, if you check Framework laptop’s recently published reduced schematics, you’ll see that the EC has it own separate power rail coming directly from the battery.
How does it even talk to the chipset? For about two decades, ECs have been using the LPC bus – a four-bit wide bus superficially resembling qSPI. Apart from ECs, it’s only really been used by TPMs in the recent times. LPC uses frequencies from 25MHz to 100MHz. Thus, if you want to put a logic analyzer to your LPC signals and capture some packets, your typical cheapo 25Msps LA won’t do, but an off-the-shelf FPGA board or a way faster LA will work wonders, and there’s a pretty cool paper using LPC manipulation and an FPGA to extract keys from TPMs.
LPC is about two decades old, and is a direct successor to the ISA bus – in fact, in some laptop schematics from 2003 you’ll find the EC connected through ISA instead, but it’s all LPC beyond that. However, recent ECs talk eSPI instead, a qSPI-like interface meant to replace LPC, and the Framework EC talks eSPI, too.
Of Course, There’s Firmware Involved
Every EC has firmware, and every laptop (and desktop, and server!) has an EC. The EC firmware is nearly always closed-source. As such, the EC firmware is one of the binary blobs we tend to miss when talking about proprietary parts inside our computers. Often, the EC firmware is stored on the same SPI flash chip as the BIOS – other times, there’s a separate external or on-chip flash, in which case, you typically have an UART bootloader you can reflash your EC through. All of that depends on which specific manufacturer and model of the EC you have.
Often, your EC is built on something like ARM or 8051 architecture, other times it’s something more obscure like CompactRISC. The common thing is – at most, you’ll get a binary blob when it comes to your EC’s firmware. At some point, when Google got into laptop business, a group of their engineers presumably said “enough”, and open-sourced their EC code – which is what Framework has been building on when it comes to their own EC firmware. Last year, System76 opened up their EC code, too. Unfortunately, the situation remains dire for other laptop manufacturers.
Could your EC get backdoored? Not likely – it tends to be harder to modify and update EC firmware than it is to do the same with BIOS images. Now, could you yourself modify your EC’s behavior? It’s at least technically possible, and I’d argue that you should have always been able to do that.
So, What About Hacking?
Of course, with every subsystem of a laptop, you’ll find a subgroup of Thinkpad enthusiasts that have already dug deep and used it to pull off some fun and useful things. The EC is one such aspect, and they sure have something to offer – reprogramming keyboard layouts and removing battery locks, mainly. With keyboard layouts, they’ve managed to make older (and apparently more superior) keyboards work with newer laptops, with a tutorial talking about how specifically you need to insulate certain pins, and a super convenient way to flash the changes.
The battery part is more vital, however – you can more often than not live with a subpar keyboard, even on supposedly otherwise-stellar ThinkPads. The problem is the “genuine” battery check in the EC, which doesn’t let you charge (or even operate from) the battery if it doesn’t pass. This isn’t just limited the third-party battery options, in case that’s what it sounds like – such checks also prohibit use of Lenovo batteries that were just meant for a different kind of Thinkpad, but otherwise mechanically, electrically and electronically perfectly suitable.
There’s a video on how ThinkPad EC hacking unfolded, and I recommend you check that one out to see what’s up. Now, Lenovo didn’t seem to like that people were swapping keyboards and enabling use of third-party batteries that Lenovo themselves stopped selling ‘genuine’ counterparts for anymore. So, at some point, they decided to close one of the most comfortable ways for EC firmware updating, and release a BIOS update citing “security improvements”. The relevant CVE says this:
A vulnerability was reported in various BIOS versions of older ThinkPad systems that could allow a user with administrative privileges or physical access the ability to update the Embedded Controller with unsigned firmware.
If you ask me, this description is bonkers. This sentence essentially means “the laptop’s owner can flash EC firmware not approved by Lenovo”. I do wonder what led to it and what the possible justification might be, but in the end, whatever the reason, it’s a distraction from what I believe. That is, updating the EC firmware on one’s own laptop should be possible, and Lenovo closed a user-friendly way to do just that.
Also, without doubt, not all manufacturers respect your right to repair when it comes to ECs. As an example, for almost a decade now, Dell has been shipping their laptops with ECs that have encrypted firmware, keys fused inside the EC. This has been a particular problem for Dell laptop repair, as EC die every now and then. While you can buy a blank EC and reflow it in place of the Dell’s dead one, it won’t have the decryption keys Dell flashes into the EC at the factory, and therefore won’t run Dell’s encrypted firmware. Modifications are off the table here – it’s not even possible to source a fitting replacement for the EC when your laptop is broken, even though the chips themselves are abundant.
What Can You Do Now?
Now there’s three manufacturers that have open source firmware for ECs – Google, System76 and Framework. What could you do with this firmware, though? As with any underutilized area of hacking, it will take time to realize its full potential. Remapping keys is not the only thing – you could implement a 80% battery charge limit for cell longevity if your laptop’s manufacturer didn’t provide you with one, add extra layers to your laptop keyboard without any need for OS support, maybe tweak your fan curves. Or, indeed, you could add some GPIOs inside your laptop, for whatever sensors or buttons your heart desires.
You can also fix bugs, which crop up in ECs every now and then, and can be quite annoying to deal with – imagine keyboard keys getting stuck every now and then, seemingly randomly, and that’s exactly what happens when you have an EC bug. Bug fixes or improvements, just like with any firmware currently closed to us, we won’t see a slew of cool hacks starting tomorrow, but there are definitely cool things on the horizon when it comes to EC hacking.
> “the laptop’s owner can flash EC firmware not approved by Lenovo”
Don’t see how it’s limited to just the owner.
These are business laptops so it’s expected of them to plug up potential security issues, even on old models.
> Dell has been shipping their laptops with ECs that have encrypted firmware, keys fused inside the EC
I’m actually surprised there are manufacturers that don’t do this.
The ability to install your own OS, disable Secure Boot, or plug in random USB devices is also a ‘potential security issue’. It is inane to view it this way, however.
We are talking about “physical access” threat model when it comes to the specifics of this hack. The laptops affected fail this model on many fronts already. This is not a decision taken with a sane threat model in mind.
Properly working on your product’s security takes consideration – this ain’t it. Google hasn’t fallen out with Big Security crowd after open-sourcing their EC firmware, neither have System76 or Framework. You’d notice that if you looked.
> I’m actually surprised there are manufacturers that don’t do this.
Well yeah, not everybody makes such moronic decisions. Dell’s consumer-grade laptops go to waste because the EC can’t be replaced without a donor. All the other manufacturers don’t feel the need to do that, and sometimes they’re fine to distribute an EC firmware blob, even.
I gotta be really adamant on this. We shouldn’t give excuses to manufacturers’ bullshit decisions. We’re doing each other a disservice if we accept their excuses.
Lenovo closed this way of updating after their community has embraced this way to give old laptops new life. I prefer this to be emphasized quite clearly. There’s nothing good about this restriction for us, and it’s not justifiable security-wise.
“Lenovo closed this way of updating after ”
Lenovo is a special case. They keep their BIOS (and thus EC) firmware encrypted to protect their system backdoors.
They use the UEFI together with a Windows feature called the ‘platform binary table’ (WPBT), to inject backdoored software such as SuperFish and even install custom root certificates so they persist across fresh windows installations.
It’s really fascinating to read about this level of hardware integration.
I wouldn’t expect they care much about the EC firmware specifically, but being in the BIOS updates it comes along on the encryption ride either way.
Ohh that’s interesting! so that’s how they made SuperFish possible, ugh. Got any links that you liked talking about this? It’d be cool to learn more, I wasn’t aware of these aspects!
> BIOS (and thus EC) firmware encrypted
This, I don’t understand. The EC firmware may stored in the SPI chip alongside the BIOS (idk if Lenovo does this often), but if what you mean is that they do encryption-at-rest for BIOS code somehow (not something I’ve heard about), the EC code would still be executed by a different CPU, aka the one in the EC – just stored in the same chip as the BIOS?
…update encryption? I know they do update signing, but… encryption? You’d need to flash decryption keys into the PCH or something, then. Or perhaps the EC decrypts data before passing it to the PCH – but then, that’s easily sniffable and moot, unless a secure channel is overlaid over LPC/eSPI/whatever. Can you help me understand that – perhaps any links that’d help here, too?
Sorry, I was referring to the BIOS updates they push out. Lenovo has a userland updater service that handles decrypting their BIOS blobs and doing the install to the system. I’d assume they are certainly signed as well.
Once it lands in the flash chip I couldn’t say.
As for links to more info, I first heard about it from a BlackHat conference talk in 2015.
Some useful search terms are “lenovo service engine”, “uefi windows platform binary table”, and also “uefi wpbbin.exe”
That last one is the name of the executable that the BIOS injects, from the UEFI data, which windows auto-runs early on in the boot process. Searching just the exe name fills the results with all sorts of AV pages and “what is this file?” pages that look quite shady.
This prefix returns stuff about lenovo and asus which looks useful, but was just the first thing to come to mind.
Interesting post, didn’t realise so much work had got into opening the system up to mods, cool !
Hmm, have a few systems, worth going through the code make sure no vulnerabilities or hidden Bitcoin IDs ;-)
Thanks for posting :-)
Glad to hear you liked it ^__^ It’s really fun what can be done! [DHowett], for instance, is working on new key handler code for the keyboard, and already has a proof-of-concept working. From here, one of the possible routes is that you could implement keyboard layer declaration parsing, QMK-style – maybe with easy user-space updates, too!
I have healthy respect for “hackers’ projects might be never finished”, so don’t treat this as any kind of promise on [DHowett]’s behalf =D But it’s definitely an idea that’s just begging to be implemented, and something we can only work on after the EC firmware’s been opened!
If laptop keyboard custom firmwares become a thing, my X260 could use a menu button…
same! my laptop could use a few custom buttons, too.
> This sentence essentially means “the laptop’s owner can flash EC firmware not approved by Lenovo”.
It also means that a bad actor with physical access can flash firmware (e.g. keylogger) onto your laptop; maybe the “cleaner” at the hotel you’re staying at for that important business/political trip; stuff like that happens. Which for some people is a concern, though admittedly not most of us.
But it also potentially means there’s other security flaws including other routes to install firmware.
But there might be worse issues. If it controls the battery charging, it could potentially be abused to turn a laptop into an incendiary.
Given that thinkpads are (or at least were?) highly regarded in big enterprises, I suspect the security issue might be the real motivation here.
The ability to drum up security concerns doesn’t make you security-savvy.
“To turn a laptop into an incendiary” is a wild and baseless speculation. Please do check a laptop schematic sometime, and read some datasheets of the ICs in it. There’s a myriad of protections against such failure modes, not just for malicious intent, but because hardware fails, and you need to protect components from each other if one of them goes haywire. Physical access can circumvent all of these protections – and it doesn’t need the EC being in the picture for that.
You can still override CC/CV charging with physical access, you don’t need an EC firmware mod for that, even – there’s a lot of things you can do with physical access. Furthermore, the things you *can* do through the EC power-wise with firmware alone, without hardware mods, are actually quite limited. This is because of the internal protections of the charger and power path ICs – the circuits are designing for being switched on and off, not for short-circuiting into each other or something. Plus, the battery itself also has protections for things like overvoltage and short-circuit.
If you’re interested in how much you can actually do through the EC firmware, reading the code for System76 and Framework laptops would give you a good start on what’s actually possible.
As for “other routes”, I’ll quote the article, and expand on the implications of what I hoped you’d notice when reading it.
> one of the most comfortable ways for EC firmware updating,
“one of the most comfortable” is the key part. You can reflash the Lenovo ECs through other ways still, with physical access. They closed down the specific way to do it that was comfortable for end users. For a bad actor with physical access, evil maid style (that you’re referring to), it’s still possible to do flash firmware, using i.e. the UART bootloader of the EC.
They closed the specific way to update firmware that was used by Thinkpad enthusiasts to replace keyboards and batteries, without a sane security justification to do so, and that’s quite apparent to me.
Being business machine ‘sane security justification’ has both a different scale and meaning to home computer vendors – and because it was the method so well loved and documented it may have been a legitimate victim of its own success with exploits being seen that do merit the change.
It might just be anti-consumer BS, but equally when you are shipping thousands, millions or maybe even billions of units, many with support clauses and some degree of liability ‘fixing’ this may actually be 100% required, as its made exploits for your giant cash filled business clients harder, may have actually be required to keep certain clients – like Military and government organisations are often rather picky (with good reason) and thus it keeps you in business.
Yeah, the descriptions in the security community have mostly been about Evil Maid Attacks installing keyloggers or whatever.
I’ve had a lot of laptops over the years, ranging from little ASUS portables to Dell and HP, that are excessively over-concerned about what power adapter is plugged in and whether they’re willing to let you use it to charge your batteries or not.
“Oh, noes, this 65W USB-C or (110W 19.2V barrel jack charger) doesn’t look like OUR 65W USB-C (or 90W 19.2V barrel jack charger), so its electricity might be inadequate to run your laptop! So we’re not going to let you get any power from it even though you’re desperate and want to get something done even if you can’t run Crysis 24 hours straight!”
Won’t be long before someones got a string of WS2812s hooked up to it and are using it as a monitor to browse the desktop :D
=D with the Framework exposing 24 GPIOs (keyboard control pins), I sure hope someone does this!
So many cool repurposed-hardware ideas seem to require cooperation from ECs and bootloaders.
You want to turn an old laptop into a remote solar-powered piratebox-style information kiosk? Gonna have to mod the EC to understand charging from a PV panel that doesn’t behave anything like a normal brick. You want to power off when the battery is on a bad trend, but automatically power back up when the sun returns and SoC climbs above a certain point? Again, the EC controls all that.
For the era before USB-C “docking stations”, docks were a many-pin connector with all sorts of signals on it. And ISTR many of those signals are bus-switched under the EC’s control. So if you don’t need the dock, you could repurpose that connector and signals, if the EC would help out by switching them over to your new peripherals when desired.
Shall we talk about fingerprint-readers and stuff? Imagine integrating the FPR into a hardware security token, resident entirely within the EC and TPM. I wouldn’t trust a manufacturer-proprietary integration as far as I could throw it, but an open and auditable firmware for same would be a gamechanger. Especially if I can solder-out the write-protect pin for the EC flash IC after burning it.
This truly is an exciting frontier. Unfortunately it moves very fast, and mods developed for one machine are likely to become substantially or entirely obsolete in the next model, entirely based on the manufacturer’s whim. I think that’s part of why we don’t see more hacking here already, but then, ThinkPad owners are a special breed.
You get it – this is a wonderful assortment of EC hacking examples! So many fun things can be done with EC firmware that’s modifiable and trustworthy. And.. give us an expansion header with some fun pins, too :3 I.e. an ADC channel, for instance – I want to add potentiometers to my laptop!
As for mod obsolescence, if the firmware is open, it should be way easier to port mods from generation to generation. Thinkpad stuff was complex to figure out, in part, because it relied heavily on RE and patching – otherwise, the keyboard mod described in the Thinkpad video would take minutes to perform. So, we get a lower barrier of entry, and with sources being available, we also get “apply sourcecode patch to new revision” kind of portability!
“And.. give us an expansion header with some fun pins, too :3 I.e. an ADC channel, for instance”
Do you recall the “Geekport”?
https://en.wikipedia.org/wiki/Be_Inc.
I should have included the Geekport description.
https://en.wikipedia.org/wiki/BeBox
“An experimental-electronic-development oriented port, backed by three fuses on the mainboard, the 37-pin D-Sub “GeekPort” provides Digital and analog I/O and DC power[13] on the ISA bus:
Two independent, bidirectional 8-bit ports
Four A/D pins routing to a 12-bit A/D converter
Four D/A pins connected to an independent 8-bit D/A converter
Two signal ground reference pins
Eleven power and ground pins: Two at +5 V, one at +12 V, one at -12 V, seven ground pins
”
The pinout:
http://www.hardwarebook.info/GeekPort
The modern version of the Geekport, of course, is the Raspberry Pi’s pins, either 26 or 40 of them.
(Or using USB to connect to whatever, of course. Lots of Arduinoish things, a few WS2812s, etc.)
your disc have microcontroler, so big! linux fit
I wish MSI would release their firmware. I want to chnage the charge voltage of the BQ24780S to 12.6 instead of 12.3V. I guess i could send SMBus command directly and maybe chnage the value for a while, but im pretty sure the EC chip would always chnage it back
Actually “Charge Voltage” and “Charge Current” are read out from the SBS uC in the battery by the EC. Then they are sent to the charger IC. EC also usually checks “Manufacturer Name” and/or “Device Name” register from SBS to be sure you are using correct battery. Also battery can send CV and CC values by itself(Broadcast mode), but I saw it only a few times. Also all this is correct only for TI uCs, I mainly work with their products.
So if it’s still relevant to you – try to hack seal and FA on your battery. Batteries from 2010 to 2014 are mostly use bq20zXX(has known FA manufacturer backdoor), from 2014 to 2018 – bq30zXX(Also has known backdoor), from 2018 mostly bq40zXX(bq9000~ not much info on them, some are locked down really hard, but most are using default pass). Very rough estimates, but you can always check DeviceID and compare it to the known IDs. Or poke registers untill you find out.