The extended filesystem, otherwise known as ext, has been a fundamental part of Linux since before the 1.0 release in 1994. Currently the filesystem is on its fourth major revision, in use since its release in 2008 thanks to its stability, reliability, and backwards compatibility with the other ext filesystem versions. But with that much history there are bound to be a few issues cropping up here and there. [Will] recently found an exploit with this filesystem that can cause a Linux kernel to immediately panic when a manipulated USB drive is inserted into a computer.
[Will] discovered this quirk when investigating the intricacies of the filesystem for problems and other vulnerabilities. A tool called tune2fs, used for administering and modifying ext filesystems, includes the ability to pass certain commands to the Linux kernel when certain situations arise with the filesystem itself, including that the kernel should panic. One situation is that the ext filesystem itself becomes corrupted, which can then cause the kernel panic. Armed with this knowledge, a USB drive can be purposefully given a corrupted ext filesystem which, when plugged into a Linux machine, can cause the computer to shut down.
The post linked above goes into some discussion about how this exploit could be used maliciously to gain access to a Linux system, including rebooting computers where no access to a power button is otherwise enabled or making other changes to the system before needing a reboot to apply the changes. In general, though, it’s good to assume an attacker could take any route to gain access to a machine. This exploit from a few years ago, for example, allowed another Linux tool to be used to gain root access.
Thanks to [Timothy] for the tip!
To clarify, since it’s not clearly stated: is this a vulnerability with any system that uses ext, or do you need to have that utility installed and running in order to force a kernel panic?
Seems to be that it’s tune2fs that’s causing the kernel panic when it sees a malformed ext partition? If so it’s not really fair to call this an issue with ext itself, and might not be very usefull on a computer you don’t have access to (if you can go install utilities on it…. presumably you can shut it down??)
It’s a flag on the filesystem itself. So you can make a flash drive that forces any Linux system to kernel panic the moment it tries to mount the FS (which most desktop ones do nowadays). It’s of very limited use, as there are way nastier things you can do if you have physical access to an USB port, but still something to keep an eye on.
I mean, if you go around sticking random USB drives into your devices and have them setup to auto mount you’re kind of an idiot just waiting to get hacked anyways.
Nice find though.
I’m not sure that mounting a filesystem should be considered risky. If the ext driver tried to mount a completely corrupt drive and it panicked rather than “nope”-ing, that’d be a bug, right? So how is this any different?
ask any filesystem developer, and they’ll tell you you’re out of your mind. it is generally considered not possible to harden any filesystem driver against all the ways a malicious actor could intentionally craft a malicious corrupted filesystem. filesystems are simply too big and complex to exhaustively verify like you would with other untrusted input. especially with any amount of speed.
generally, people will instead tell you to at least mount the filesystem with the driver running in userspace using FUSE, though I don’t think anyone has done an exhaustive security analysis of this.
I don’t understand what you’re saying. Filesystems inherently will accrue errors – it’ll happen, they’re using storage devices which fail.
So if it’s true that the drivers should be considered toxic to failed data, why wouldn’t “panic on error” be the default option? Otherwise if a filesystem error occurs which will torch the driver (say, via an out of bounds error) it could end up destroying the rest of the system even without a panic.
I understand the idea of “you can’t trust filesystem data to not mess something up” or “you can’t trust a filesystem to not have maliciously hidden data” but “you can’t trust the filesystem driver to not explode the system” seems a stretch.
How is this even a “find”? It’s literally in the documentation.
Oh wait, I just came up with a new “exploit” that involves putting a copy of bash on a thumbdrive, flagged suit root. Bam, instant root level access anywhere someone has both a local account and a complete lack of sense.
It’s definitely of limited use but dear lord, that flag is crazy. That’s not something the filesystem should be able to specify, it’s something that mount should be able to specify so it can be a device-specific thing.
I can think of cases where you would want an FS error to panic (not good ones, but some), but I can’t think of one where it should follow the filesystem versus the mount.
Tune2fs is used to configure parameters within the filesystem itself, in this case “what to do if this filesystem is corrupted”. So on my system, I set that flag using tune2fs, and then intentionally corrupt the filesystem. Whenever I connect that drive to another system, it’ll panic as part of the builtin kernel ext driver, no tune2fs needed.
If someone has the ability to plug in a flash drive, they probably also have the ability to yank the power cord.
Yanking the power cord empties RAM, while crashing the OS doesn’t. If you can shut the operating system down and then manage to reboot into something else, you may be able to recover things like encryption keys or logins.
Removing power won’t empty the ram. There have been previous exploits where disk encryption keys have been recovered from ram after power was removed.
Yeah, but opening the box and applying liquid nitrogen (IIRC they freeze the chips?) might be a teeny bit more obvious than this.
Recovering data from RAM after power has been removed generally involves freezing the RAM, something that isn’t all that easy to do except in a controlled environment.
Unless you are doing something physical to prevent the RAM from losing data when unpowered then what they said is true, removing the power empties the RAM.
More to the point, if you can lock up the system by crashing it, not only RAM but CPU registers and cache will remain untouched until power is removed or the CPU is reset. Suppose a security agency does this to your laptop: now they have infinite time to carry it away and keep it powered up until they can connect it to some sort of debugger to read the various memory contents.
Physical access generally means you have lost. What you need is a lock on the door. Now i realize that there are a lot of laptops running around these days where physical access is a big deal, but turning off auto mount completely nullifies this attack. If you are in a threat model where this would be an issue you aren’t just going to be mounting random usb drives.
USB ports don’t have to be physically near the device at all.
I get the “automounting USB drives is bad” idea but I’m not super-sure I agree. USB is pure command/response, so it should be able to be made safe (it isn’t, but it could be) and mounting shouldn’t really be risky if you control the mount point.
What are you on about? USB ports are physical ports and they are physically connected to the device and computer, so they need to be physically near both. Yes you could run a long cable but it is still physically connected and near.
“What are you on about? USB ports are physical ports”
https://www.kernel.org/doc/html/latest/usb/usbip_protocol.html
Then trust those of us with decades of experience when we say it’s bad, it’s always been bad, and it will likely always remain bad for the forseeable future.
“it’s bad, it’s always been bad, and it will likely always remain bad for the forseeable future.”
Okay, so assume that filesystem images are always untrustworthy.
Why the hell would you attach the “panic on error” flag to the filesystem which you’ve explicitly defined as untrustworthy?
My PC speakers are powered via USB. Doing that over a network would be quite a challenge.
I think there’s there’s a confusing of USB, USB device, USB port, and USBIP going on here.
Why? The remote host provides the power.
There are USB hubs you can buy that you connect to with USB/IP over a network. They act just like a local port. You plug this thing into it, if your host automounts, panic.
This works on Win7 systems with certain Linux filesystems the OS crashes. They did not plan on patching this when I discovered it. Is there a patch? I don’t know.
I wonder if this only happens on systems configured to auto-mount USB drives. That is a bit scary in itself, as I’m not so sure whether all filesystem drivers supported by Linux are sufficiently secure against malicious disk modifications.
None of them are, filesystem are not designed nor tested against this kind of threats. The USB stack is not safe against malicious hardware either, so really once someone has physical access to a machine security is a lost game. Happily we at least have a syzbot running fuzzing for various parts of the Linux kernel, but it’s far from optimal.
hmm, I’ve never had any of my linux systems auto mount a usb key – and don’t intend to ever have that. And in windows I have autorun etc turned off..
Anyone using an unknown usb key and auto doing something with it is living dangerously..
As a kernel developer who wrote an ext2 driver, this is not an exploit. This is intentional EXT behavior and has been for all versions (1-4). It might not be good design choices now, but it isn’t an “exploit”
It’s more an exploit against the crappy desktop environments that default to automounting everything that gets plugged in. Which I think is roughly all of them.
The general assumption seems to be that the user is too stupid to know what mounting a filesystem is, too lazy to bother, and/or lives such a simple life that they always, invariably, without exception, want to mount any drive they insert.
KDE doesn’t automatically mount things. It pops up an notice that there’s a new device, which you can then mount (or not.)
Isn’t this most people? Why else would you plug something in to your computer?
… to dd an image to it?
” It might not be good design choices now”
Might not be? I would love to know a use case for this. Sure, turning on “panic on FS error” could have a use case but what the heck is the use case for that following the filesystem?
In every case I can think of (maybe it’s a really unreliable FS and it’s important) you would want it to follow the device, not the FS.
Well, when EXT was designed (IIRC) mass storage devices that auto mount weren’t taken into consideration. If your hard drive has an error you don’t want the kernel to keep booting, so panicking makes the most sense. But nowadays, a design revision is required.
Yeah, but that shouldn’t follow the filesystem, it should follow the device. If I dd the drive to a file and try to mount it loopback on another machine to try to figure out what’s going on and that machine panics, that’s crazy.
This value is ignored if the mount options specify what to do about it, but I fail to see how it following a “device” would be a substantially different outcome. Anyway, presuming everyone else’s use case is one of those things Linux does not do for good reason.
Like, if you want an effective chainsaw, it pretty much has to be capable of removing stray limbs of other kinds. You just don’t let children play with them.
“Anyway, presuming everyone else’s use case is one of those things Linux does not do for good reason.”
This is presuming everyone else’s use case. The only options you have are “use the filesystem spec’d option” or “use my option,” as opposed to “use the filesystem spec’d option unless it’s this, in which case do this.”
Yes, sure, you could wrap mounts around checking if it’s ext, checking if it’s got that flag set, and refusing to mount if it does. It’s absolutely a userspace solvable problem. But mount (the program) isn’t the only one that does mount (the system call), so it’s much harder to provide this option in userspace than in the kernel.
The ext driver already does sanity checking on the filesystem at mount time for security purposes: it seems totally sane to have a “refuse to mount a panic on error filesystem” parameter.
Might be useful to pry open some smart-TVs. Does this work on Android?
Calling a built in diagnostic flag an exploit is a bit much, but this is an argument for user-space execution of filesystem drivers and allowing them to crash independently of the kernel or other processes. I don’t really see this as an issue, though a reasonable fix would be requiring the flag to be “active” in order to function, or allowing the drive to be built without it.
User-space filesystems are handy for custom filesystems and experimental stuff, but ext is super-production and really should be bulletproof: if ext panics, it should be because something went horribly wrong.
I mean, if the ext driver panicked due to reading a malformed header it’s obviously a bug and should be fixed. It’s not a problem to have it available but it really shouldn’t be enabled in the driver (in the driver, not the FS, obviously it’s not enabled by default in the fs).
I agree calling it an exploit’s an exaggeration, but it’s seriously hard to see how this makes any sense.
“Physical access generally means you have lost.” Amen. Do whatever you want with the machine at that point, even removing the machine to leisurely go after the data you are looking for, or pull parts out of the machine… Or if you are in stealth mode, the idea is let your usb access the system without anyone knowing… Crashing the system with a panic isn’t stealthy. All kinds of bells may go off if system is down. Nothing to see here (in my context).
if someone can plug in usb drive in your computer it is game over anyway. because if they want to turn it off they can press the power button. or reset. or kick it. or pee in it. no need to plug in this usb stick.
“No access to a power button”
If you’re close enough to insert a USB you’re close enough to unplug the machine or boot from a (good) external drive.
“No access to a power button”
If you’re close enough to insert a USB you’re close enough to unplug the machine or boot from a (good) external drive.
“If you’re close enough to insert a USB”
You can run USB over a network, the actual ports for a machine don’t need to be near it.
If you want to argue against this being a concern (which I’d agree with) it’d be “being able to plug USB devices in is always dangerous.”
I appreciate the correction but help me out if you will. How does “You can run USB [Universal Serial Bus; Universal Serial Buss is another matter] over a network, the actual ports for a machine don’t need to be near it.”
Do you mean “ports” as in 123.123.123.123:8081 or the physical female jack (which it is) on the physical machine. How does it work? How do you get a remote machine you don’t have access to (otherwise why bother?) to accept your “mount”?
” [Will] recently found an exploit with this filesystem that can cause a Linux kernel to immediately panic when a manipulated USB drive is inserted into a computer.” A “drive” being “inserted” sounds like being in the same room as the machine to me. If you can MOUNT a USB drive to a computer you shouldn’t have access to that’s a separate problem.
You can easily imagine a system where they’ve got remote USB ports (like those Digi devices) set up like a kiosk. No physical danger (except to the Digi devices), so you figure sure, we can just mount a FS and examine it, it can’t do anything to us. Especially because the touchy parts of USB are abstracted away.
Could be even dumber where you just dump the device image and try to loopback mount it. It’d still do the same thing. Mounting shouldn’t really be dangerous if you control the mount point.
I appreciate the correction but help me out if you will. How does “You can run USB [Universal Serial Bus; Universal Serial Buss is another matter] over a network, the actual ports for a machine don’t need to be near it.”
Do you mean “ports” as in 123.123.123.123:8081 or the physical female jack (which it is) on the physical machine. How does it work? How do you get a remote machine you don’t have access to (otherwise why bother?) to accept your “mount”?
” [Will] recently found an exploit with this filesystem that can cause a Linux kernel to immediately panic when a manipulated USB drive is inserted into a computer.” A “drive” being “inserted” sounds like being in the same room as the machine to me. If you can MOUNT a USB drive to a computer you shouldn’t have access to that’s a separate problem.
Didn’t make double posts. That’s on you Hackaday.
“If you’re close enough to insert a USB..”
Hi. It’s ‘If you’re close enough to insert an USB..’
If there’s an a/e/i/o/u sound, the “a” becomes “an”.
“If you’re close enough to insert a USB..”
Hi. It’s ‘If you’re close enough to insert an USB..’
If there’s an a/e/i/o/u sound, the “a” becomes “an”.
No, you’re just flat-out wrong. Normally I try to be gentler and the nature of what is wrong with you is none of my business but I’m guessing the good ship Popularity has sailed. Since it’s implicit that we’re talking about English usage what exactly is wrong with you?
“An Yoo Ess Bee” That’s just effing stupid and I think it should stop effing him/her/it/them. “a/e/i/o/u sound”? Where is “a/e/i/o/u sound”? Show me “a/e/i/o/u sound”! You are very bad man. [Wags finger like Babu Bhatt]
Is this one of those “an ‘istorical occasion” things? 😂
Uh, does this actually have anything to do with USB? Any kind of “device” should work, including unprivileged-user generated files. Methinks that if you, as a sys admin, have enabled any kind of user mount privileges to user-modifiable files–FUSE, NFS, user namespaces, even local files (-o loop). Oh, and from the man page:
Since util-linux 2.35, mount does not exit when user permissions are
inadequate according to libmount’s internal security rules. Instead, it
drops suid permissions and continues as regular non-root user. This
behavior supports use-cases where root permissions are not necessary
(e.g., fuse filesystems, user namespaces, etc).
FUSE filesystems do not use the in-kernel FS drivers and can not cause the system to panic (absent a separate vuln).
If they have unrestricted access to the box, it’s game over anyway.
Ext2 was an adventure. Still have nightmares about missing i-nodes..
I’m so glad for NTFS. It’s old as time itself, but has Journaling enabled since forever.
Same can’t be said about Mac OS X’ HFS+. Up until 10.3 or so, Journaling wasn’t available.
Ext3 has been around since 2001. I believe NT came to consumer systems with XP, released in… 2001.
(Sure, NT had it in the 90s; go get a UPS.)
Oh no, an attacker with physical access can crash the box!
… Wait, they could also unplug the power cord.
🤦♂️