Netflix isn’t the first name to come to mind when considering security research firms, but they make heavy use of FreeBSD in their content delivery system and do security research as a result. Their first security bulletin of the year, not surprisingly, covers a FreeBSD vulnerability that happens to also affect Linux kernels from the last 10 years. This vulnerability uses SACKs and odd MSS values to crash a server kernel.
To understand Selective ACKs, we need to step back and look at how TCP connections work. TCP connections provide guaranteed delivery, implemented in the from of ACKnowledgement (ACK) packets. We think of a TCP connection as having a dedicated ACK packet for every data packet. In reality, the Operating System makes great effort to avoid sending “naked” ACK packets, and combines multiple ACKs in a single packet. An ACK is simply a flag in a packet header combined with a running total of bytes received, and can be included in a normal data packet. As much as is possible, the ACK for data received is sent along with data packets flowing in the opposite direction.
One problem with this approach is that when a transmission failure occurs, it’s not clear which packet was dropped, and multiple packets must be re-transmitted. Another strategy for handling ACKs is to use Selective ACKs, or SACKs. A SACK will include the ACK flag, the total number of bytes, as well as the TCP sequence numbers. When data is dropped, the SACK packet specifies precisely which packets were lost.
The other term important to understand is the Maximum Segment Size (MSS). This value is usually specified during the initial TCP handshake, and specifies how much data can be transmitted in a single TCP segment. A MSS set to a lower number often results in data being split into multiple segments.
Netflix outlined several problems related to SACK , but the most serious vulnerability is triggered when an attacker makes a TCP connection to a Linux or FreeBSD server, and sets the MSS to the lowest possible value. After data is transferred, the attacker sends a sequence of SACK packets, requesting the re-transfer of specific multiple packets. This specially crafted series of packets causes the multiple fragmented messages to overflow the server’s outgoing buffer. It appears this attack cannot lead to code execution, but it does cause an immediate kernel panic, which essentially knocks the target machine offline.
Patches fixing the problem have been released,
but aren’t yet available for easy install on live systems. The patches haven’t yet been part of an official kernel release, but most distributions have already backported the patches and made them available as updates. For more information, see a very helpful comment from an anonymous commenter below.
As a workaround, Netflix suggests either disabling SACK altogether, or filtering packets with very low MSS values. More information about these mitigations is available in their bulletin.
Building on the concepts of Rowhammer, Rambleed attacks the memory of other processes, but by reading that memory instead of just writing to it. Just as with Rowhammer, the central idea is that modern RAM is so dense that individual bits have a detectable effect on nearby bits. Rowhammer allowed an attacker to flip nearby bits even though they may have belonged to a different process, or even the kernel itself.
Rambleed depends on the physical layout of memory — it’s essentially a two dimensional grid. The bits above and below have an effect on the bit flips of a given bit. If an attacker can control a row of memory, a Rowhammer attack can be mounted on one of the bits of that row. By measuring how effective that attack was, the status of the bits above and below can be statistically determined.
Historically, physical RAM attacks of this nature is defeated by ECC memory. The Rambleed researchers suggest two approaches to overcome ECC. The first is to flip multiple bits so that the ECC algorithm still evaluates the pattern as correct. The second technique is a timing attack, where an error-corrected read takes measurably longer than an uncorrected read. Since the presence or absence of a flipped bit is enough to determine the target bit’s value, the ECC mechanism is defeated. As their coup de grâce, the authors demonstrated Rambleed by recovering an RSA-2048 key from an OpenSSH 7.9 server.
Have I Been Pwned… For Sale?
First off, if you haven’t already, go check out Have I Been Pwned. Give the website an email address, and it will return the list of websites that have been compromised where an account was using that email address. It’s extremely useful to keep track of where your accounts have been scraped and exposed. While some hits are benign, like your email address scraped from public Github data, you might just discover an old forum or service that leaked an important password or other data.
As useful as this service is, it’s surprising to see a virtual for sale sign show up. [Troy Hunt] has been running the site single-handedly for over 5 years. He now measures traffic by the millions, and records by the billions, and recently had the epiphany that personal burnout was looming on the horizon, unless changes were made. He’s looking for a parent organization or company to acquire HIBP, stay true to his core principles, and let him make some changes to keep the ship afloat.
An April commentary on the vulnerability seems particularly apt, given the current resurgence of the problem. [Rob VandenBrink] observed that Oracle’s resolution for the problem is simply to blacklist the specific attack vector, rather than take action to fix the underlying deserialization problem.
Firefox has released two point releases in the last week, patching two vulnerabilities that are reported to be actively used in an attack against Coinbase employees. Not all the details have been released yet, so look forward to more details next week. For now, just make sure your version of Firefox is at least 67.0.4.