25 GPUs Brute Force 348 Billion Hashes Per Second To Crack Your Passwords

It’s our understanding that the video game industry has long been a driving force in new and better graphics processing hardware. But they’re not the only benefactors to these advances. As we’ve heard before, a graphics processing unit is uniquely qualified to process encryption hashes quickly (we’ve seen this with bitcoin mining). This project strings together 25 GPU cards in 5 servers to form a super fast brute force attack. It’s so fast that the actual specs are beyond our comprehension. How can one understand 348 billion hashes per second?

The testing was used on a collection of password hashes using LM and NTLM protocols. The NTLM is a bit stronger and fared better than the LM, but that’s not actually saying much. An eight character NTLM password will fall in 5.5 hours, while a 14 character LM hash makes it only about six minutes before the solution is discovered. Of course this type of hardware is only good if you have a copy of the password hashes themselves. Login protocols will lock out after a certain number of attempts and have measures in place to slow down automated systems like this one.

[via Boing Boing]

66 thoughts on “25 GPUs Brute Force 348 Billion Hashes Per Second To Crack Your Passwords

        1. You can’t just plug the hash rate of cracking passwords into bitcoins, they’re different operations. And then there’s the current difficulty factor you need to worry about.

          Looking at the specs of the cards, and comparing them to the Bitcoin wiki’s hardware list, it looks like the guy crunches at a rate of about 11 Ghash/sec, not too shabby, but still only about 1.77 BTC/day ($23.96 USD at mt.Gox’s current exchange rate) at the current difficulty.

          If he’s using 7kW of electricity, like he claims in the slides, and a reasonable rate for electricity in Oslow (~$0.11 USD/kWh), he’d earn about $5.48 USD per day.

          Yeah, we passed the timeframe where mining for bitcoins can actually earn you anything substantial…

          1. This is true, especially with the drop to the 25BTC per block reward sometime in novemeber (I want to say 28th).

            2 thousand (untaxed) dollars a year isn’t too shabby…

          2. But how much of that 2 thousand untaxed dollars did the hardware cost? And don’t forget that as difficulty increases rapidly this hardware will need to be replaced.

          3. If we all knew those “little 1.7BTC” now would represent $2000 dollars a day =/…
            I have 3 Arduinos… could I make a time machine with them? What do you say guys? haha

  1. Sure, using a lot of hardware to compute hashes is neat and all, but I want to find that motherboard that has 8 PCI-Express x16 slots. (Not that I particularly have a use for it, but I get excited over such expandable hardware options.)

    1. According to this post I read a while back if you are looking just to crack passwords (low throughput) then you can use PCI 1x on a 16x card just fine (i built one similar it works). It will throttle the throughput back obviously though. This would mean that you could get a less exotic mobo for a project like this.

      1. Might not throttle the throughput at all, or by much at least. There’s a lot of RAM on board those cards. Cracking involves permuting the same data (ie the dictionary) over and over, CPU-bound work. The PCI-E bus would only need to load data in when the card’s finished permuting what it has.

        Obviously this is not the case for graphics, where so much texture data needs to speed through the card from the system RAM.

      1. Useful if you have the hashed passwords. If you can get those then the security’s not up to scratch anyway.

        Still there’s encryption in use now that can withstand being permuted like this by a computer the size of the Universe for longer than the expected lifetime of the Universe. So somebody spent a lot of money on something that might’ve been a useful tool in the 1970s.

  2. There is an error in this article. You mention login protocols having lockouts to prevent setups like this, but that is not true. This type is setup is meant to crack passwords OFFLINE, which render online protocols useless. This entire setup depends on you already having the hashes, which in today’s age is actually easier to get than most people believe.

    1. I think the goal was to convey that this technique does not directly endanger anyone’s online security since, as you said, this is done with a local copy of the hash and not brute forcing the login of an actual online service/protocol.

      Though as usual, the HaD writers didn’t quite get the point across.

    2. Companies and large websites treat compromised usernames and hashed passwords as no big deal. Like you say do all the brute force offline and only require a single login attempt.

      1. Yeah but those companies are st00pid! Not that it seems to bother the world. I suppose as long as your credit card company pays up for any fraud, it doesn’t need to bother ordinary people.

    1. Unix Ninja – I’ve seen yours before on the hashcat irc channel but what this guy has done is go down the route of using VCL. Hopefully atom will look to make hashcat more distributed so that i can use the magnitudes of machines i have at home: Desktop, 2x Work Laptops, 2x Personal Laptops, Mac Mini, and so on….

  3. To be fair, NTLM can hardly be considered a cryptographic hash. It’s full of flaws, and can be broken very easily. Not to mention the complete lack of salting, which allows a simple rainbow table to crack it in minutes if not seconds. Why people continue to use it, I’ll never know.

    A real benchmark would be crunching SHA-256 or 512 hashes.

    1. So what are these flaws in NTLM that allows them to be broken very easily? Still needs brute force? Then it is cryptographically perfect. Oh and salting cannot be done on just one hash, it is a mechanism of an authentication suite, not the hashing mechanism.

      Seems that ignorance is not bliss after all.

    1. lets say you know the method is md5.. the hash you got is called “x”.. you can do every combination you can from a b .. aa .. check the hash from those with X if its the same bingo.. thats why md5 is “insecure” .. look up rainbowtables there are a limited number of “hashes”. So 2 different passwords can have the same hash.

    2. To a large degree it doesn’t matter if you guess the “correct” password. The comparison of passwords will be hashed password to hashed password (only the hashed password is stored against). So if “asd” and “fgh” both hash to the same thing, either will work as your login password.

      Now, if you want to determine the actual plaintext password to try on other sites with different hash algorithms or different salt values, then the best reverse hashing can get you is a list of possibilities.

      1. “the best reverse hashing can get you is a list of possibilities” Yeah, one of which is based on a dictionary word with a number stuck on it.

        Most cracking isn’t done in pure brute force (1,2,3…a,b,c…aa, bb, cc…a1,a2,a3 and so on) Most cracking uses large dictionaries plus rule sets. Maybe the entire dictionary plus case permutation plus L337 $p3@k substitutions plus attempt 1-99 at the end of the password. After cracking enough passwords you’ll find that most users pick predictable types of words based on the terrible restrictions we enforce. Things like ‘Password1’ or ‘December12’ or ‘December12!’. If we make rules to exploit this, we can be hugely more efficient at cracking most passwords than if we attempt pure brute force.

  4. mosheen:My simplified understanding is something like this:
    HASH=abc292abcf493cc1192ff and you need the password…you just start guessing what password produces that hash…

    cat=acccff23c292abcf493cc119 (MISS!)
    … (MISS!)
    … (MISS!)
    … (MISS!)
    dog=abc292abcf493cc1192ff (MATCH!)

    so ‘dog’ gave you the correct hash, that means dog is the clear-text password of hash abc292abcf493cc1192ff.

    Someone correct me if im wrong :)

    1. Even if dog in this case were not the same text that created the original hash it could still be used as the password. Nearly any software is going to be hashing the input then comparing the hashed results. Proper salting would pretty much eliminate that very rare incidence though.

    2. In a nutshell. Once you know what generates that hash, any other combination that produces that same hash is moot (a collision). I’m not familiar with NTML, but the problem exists across all hashes. It’s just a matter of mathematics, probabilities and available computing power. If you want to get an idea of what happens and how often a collision can occur play around a bit CRC16 or CRC32’s. Create a simple program or script to generate CRC’s until you find a collision.

      As hardware becomes cheaper and more powerful, we’ll continue to see cryptos fall down only to be replace my more powerful methods and so on and so forth.

      1. CRCs are checksums, not cryptographic hashes. Further, we are not attempting to break any hash algorithms, nor are we even trying to attack the algorithms themselves. Single block collisions are very rare for even the weakest cryptographic hash algorithms. For example, only one single block MD5 collision has been found to date, and no single block collisions have been found in any of the SHA family of algorithms. So for all intents and purposes, we are not finding any collisions — the passwords that we find are most certainly the actual password.

  5. What more that cracking passwords could that type of setup be used for? For example could it be used to produce much better file compression by finding some shorter mathematical representation for the longer binary file string?

    1. it doesnt show that windows hash are insecure it shows how fast bruteforcing a hash can be if you have enough power behind you. the only way to slow this kind of attack down is to use a hashing algo that is slow to hash. salting your hash isnt going to slow it down at all, if the attacker has access to your database that stores your password hashs he also has access to the salt for the hash so its just a matter of putting the salt with the password before hashing and checking against that, the only thing a salt will stop is rainbow tables it has nothing to do with stoping a bruteforce attack against the actual hash.

  6. That is a real purdy computer and all.. but NTLM? He might as well use it for benchmarking with Wolfenstein 3D! Let me know when he breaks a simple 20 character AES256 password…remotely and w/o backend access.

    1. Not at all what this rig is designed for. Also, where’s your AES256 encryption key stored at? Because if it’s on the sever you’re storing the password on then good work, you’ve just implemented encryption the wrong way.

      If it’s not on the same system then it’d sure be funny watching you try to log in.

  7. Hi, I’m Jeremi Gosney, the subject of this article. I see a lot of misunderstandings in the comments, and I have selected a few for response:

    > “Sure, using a lot of hardware to compute hashes is neat and all, but I want to find that motherboard that has 8 PCI-Express x16 slots.”

    The one large rig is a TYAN FT77B7015 barebones, the other systems in the cluster are primarily Chenbro RM41300-FS81 + Gigabyte GA-990FXA-UD7.

    > “There are lots of bitcoin fpga mining farms 10 – 100 times more powerful than this..”

    FPGAs are great for brute forcing a single raw cryptographic hash — which is why they’re great for Bitcoin mining. For something as dynamic and flexible as password cracking, FPGAs are less than optimal. FPGAs do not provide the flexibility needed to support multi-hash, multi-algorithm, and multi-attack modes. The complexity of password cracking demands something in the middle between CPU and FPGA, and GPUs are by far the sweet spot.

    > “Useful if you have the hashed passwords. If you can get those then the security’s not up to scratch anyway.”

    You say that as if it’s hard to obtain hashes. It is not uncommon to exploit vulnerabilities which only grant some level of read permission, without the ability to obtain a shell or escalate privileges. KoreLogic estimates over 467 million password hashes have been leaked in the past year, not including stuff on pastebin. Cruise on over to InsidePro’s forums sometime.

    There is also the legitimate side of hash cracking as well, such as penetration testing and domain auditing.

    > “How big a room can one of these heat?”

    I don’t know, it’s not exactly sitting in my house. It is in a proper datacenter.

    > “To be fair, NTLM can hardly be considered a cryptographic hash. It’s full of flaws, and can be broken very easily. Not to mention the complete lack of salting, which allows a simple rainbow table to crack it in minutes if not seconds. Why people continue to use it, I’ll never know. A real benchmark would be crunching SHA-256 or 512 hashes.”

    – AND –

    > “Bit of a waste for cracking NTLM. Just download the rainbow table and can do the same thing with much less processing power.”

    Well no, NTLM cannot be considered a cryptographic hash, because isn’t a cryptographic hash. The NT Hash algorithm employs MD4, which is a cryptographic hash. MD4 is in fact a broken cryptographic hash; however, its cryptographic strength has nothing to do with why it is a poor choice for hashing passwords.

    Rainbow tables are great if you only have a couple hashes, and are sure the password is less than 8 chars. When you have an entire AD dump of 60k+ hashes however, rainbow tables are way too slow, even if you are using GPU tables and SSDs. And of course the fixed length of rainbow tables is also problematic. We have the flexibility to crack passwords of various lengths and run multiple attack modes.

    Anyway, I’m not sure why the media chose to focus on LM/NTLM. We support all 45+ algorithms supported by oclHashcat-plus and oclHashcat-lite. We have posted full benchmarks for -lite on the Hashcat forums. We will be posting benchmarks for -plus soon as well. The media is also focused on brute force times, and the cluster supports far more than just brute force. Indeed the cluster supports everything that Hashcat supports.

    > “Even microsoft admits NTLM is weak and should not be used”

    You are confusing NTLM with NetNTLM, NetNTLMv2, NTLM Challenge/Response, etc. When we say “NTLM hashes,” we are referring specifically to passwords stored using the NT Hash algorithm, which is how all Windows passwords — both local and AD domain — are stored at rest. There is no alternative to using NT Hash on Windows systems, including Windows 8 and Server 2012. This has nothing to do with NetNTLM.

    > “Ditto. Cool setup but not nearly fast enough to crack anything like SHA-256.”

    This makes my brain hurt, and I’m not sure what makes you think we cannot crack SHA-2 hashes with this cluster. We can crack SHA-256 at a rate of 21.4 G/s, and SHA-512 at a rate of 2.2 G/s. We can also crack SHA-3 at a rate of 2.2 G/s. If it is still not obvious to you, those are insanely fast speeds.

    Please also understand that while SHA2 is a bit slower than some of the other crypto hash functions, it is still not sufficient for password storage.

    > “All this shows is how insecure the Windows hashes are and how a proper hash (such as SHA2 with a salt) wouldn’t have fallen so easily.”

    No, it would have fallen just as easily. We can crack salted SHA2 at a rate of 21.4 G/s. While that is far from the 340+ G/s we can achieve with MD4, that’s still ridiculously fast. Again, not sure why the media focused on LM/NTLM, but we do much more than that at highly accelerated rates.

    I am also concerned that you feel salted SHA2 is a proper means of storing passwords. Let me set the record straight: No cryptographic hash algorithm — salted or unsalted — is sufficient for password storage. It does not matter if you are using MD4, MD5, SHA1, SHA2, or SHA3 — they are all equally bad. You must only store passwords using an algorithm specifically designed for password storage, such as scrypt, pbkdf2, or any modern crypt(3) algorithm including bcrypt and sha512crypt.

    > “That is a real purdy computer and all.. but NTLM? He might as well use it for benchmarking with Wolfenstein 3D! Let me know when he breaks a simple 20 character AES256 password…remotely and w/o backend access.”

    You seem to lack a fundamental understanding of the differences between encryption and hashing. We deal with hashes, not encryption. Passwords are (typically) not stored using reverisible encryption, but rather with hash algorithms which are one-way (non-reversible.) We are not breaking any encryption with this cluster.

    And again, we’ve benchmarked (and also actively crack) much more than just the two algorithms the media is focusing on.

    Hope this makes things more clear for all. Thanks for your interest!

Leave a Reply to mosheenCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.