64 Rasberry Pis Turned Into A Supercomputer

In retrospect, it was only a matter of time before someone turned a bunch of Raspberry Pis into a supercomputer.

The Raspi supercomputer is the result of a project headed up by University of Southampton professor [Simon Cox]. Included in the team are a gaggle of grad students and [Simon]’s 6-year-old son who graciously provided the material, design, and logistics for the custom LEGO case.

The Iridris-Pi supercomputer, as the team calls their creation, consists of 64 Raspberry Pis, all configured for parallel processing using a lightweight version of MPI. [Simon] was kind enough to put up an excellent guide for turning two (or more) Raspberry Pis into a supercomputer.

The machine has a full 1 TB of disk space provided by a 16 GB SD card in each node. Although the press release doesn’t go over the computational capabilities of the Iridris-Pi, the entire system can be powered from a single 13 A supply.

If you’re wondering what it would take to get a Raspberry Pi supercomputer into the TOP500 list of supercomputers, a bit of back-of-the-envelope computation given the Raspi’s performance and the fact the 500th fastest computer can crank out about 60 TeraFLOPS/s, we’ll estimate about 1.4 Million Raspis would be needed. At least it’s a start.

71 thoughts on “64 Rasberry Pis Turned Into A Supercomputer

    1. It is a super computer, 64 raspis equalls enough juice to down 76.5 Microsoft servers with out the use of a low orbit Ion cannon node connection. Obviiusly You do not understand power, much less higher mathematics. Noobs like you should learn more before they choose to dain us with their special brand of ignorance. That being said, you can get the much power from 4 factory specked, windows 7 laptops (8-10 versions are as per microsofts par complete crap.) Linux’s processing prowess and speed would orobably need 3. Point being a true beowulf cluster is any number of factory based pc’s or Laptops that when combined together create a stronger set of variables to complete a set task. All beowulf clusters are super computers, but not all super computers are beowulf clusters. In all honesty, if I combined the motherboards from 3,250 tandy made Commodore Vic 20’s the speed of tgat creation would be on par with a small oortable mainframe from the 80’s. I could then argue that it too was a beowulf cluster. My point is be inventive and original nit ignorant and uninformed like Don. Oh and Don, before you give me lip, I should point out that I am a genius, with an autodidactic nsture, who is completely self taught in all things computer. This includes programming, building, soldering, and anything else I need to learn on the fly that has to do with computers. I have 33 years of experience in the technical side of things and promise you that I know where if I speak. Also have 12 degrees including 5 from M.I.T. GRADUSTED WITH HONORS AND I AM A PROUD MEMBER OF MENSA. Just letting you know before you try arguing. It saves you time and energy, and me the trouble

      1. 12 degrees, not one in English I presume, and not even bright enough to use grammarly, or similar. Also spending money to join mensa is not worth it. I turned down their offer to join.

  1. Some time ago i had a dream to make this happen with beagle boars. There were no raspis invented.

    I’d say this micro – super – computer is one great educational tool to teach parallel processing, at a low cost.

  2. “The team wants to see this low-cost system as a starting point to inspire and enable students to apply high-performance computing and data handling to tackle complex engineering and scientific challenges as part of our on-going outreach activities.”

    Wouldn’t an average i5/i7 desktop do just fine for this at a lower cost than *2500* GBP?

    1. The complex engineering and scientific challenges they’re talking about are the problems specific to parallel computing. Designing algorithms that work well in parallel, handling communication and dealing with communication bottlenecks, sharing loads, etc. The actual power of the system is not important.

        1. 64 virtual machines on a powerful i7 would still be limited to at most the 12-core hyperthreaded. Sure 12 cores on here are faster than 12 cores on a rasp pi, but data bandwidth is all that is important. GPUs themselves are powerful compared to CPUs for certain tasks (especially linear algebra) because the parallel computing potential is much, much greater on a GPU than even high end desktop CPUs.

    2. I completely agree. You can teach the principles of high-performance computing and data handling with any multi-core machine effectively (and more cheaply). There are plenty of available libraries that will handing IPC across an IP framework (DBUS and zmq to name a few I’ve used…) for which programming complex engineering/scientific challenges can be used.

    3. It would be even cheaper to go to a local electronic scrap yard and get a bunch of pentium III/IV/AMD K7 mainboards, buy some PCI-GBit ethernet cards and a many port gigabit switch and connect them as a Beowulf/OpenMPI cluster. => More GFlops/s for sure, cheaper, recycling of scrap electronics.

  3. The problem is the ethernet throughput: with my IGEPv2 (a Beagle Board clone) I can’t exceed 5 MB/s transfering file from it to my notebook.

    The maximum computing power is too limited by the speed of data exchange between devices

  4. I don’t understand the 13A current number. If 64 RPis can be powered by 13A, that would be 203 mA– much lower than the recommended 700mA rating. OK, 3.5 fold headroom– I can of see that (sort of). But looking at the original article, their cluster “runs off a single 13 Amp mains socket.” In which case, that number means nothing. The UK uses 220V, which at 13A is 2800W. Assuming 80% efficiency going down to 5V, means it could supply 457 A at 5V which should be enough to power a lot more than 64 RPis.

      1. Looking into this further, I don’t think the 13A refers to the power supply but rather to the outlet. Look at http://en.wikipedia.org/wiki/BS_1363#Fuses

        which describes the electrical outlet standard used in the UK. It turns out that 13A is one of the outlet current ratings.

        I agree with you that the current consumption _might_ be that low (or even lower) without using USB/HDMI but I think the summary “entire system can be powered from a single 13 A supply” part is an incorrect reading of “runs off a single 13 Amp mains socket” from the original article.

      2. Furthermore a 13a plug in the UK has a 13 amp fuse incorporated into the plug which will blow if you take more than that over time.
        By blow of course, it doesn’t pop with the first overcurrent, its a function of time, fuse type, overcurrent demanded etc. But you get the picture, if it plugs into one outlet, it’s obeying the 13amp rating because that 13a fuse in that plug is mandatory.

    1. From my observations over the last few days, I’d say ~250 mA is the standby current for an idle RasPi, with around 400 mA being normal for one under high load (e.g. compiling programs).

      The only thing connected at the time was a network cable, but the only thing it was being used for was SSH.

      Therefore, I’d say 200 mA is too low – 13 A almost definitely refers to the mains supply, and being yet another reminder that the unit they should have used is watts, esp. since with that number of nodes, it might be more efficient to have a single 3.3 V regulator for the entire cluster…

  5. Seriously? I have resisted the urge to razz you guys about diluting your content with worthless Raspberry Pi posts as it seemed you were doing better about that, but this one is bad.

    When all the comments are negative, you have a problem.

    1. I disagree with your statement. This is an interesting build and you’re upset because it has the Raspberry PI “buzzword” in it. The people who developed the Raspberry PI are trying to further the education of this field. Most of the negative comments are about shipping of personal units and not about the build.

      Is hackaday not about making or modifying your own equipment (make a supercomputer). This site has a broad audience and isn’t tailored to an esoteric group that wants to exclude others because there’s a high standard not met.

      How about elaborate on your comment and maybe people can debate about it, a passive aggressive remark doesn’t prove anything.

      My comment on the build:
      It is interesting and the experience making one is probably worth it but I don’t think of it as an alternative money/dollar value for FLOPS. I would love to get a couple RPI’s and start getting my nephew interested this field!!!

      1. So find me an R-Pi story that has a comment section that doesn’t have..
        1) A list of vaguely similar products mentioned as better, where better = not an R-Pi.
        2) Endless “But mine hasn’t come yet” posts.
        3) Some twit complaining that this is getting too much coverage.

        Reality.. The R-Pi is a good idea. If you don’t see why, that is your problem.

        And remember.. Real engineers concede there is more than one solution to any problem.

  6. If anyone remembers the FAWN cluster, there are actually real-world problems that this type of build can excell at, like hash lookup. Basically compute needs are neglegable, and what you need is memory capacity and speed per node.

  7. Please correct me wrong in terms of calculation. Does this tell me with 64 Raspberry Pi, assuming all model is based on ARM1176JZF-S which is 700MHz, does that mean in math, it is 64 RP x 700MHz = 44.8 GHz of supercomputer?

    1. Nope. Clustering won’t increase the clock-speed of the CPU’s. It increases the processing power and lowers the load. It’s like saying, a 1.5 GHz Dual-Core CPU can process more than a 2 GHz Single-Core CPU.

      1. Wait a minute R
        ” It’s like saying, a 1.5 GHz Dual-Core CPU can process more than a 2 GHz Single-Core CPU.”

        You have to be careful about making statements like this – two 1.5GHz cores working concurrently CAN do more work than a single 2GHz CPU.

        This differentiating factor is that you’ll need to be able to split the processing between two cores. A single-threaded application will perform better on a single 2GHz CPU. But if it was a two-process application, the dual core will perform much better.

        Some graphics cards have over 1000 cores so they can divide a large problem into much smaller problems which can be distributed amongst the cores.

  8. Missing the obvious bonus – it has 64 HDMI outputs. Video wall!

    The total bandwidth is a pretty significant – I’ve often toyed with the idea of using HDMI/DVID-D interfaces for inter-FPGA comms. Cables are cheap, and a some FPGA boards have two or more interfaces.

  9. oh the horror, if someone did this as a PR stunt im sure theyr getting fired or sued.

    its one thing to pull off a technical feat to get people to buy, but when some people have been waiting over 6 months for ONE unit PREordered, it makes people cancel thier orders and buy something else, something they can own without having to “pray” or “hope for” (*%*&% sakes, this isnt a date, lottery, or even a mystery.

    if you cant fucking ship them, dont fucking take orders and then deliver to someone else.

    thats called being an ass and im sure there are people canceling thier orders en-masse right now.

    and someone getting fired

    EITHER THAT OR THE PERSON THAT MADE THIS SUPER-PI ROBBED A TRUCK FULL OF EM!

  10. the 500th fastest computer can crank out about 60 TeraFLOPS/s, we’ll estimate about 1.4 Million Raspis would be needed. At least it’s a start.

    Yet again somebody used an acronym they didn’t understand. FLOPS = FLoating point OPeration per Second. Thus FLOPS/s becomes floating point operation per s^2 which is an acceleration.

  11. This is not so nice. As agreed, no need to use RPi’s for such a project, and furthermore, I still have not gotten my Pi which I ordered months ago, scheduled to be delivered around weeks ago.

    1. To all the people whining about how you still haven’t got your Pis, how about naming and shaming your suppliers?

      My first RasPi arrived months back. Recently I ordered a 2nd one, and it arrived with 6 weeks, as promised. (Both times via element14.)

      Don’t be angry at the people who already have RasPis – we’re not the problem. Be angry at the ones who have failed to deliver yours in the time promised.

  12. Quit whining. Yes you could do this with virtual machines, but nothing beats the hands on experience of physical hardware. For example, virtual machines don’t overheat but as hot as my pi gets, I’m sure 64 units can have a few issues.

    It’s a college, you know, that place where you learn stuff. This is about the cheapest way to build such a parallel system and I am sure that class is learning a lot. Great project.

  13. What’s next, 64 Casio calculators connected in one “supercomputer”… Raspberry Pi is not a high performance computer, it’s for network/multimedia. If they wanted a “supercomputer”, they could get 4 intel i7 processors and required motherboards and probably get a much higher performance and lower power consumption for much less money. I’m sorry, but this is just stupid waste of resources and time, and that would be ok if they done it with their own money and time, and not wasting public money.

    1. Sorry, I can’t agree with this project being a waste of pubic money. It effectively represented a hands-on project that included quite few concepts that had to be learned in order to complete it. Physically building the array using legos was innovative. However, knowledge of networking, switching, IP addressing, DHCP, NAT, power considerations, cooling, and programming, to name a few, would also be involved during a project such as this. For learning, 64 RPi’s might be overkill but the physical appearance and satisfaction of building it can spark the imagination and lead to greater experiences.

  14. Don’t forget about the 26 GFlops capability of the graphics unit that can be used for Gen Purpose number processing. A rough order of magnitude comparison is an i7 980X gets about 109 GFlops. So about 5 Raspis running a fully parallel benchmark could equal an i7. Assuming 100% efficient use of GPU processing would only require 2308 Raspis to make 60 TFlops. Derate that to 50% and you are still fewer processors and less heat than some of the other massively parallel processors in the top 500. Many of these are also GPU based.

  15. That’s exactly what I wanted to do. I got 2 RasPis at the moment (one as media player, one as a “server”), but I think I’ll order 6 to 8 more. I hoped the Model A would get a ethernet port, then they would be even cheaper for that “supercomputer” purpose. But they don’t have ethernet, so I’ll go for model B.

  16. Hey guys.
    Has so eine experience with this kind of cluster and truecrypt? Could i buy a second ( or third ) pi and run an reasonable fileserver ( fully encrypted HDD ) with VPN etc. or do i need some more pi` for this task?

    Thanks,
    B_S

  17. Hello,
    I’m really interested about this clustering stuff, I want to build a home data center of 10 PIs so I can run a web server accessible by the external world through the domain and the static IP address so I need your help about it, if each Pi is 700mb speed and 512 ram does it mean that the web server will act like 700 x 10 =7000mb + 512 x 10 = 5120 mb ram ? If not then how can you evaluate the server’s performance? I mean what so i get when I cluster 10 PIs or 20 or 64??

    Thank you

  18. now with new raspberry pi 2 which is 6x faster then previous one (Model A/B) and with more memory it is already a little bit closer
    what raspberry pi really needs is to connect each other faster than with 100MB LAN
    even 1GB is not enough
    for input /output messages is OK, but for communication with other rasp pi is not enought

  19. I like the idea of using the low cost Raspberry Pi. I tried, in the past, to use two VIC-20’s to share processing…but this was a pipe dream of an 8 year old. I am an avid user of the Raspberry Pi AND Arduino architectures, The idea of nodal computing is fascinating, and should not be berated. I have a dislike when it comes to people saying that they “wasted money” or “get a bigger machine.” I applaud the users in their responses, but please…remember…thinking outside of the box makes you the imaginative genius that separates you from the molded masses. Happy Friday everyone, and help enlighten those around you of what IS possible, and NOT what isn’t. ;)

  20. I was fortunate enough for my college to find me for this project. I did use the documentation for building my Raspberry Pi supercomputer but I had to face a lot of challenges and overcome them and also build an application to demonstrate it’s capabilities. I had a lot of fun building it. I have made a list of common errors that I faced while I was building it and it can help a lot of people: http://sourcedexter.com/6-common-errors-when-building-a-raspberry-pi-supercomputer/

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.