When the Raspberry Pi was introduced, the world was given a very cheap, usable Linux computer. Cheap is good, and it enables one kind of project that was previously fairly expensive. This, of course, is cluster computing, and now we can imagine an Aronofsky-esque Beowulf cluster in our apartment.
This Hackaday Prize entry is for a 100-board cluster of Raspberry Pis running Hadoop. Has something like this been done before? Most certainly. The trick is getting it right, being able to physically scale the cluster, and putting the right software on it.
The Raspberry Pi doesn’t have connectors in all the right places. The Ethernet and USB is on one side, power input is on another, and god help you if you need a direct serial connection to a Pi in the middle of a stack. This is the physical problem of putting a cluster of Pis together. If you’re exceptionally clever and are using Pi Zeros, you’ll come up with something like this, but for normal Pis, you’ll need an enclosure, a beefy, efficient power supply, and a mess of network switches.
For the software, the team behind this box of Raspberries is turning to Hadoop. Yahoo recently built a Hadoop cluster with 32,000 nodes used for deep learning and other very computationally intensive tasks. This much smaller cluster won’t be used for very demanding work. Instead, this cluster will be used for education, training, and training those ever important STEAM students. It’s big data in a small package, and a great project for the Hackaday Prize.
Raspberry Pi clusters are a dime a dozen these days. Well, maybe more like £250 for a five-Pi cluster. Anyway, this project is a bit different. It’s exquisitely documented.
[Nick Smith] built a 5-node Pi 3 cluster from scratch, laser-cutting his own acrylic case and tearing down a small network switch to include in the design. It is, he happily admits, a solution looking for a problem. [Smith] did an excellent job of documenting how he designed the case in CAD, prototyped it in wood, and how he put the final cluster together with eye-catching clear acrylic.
Of interest is that he even built his own clips to hold the sides of the case together and offers all of the files for anyone who wants to build their own. Head over to his page for the complete bill of materials (we didn’t know Pis were something you could order in 5-packs). And please, next time you work on a project follow [Nick’s] example of how to document it well, and how to show what did (and didn’t) work.
If 5 nodes just doesn’t do it for you, we suggest this 120-node screen-equipped monster, and another clear-acrylic masterpiece housing 40 Pis. This stuff really isn’t only for fun and games. Although it wasn’t Pi-based, here’s a talk at Hackaday Belgrade about an ARM-based SBC cluster built to crunch numbers for university researchers.
Kristina Kapanova is a PhD student at the Bulgarian Academy of Sciences. Her research is taking her to simulations of quantum effects in semiconductor devices, but this field of study requires a supercomputer for billions of calculations. The college had a proper supercomputer, and was getting a new one, but for a while, Kristina and her fellow ramen-eating colleagues were without a big box of computing. To solve this problem, Kristina built her own supercomputer from off-the-shelf ARM boards.
Continue reading “Designing a High Performance Parallel Personal Cluster”
[James J. Guthrie] just published a rather formal announcement that his 4-node Raspberry Pi cluster greatly outperforms a 64-node version. Of course the differentiating factor is the version of the hardware. [James] is using the Raspberry Pi 2 while the larger version used the Model B.
We covered that original build almost three years ago. It’s a cluster called the Iridris Pi supercomputer. The difference is a 700 MHz single core versus the 900 Mhz quad-core with double-the ram. This let [James] benchmark his four-node-wonder at 3.048 gigaflops. You’re a bit fuzzy about what a gigaflops is exactly? So were we… it’s a billion floating point operations per second… which doesn’t matter to your human brain. It’s a ruler with which you can take one type of measurement. This is triple the performance at 1/16th the number of nodes. The cost difference is staggering with the Iridris ringing in at around £2500 and the light-weight 4-node built at just £120. That’s more than an order of magnitude.
Look, there’s nothing fancy to see in [James’] project announcement. Yet. But it seems somewhat monumental to stand back and think that a $35 computer aimed at education is being used to build clusters for crunching Ph.D. level research projects.
Multi-node RasPi clusters seem to be a rite of passage these days for hackers working with distributed computing. [Dave’s] 40-node cluster is the latest of the super-Pi creations, and while it’s not the biggest we’ve featured here, it may be the sleekest.
The goal of this project—aside from the obvious desire to test distributed software—was to keep the entire package below the size of a full tower desktop. [Dave’s] design packs the Pi’s in groups of 4 across ten individual cards that easily slide out for access. Each is wired (through beautiful cable management, we must say) to one of the 2 24-port switches at the bottom of the case. The build uses an ATX power supply up top that feeds into individual power for the Pi’s and everything else, including his HD array—5 1TB HD’s, expandable to 12—a wireless router, and a hefty fan assembly.
Perhaps the greatest achievement is the custom acrylic case, which [Dave] lasered out at the Dallas Makerspace (we featured it here last month). Each panel slides off with the press of a button, and the front/back panels provide convenient access to the internal network via some jacks. If you’ve ever been remotely curious about a build like this one, you should cruise over to [Dave’s] page immediately: it’s one of the most meticulously well-documented projects we’ve seen in a long time. Videos after the break.
Continue reading “40-Node Raspi Cluster”
Not only did [Josh Kiepert] build a 33 Node Beowulf Cluster, but he made sure it looks impressive even if you don’t know what it is. That’s thanks to the power distribution PCBs he designed and etched. In addition to injecting power through each of the RPi GPIO headers they host an RGB LED which is illuminated in blue in the images above.
Quite some time ago we saw a 64-node RPi cluster. That one used LEGO pieces as a rack system to hold all of the boards. But [Josh] used stand-offs to create the columns of hardware which are suspended between top and bottom plates made out of acrylic. The only thing that’s unique about each board is the SD card and that’s why each has a label on it that identifies the node. These have been flashed with almost identical images; the host name and IP address are the only thing that changes from one to the next. They’ve been put in order physically so that you can quickly find your way through the rack. But functionally this doesn’t matter… put the card in any RPi and it will automatically identify itself on the network no matter where it’s located in the rack.
Don’t miss the demo video where [Josh] explains the entire setup.
Continue reading “33 Node Beowulf Cluster built with Raspberry Pi”
This setup helps to represent data in a meaningful way to for visually impaired people. It uses a combination of physical objects to represent data clusters, and audio feedback when manipulating those objects. In the video after the break you’ll see that the cubes can orient themselves to represent data clusters. The table top acts as a graphing field, with a textured border as a reference for the user. A camera mounted below the clear surface allows image processing software to calculate the locations for the cubes. Each cube is motorized and contains an Arduino and ZigBee module, listening for positioning information from the computer that is doing the video processing. Once in position, the user can move the cubes, with modulated noise as a measure of how near they are to the heart of each data cluster.
The team plans to conduct further study on the usefulness of this interactive data object. We certainly see potential for hacking as this uses off-the-shelf components that are both inexpensive, and easy to find. It certainly reminds us of a multitouch display with added physical tokens.
Continue reading “Data plotting for the visually impaired”