cluster

44 Articles

Hackaday Links: September 25, 2016

September 25, 2016 by 21 Comments

So you like watching stupid stuff? Here you go, a scene from Bones that tops the infamous ‘IP backtrace with Visual Basic’ or ‘four-handed keyboard’ scenes from other TV shows. Someone hacked the bones by embedding malware in a calcium fractal pattern. Also, when she uses the fire extinguisher, she doesn’t spray the base of the fire.

Raspberry Pi! You have no idea how good the term Raspberry Pi is for SEO. Even better is Raspberry Pi clusters, preferably made with Raspberry Pi Zeros. Here’s a Raspberry Pi hat for four Raspberry Pi Zeros, turning five Raspberry Pis into a complete cluster computer. To be honest and fair, if you’re looking to experiment with clusters, this probably isn’t a bad idea. The ‘cluster backplane’ is just a $2 USB hub chip, and a few MOSFETs for turning the individual Pis on and off. The Zeros are five bucks a pop, making the entire cluster cost less than two of the big-boy sized Pi 3s.

Do you think you might have too much faith in humanity? Don’t worry, this video has you covered.

Hacking on some Lattice chips? Here’s a trip to CES for you. Lattice is holding a ‘hackathon’ for anyone who is building something with their chips. The top prize is $5k, and a trip to next year’s CES in Vegas, while the top three projects just get the trip to Vegas. If you already have a project on your bench with a Lattice chip, it sounds like a great way to wait an hour for a cab at McCarran.

UPSat. What’s an upsat? Not much, how about you? The first completely open source hardware and software satellite will soon be delivered to the ISS. Built by engineers from the University of Patras and the Libre Space Foundation, the UPSat was recently delivered to Orbital ATK where it will be delivered to the ISS by a Cygnus spacecraft. From there, it will be thrown out the airlock via the NanoRacks deployment pod.

The Voyager Golden Record is a message in a bottle thrown into the cosmic ocean and a time capsule from Earth that may never be opened. Now it’s a Kickstarter. Yes, this record is effectively Now That’s What I Call Humanity volume 1, but there are some interesting technical considerations to the Voyager Golden Record. To the best of my knowledge, no one has ever tried to extract the audio and pictures from this phonographic time capsule. The pictures included in the Golden Record are especially weird, with the ‘how to decode this’ message showing something like NTSC, without a color burst, displayed on a monitor that is effectively rotated 90 degrees counterclockwise from a normal CRT TV. Want to know how to get on Hackaday? Get this Golden Record and show an image on an oscilloscope. I’d love to see it, if only because it hasn’t been done before by someone independent from the original project.

Raspberry Pi Hive Mind

August 29, 2016 by 34 Comments

Setting up a cluster of computers used to be a high-end trick used in big data centers and labs. After all, buying a bunch of, say, VAX computers runs into money pretty quickly (not even counting the operating expense). Today, though, most of us have a slew of Raspberry Pi computers.

Because the Pi runs Linux (or, at least, can run Linux), there are a wealth of tools out there for doing just about anything. The trick is figuring out how to install it. Clustering several Linux boxes isn’t necessarily difficult, but it does take a lot of work unless you use a special tool. One of those tools is Docker, particularly Docker Swarm Mode. [Alex Ellis] has a good video (see below) showing the details of a 28 CPU cluster.

Continue reading “Raspberry Pi Hive Mind” →

Hackaday Prize Entry: An Oven Of Raspberry Pis

August 13, 2016 by 49 Comments

When the Raspberry Pi was introduced, the world was given a very cheap, usable Linux computer. Cheap is good, and it enables one kind of project that was previously fairly expensive. This, of course, is cluster computing, and now we can imagine an Aronofsky-esque Beowulf cluster in our apartment.

This Hackaday Prize entry is for a 100-board cluster of Raspberry Pis running Hadoop. Has something like this been done before? Most certainly. The trick is getting it right, being able to physically scale the cluster, and putting the right software on it.

The Raspberry Pi doesn’t have connectors in all the right places. The Ethernet and USB is on one side, power input is on another, and god help you if you need a direct serial connection to a Pi in the middle of a stack. This is the physical problem of putting a cluster of Pis together. If you’re exceptionally clever and are using Pi Zeros, you’ll come up with something like this, but for normal Pis, you’ll need an enclosure, a beefy, efficient power supply, and a mess of network switches.

For the software, the team behind this box of Raspberries is turning to Hadoop. Yahoo recently built a Hadoop cluster with 32,000 nodes used for deep learning and other very computationally intensive tasks. This much smaller cluster won’t be used for very demanding work. Instead, this cluster will be used for education, training, and training those ever important STEAM students. It’s big data in a small package, and a great project for the Hackaday Prize.

The HackadayPrize2016 is Sponsored by:

Raspberry Pi Cluster Build Shows How And What

May 26, 2016 by 36 Comments

Raspberry Pi clusters are a dime a dozen these days. Well, maybe more like £250 for a five-Pi cluster. Anyway, this project is a bit different. It’s exquisitely documented.

[Nick Smith] built a 5-node Pi 3 cluster from scratch, laser-cutting his own acrylic case and tearing down a small network switch to include in the design. It is, he happily admits, a solution looking for a problem. [Smith] did an excellent job of documenting how he designed the case in CAD, prototyped it in wood, and how he put the final cluster together with eye-catching clear acrylic.

Of interest is that he even built his own clips to hold the sides of the case together and offers all of the files for anyone who wants to build their own. Head over to his page for the complete bill of materials (we didn’t know Pis were something you could order in 5-packs). And please, next time you work on a project follow [Nick’s] example of how to document it well, and how to show what did (and didn’t) work.

If 5 nodes just doesn’t do it for you, we suggest this 120-node screen-equipped monster, and another clear-acrylic masterpiece housing 40 Pis. This stuff really isn’t only for fun and games. Although it wasn’t Pi-based, here’s a talk at Hackaday Belgrade about an ARM-based SBC cluster built to crunch numbers for university researchers.

Designing A High Performance Parallel Personal Cluster

May 9, 2016 by 86 Comments

Kristina Kapanova is a PhD student at the Bulgarian Academy of Sciences. Her research is taking her to simulations of quantum effects in semiconductor devices, but this field of study requires a supercomputer for billions of calculations. The college had a proper supercomputer, and was getting a new one, but for a while, Kristina and her fellow ramen-eating colleagues were without a big box of computing. To solve this problem, Kristina built her own supercomputer from off-the-shelf ARM boards.

Continue reading “Designing A High Performance Parallel Personal Cluster” →

Moore’s Law Of Raspberry Pi Clusters

April 17, 2015 by 52 Comments

[James J. Guthrie] just published a rather formal announcement that his 4-node Raspberry Pi cluster greatly outperforms a 64-node version. Of course the differentiating factor is the version of the hardware. [James] is using the Raspberry Pi 2 while the larger version used the Model B.

We covered that original build almost three years ago. It’s a cluster called the Iridris Pi supercomputer. The difference is a 700 MHz single core versus the 900 Mhz quad-core with double-the ram. This let [James] benchmark his four-node-wonder at 3.048 gigaflops. You’re a bit fuzzy about what a gigaflops is exactly? So were we… it’s a billion floating point operations per second… which doesn’t matter to your human brain. It’s a ruler with which you can take one type of measurement. This is triple the performance at 1/16th the number of nodes. The cost difference is staggering with the Iridris ringing in at around £2500 and the light-weight 4-node built at just £120. That’s more than an order of magnitude.

Look, there’s nothing fancy to see in [James’] project announcement. Yet. But it seems somewhat monumental to stand back and think that a $35 computer aimed at education is being used to build clusters for crunching Ph.D. level research projects.

40-Node Raspi Cluster

February 17, 2014 by 49 Comments

Multi-node RasPi clusters seem to be a rite of passage these days for hackers working with distributed computing. [Dave’s] 40-node cluster is the latest of the super-Pi creations, and while it’s not the biggest we’ve featured here, it may be the sleekest.

The goal of this project—aside from the obvious desire to test distributed software—was to keep the entire package below the size of a full tower desktop. [Dave’s] design packs the Pi’s in groups of 4 across ten individual cards that easily slide out for access. Each is wired (through beautiful cable management, we must say) to one of the 2 24-port switches at the bottom of the case. The build uses an ATX power supply up top that feeds into individual power for the Pi’s and everything else, including his HD array—5 1TB HD’s, expandable to 12—a wireless router, and a hefty fan assembly.

Perhaps the greatest achievement is the custom acrylic case, which [Dave] lasered out at the Dallas Makerspace (we featured it here last month). Each panel slides off with the press of a button, and the front/back panels provide convenient access to the internal network via some jacks. If you’ve ever been remotely curious about a build like this one, you should cruise over to [Dave’s] page immediately: it’s one of the most meticulously well-documented projects we’ve seen in a long time. Videos after the break.

Continue reading “40-Node Raspi Cluster” →