Raspberry Pi Zero Cluster Packs a Punch

If you could actually buy 16 Raspberry Pi Zeros, you might be able to build your very own Raspberry Pi Cluster for only $80! Well… minus the cost of the board to tie them all together…

A Japanese company called Idein is developing a Raspberry Pi module called the Actbulb for computational sensing and data analysis. In order to perform internal testing they decided to make things easier for themselves by developing a board to allow them to plug in not one, not two, but sixteen Raspberry Pi Zeros:

Since we will use Pi’s GPU for image processing, deep learning, etc. We need real Pis but not just Linux machines. Another reason. It can be used for flashing eMMCs of our devices via USB ports when we have to do that by ourselves.

The board has PC mount micro-USB connectors, two for each Pi. These are shown as the only think holding the Pi in place, but with 16 of them mounted you could easily use spacers and rods to make a frame through the Pi mounting holes. The carrier board breaks out each USB port and add 16 Ethernet connectors. Future improvements aim to add a switch to the board and reduce this count to just 2 ports.

Unfortunately, they only have a single Zero for now. We look forward to seeing them populate the rest of the sockets! Overwhelming interest in the project has caused the company to consider actually making and selling these in the future. Just imagine how many clusters of clusters you could make with this!? Compared to this 120 Raspberry Pi monstrosity, you could probably fit 120 Raspberry Pi Zero’s into a shoebox!

[Thanks Nichin! via CNX-Software]

88 thoughts on “Raspberry Pi Zero Cluster Packs a Punch

    1. The throughput on that final hub must be dismal.

      Ever wonder why it’s easy to find a gaggle of 4 and 7 port hubs but not so much an 8 port? Not saying 8 ports don’t exist, pay attention to the disparity.

      A lot of hub makers will gang two 4-port ICs together to make a 7 port hub (or three to make 10). To make 255 ports, the throughput at the end of that chain must be awful.

      I think the Zero has 2.0 ports (I could be wrong here) so it might be more effective to employ an ethernet switch if the purpose of the USB ports is to bring out network access and you’re running a network of recent vintage.

    1. Best of luck, finally managed to get one and for the life of me I can’t get the thing to reboot (via command prompt) more than like three times without borking the SD card somehow. Tried three SD cards and two power supplies that I’ve used with all of my other SBCs without any trouble.

      Mulling over ordering a couple new SD cards to give it one last shot but at this point it’s probably less expensive to just eat the cost of the thing :/

          1. @MetalPhreak, Thanks, I stand corrected, ENC28J60 is a 10mbit part.
            [Truth] below, states “The data throughput is not as critical as the matrix multiplication ability of the GPU”
            There are other PHY that go upto 100Mb the real point here is: will SPI help?

    1. The article already mentions that, “Future improvements aim to add a switch to the board and reduce this count to just 2 ports.”

      You don’t need a router on the board, just a switch.

  1. supporting the boards by only the usb connector is asking for trouble.

    this is part of the problem of the pi boards; they were never planned out well enough for mounting, rackmounting or concentrating them in a density box.

    plus, the usb system is actually the sore spot of the whole design!

    too bad beaglebone does not have a small board like this. they don’t have the elephant-in-the-room usb bug that has never been truly fixed (since its a design issue and can’t really BE fixed).

    1. In their application, moving the data in and out works even with the suboptimal/poor/dire USB performance that is common to all the BroadCom chips used in the RPi products. The data throughput is not as critical as the matrix multiplication ability of the GPU. https://twitter.com/9_ties/status/677267898474225664 “8GFLOPS with $5 computer and $3 microSD means $1/GFLOPS.”, excluding the cost of their board. And that cost will now drop through the floor for them as they are going to make an improved PiZero Cluster Board and raise the money through a crowd funding campaign.

          1. Lordadmiral Drake

            GeForce 980 (No RAM, No Motherboard, no CPU, no PSU, no HDD – add another $450 for that)
            Power: 185 W
            cost: ~$550
            Cores (CUDA) 2048
            clock Rate: 1126MHz (boost 1216MHz)
            RAM: 4GB
            Memory speed 7GBPS

            Performance: ~ 2555 (actual test) 4612 (theory) single-precision GFLOPS*
            Performance: ~ 80 144 double-precision GFLOPS* (1/32 of the single-precision)

            * https://devtalk.nvidia.com/default/topic/810614/what-to-buy-now-for-cuda-calculations-/
            * https://www.pugetsystems.com/labs/articles/GTX-980-Ti-Linux-CUDA-performance-vs-Titan-X-and-GTX-980-659/

            So after you add the hardware to get it to work about 2.5GFLOP per $ (actual) or 4.6 GFLOP per $ (theory)
            Cost of running for 1 year at 10cents per KWH 0.1*365.25*24*(300/1000)W = $263 or 0.1029$/GFLOP
            Total cost for 1 year: initial cost: $1000, running costs: $263

            PiZero (no MicroSD card, add another $3 for that)
            Power: 3.75W
            cost: $5
            Cores: (QPU – Quad Processing Unit) 12
            clock Rate: 1 GHz
            RAM: 512 MB (shared with GPU)
            Memory speed: ~1.2GBPS

            Performance: 8 (actual test) 24 GFLOPs (theory)

            So after you add the hardware to get it to work about 1GFLOP per $ (actual) or 3 GFLOP per $ (theory)
            Cost of running for 1 year at 10cents per KWH 365.25*24*1/(3.75/1000)W = $3.29 or 0.4109$/GFLOP
            Total cost for 1 year: initial cost: $8, running costs: $3.29

            Yes it is about 4x less effcient, but the money involved is two orders of magnitude lower.

    2. Exactly: ” the elephant-in-the-room usb bug that has never been truly fixed”

      The A+ has massive, massive issues with USB. Based on 30 A+’s passing through my hands, the chances of hitting problems with USB hubs is 50%, even Wifi dongles are hit and miss. Problems seem random, depend on the particular unit and time of day/lunar cycle… It’s something often complained about but – what **is** the problem? And why is it not discussed more?

      So the Zero, as I understand it, has this lethal flaw as well? I had to design the A+ out of my project – it caused so much misery with its USB reliability problems I had to discard about 10 of the 30 units.

        1. Yes, there’s lots of postings on this problem: some A+s refuse to work with some USB devices – like hubs, but also wifi dongles or even keyboards. But they then are fine with other other devices, even of the same type. And a hub that won’t work on one A+ will be fine on another, for instance. I tried about 30 A+’s bought from different places. Some A+s are very choosy on what they want on their USB port, others have no problem. It is known that (early?) A+s had mechanically poor USB connectors, but there’s something else. Of course the A+ has different electronics to drive the USB than the B+. That much I know. What the problem is exactly, I never found out. Not power. You see postings on lots of fora on this, but there’s not been a clear answer as to what is causing it. AFAIK.

          1. A bad bin?

            A bit like ordering LEDs from China. You can order guaranteed LEDs from Digikey and pay extra or order you LEDs from China where 20% of the LEDs are out of spec. Or in the case of those high powered LEDs, only 20% of the grid actually work. You pay about 20% of the cost for these defective parts and no mention of their status.

            Maybe the Pi foundation worked out a deal with Broadcom where they’re being sold the bottom most bin. The stuff that barely works and gets sold on Aliexpress for crazy low prices. Would help explain the price point there.

        1. Did you ever try with USB hubs? That’s where they seem to stumble worst. I had one batch from Aliexpress – of all places – which were mostly fine, then two other batches from more – expensive? – sources, which had problems. That was when I got on this track of forum postings. A dozen of other people I know directly reported similar problems. I just never heard what was the root cause. I just hope te Zero is not afflicted by the same issue. It’s such a great board (when it becomes available, that is).

    3. “this is part of the problem of the pi boards; they were never planned out well enough for mounting, rackmounting or concentrating them in a density box.”

      It’s almost like they were designed to be cheap and to sit on a school desk for children to use. What you think????

      My iPhone is badly designed as it has no rack mounting holes………………………..

  2. Putting a switch chip on board will help reduce part count significantly. By connecting the USB Ethernet MAC’s MII interface directly to the switch’s MII interface you can omit all the PHY chips and associated passives. There are 16+1 switch ICs out there so it’s just a case of doing the leg work.

    1. USB is not the only available datapath. The RPi0 to ESP hack uses SDIO for Wifi.
      Make SDIO to MII bridges in FPGA and connect that up to Ethernet switch. Strip down the driver and use it to dump raw Ethernet packet to SDIO.

        1. I can offer some basic knowledge of VHDL and Verilog and hands-on experience with MII to a KSZ8873MLL ethernet chip. A KSZ8873MML has not one but two 100MBit/s MII ports and would be a good starting point to explore SDIO-to-Ethernet interconnections.

  3. Rspi zero makes me think of the One plus one phone…. by the time the suckers got it, it was already a second gen phone from any of the big manufacturers.
    Kudos on the cluster though, i could use a 64 node one.

  4. Can somebody please explain to me what this thing is supposed to do?
    I only see 16 PIs, connected to 16 USB-to-Ethernet-converters, and 16 USB connectors.

    Whats the advantage over an old 20$ thinclient with USB-hubs?
    Why is everyone trying to fit a PI into products for all one is worth?
    Why would i pay around 200$+ (just a rough estimate) for a board to connect 16 PIs together (another 80$), if i can get a whole f*cking NEW Computer/Laptop for the same Money?
    And its most probably got more computing power, too!

    Till now, i cant think about a sensible project to do with my PI.
    I cant use it as NAS, because it lacks SATA ports. cant use it for small embedded projects, because its bootup-time. Peripherals not realtime capable. Needs to shut-down properly, otherwise Filesystem might get damaged. And i dont need another media player…

    And then, the new PI zero doesnt even have Ethernet!
    I wouldn’t even know how to USE this thing, even if i had an application!

    1. It’s great for idling on IRC!

      Srs tho, sensors with a need for a bit of compute appear to pair up well with it. The USB standard has great realtime support but it gets so little use that documentation is rare. Also DMA drivers are sabotaging rt-USB. No doubt stuff like Intel ME isn’t helping here.

  5. With all the GPIO pins, why go through the USB at all? This seems like the perfect example of a problem to be solved with a non-standard communication back plane. I doubt the Pi has the ability to do fiber channel (yes, it can be over copper). And while it isn’t my field of computers, I’ve often wondered at the obsession of using USB/Ethernet/802.11bgnac comes from. It wasn’t that long ago that every big company had some different net supporting inter-chip communication for their large rack-based equipment; and rather than cook up a Pi-powered-network it seems the focus is “how to make the USB not suck”.

    *rant does not apply to chips where the ethernet PHY is already built in: it makes sense to use what you have. But if USB is too crap, why not tie those GPIO to . . . a DSL SOC or something?

  6. I came up with a tentative workaround.
    It uses the HDMI port as a high speed network and an FPGA (thanks Jeri!) to route the hundreds of signals back through a USB interface allowing each node to communicate with all other nodes at close to the maximum possible speed.
    Turns out that with some clever code it only needs a single e2prom to store the FakeDMI that emulates a connected monitor and you can use two of the four pairs as a single direction networking setup.
    This also allows the GPGPU to be used as processing which quadruples the effective throughput of the system.

  7. http://www.sequoia.co.uk/product.php?id=1257
    This could work although an FPGA might be slightly cheaper and also add the ability to isolate damaged nodes without compromising overall system integrity.
    Also each Pi still has USB so if some of the channels are broken it can still work albeit slower than normal.
    Hell, just use infrared or WDM over fiber and have it both ways, LEDs can be very good at distinguishing different wavelengths of detected light as I (re)discovered in 1999.

    1. reliably detecting information carried by a single wavelength out of many is not that easy. most of the current telco stuff relies on AWGs to separate the passed wavelengths from the line, and lead them to the different ROSAs. and all receivers are wideband ones, so they see the whole spectrum, no matter if it’s C or DWDM. Without filters you just get noise if there are multiple wavelengths.
      coherent wavelength detection is available as well, but only in the 100Gbps receivers (which in fact does 25Gbps DQPSK) and requires tuneable lasers. to my best knowledge this is the only way to get a single wavelength carrier out of the line without using passive filters.

      so i’d guess this is not that simple to pull this off.

  8. This is a great idea and would definely buy that board. And more Zeros.

    Does anyone know why RPi Zero supplies are so damn low still?

    It’s enough long from the release to make out more Zero’s but still all stocks are at Zero. Maybe that’s where the name comes? Not available?

    Is this another “5$ PC” hoax? Maybe it’s not really possible to make 5 dollar/pound/whatever computer.

    1. Broadcom made the chip in the pi zero. Avago bought Broadcom and is laying off Broadcom employees: https://www.thelayoff.com/broadcom
      Not sure if the $5 raspberry pi zero is even sustainable, or just a loss leader, something akin to the netpliance i-opener.

      The next cheapest thing is the $10 orange pi one:

  9. Please be patient, the code is under development..
    I am working on a way to get this to run using just the composite output and an external LM1881 for logic, etc.
    Turns out separating phase sequential colours is not difficult if you have all the syncs and back porch etc available.

  10. Also LEDs are not quite as bad as people think, for my FYP I actually implemented a dual mode LED/photodiode system which used polarized light reflecting off an LCD to send data. Got something like a 40% gain in ambient light rejection too

  11. Hello.

    Please do not replaceethernet ports with ethernet hub/switch. It`s will be a “bottleneck” for future expanding !!! The best will be use old cisco stackable switches for future growing.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s