If you could actually buy 16 Raspberry Pi Zeros, you might be able to build your very own Raspberry Pi Cluster for only $80! Well… minus the cost of the board to tie them all together…
A Japanese company called Idein is developing a Raspberry Pi module called the Actbulb for computational sensing and data analysis. In order to perform internal testing they decided to make things easier for themselves by developing a board to allow them to plug in not one, not two, but sixteen Raspberry Pi Zeros:
Since we will use Pi’s GPU for image processing, deep learning, etc. We need real Pis but not just Linux machines. Another reason. It can be used for flashing eMMCs of our devices via USB ports when we have to do that by ourselves.
The board has PC mount micro-USB connectors, two for each Pi. These are shown as the only think holding the Pi in place, but with 16 of them mounted you could easily use spacers and rods to make a frame through the Pi mounting holes. The carrier board breaks out each USB port and add 16 Ethernet connectors. Future improvements aim to add a switch to the board and reduce this count to just 2 ports.
Unfortunately, they only have a single Zero for now. We look forward to seeing them populate the rest of the sockets! Overwhelming interest in the project has caused the company to consider actually making and selling these in the future. Just imagine how many clusters of clusters you could make with this!? Compared to this 120 Raspberry Pi monstrosity, you could probably fit 120 Raspberry Pi Zero’s into a shoebox!
[Thanks Nichin! via CNX-Software]
91 thoughts on “Raspberry Pi Zero Cluster Packs A Punch”
In before “it would be easier to use the OTG port to be an ethernet adapter, then connect the cluster like that”
Who has a link to a 255-port USB hub? The rest of the hardware would only be $1300 for a 255-node cluster.
The throughput on that final hub must be dismal.
Ever wonder why it’s easy to find a gaggle of 4 and 7 port hubs but not so much an 8 port? Not saying 8 ports don’t exist, pay attention to the disparity.
A lot of hub makers will gang two 4-port ICs together to make a 7 port hub (or three to make 10). To make 255 ports, the throughput at the end of that chain must be awful.
I think the Zero has 2.0 ports (I could be wrong here) so it might be more effective to employ an ethernet switch if the purpose of the USB ports is to bring out network access and you’re running a network of recent vintage.
256 would only be 4 ^ 4 so all devices would be at most 4 hubs down. That’s not going to be abysmal bandwidth. Remember hubs don’t need to be in chains.
USB only supports 127 devices, including all the hubs.
And that’s if you are lucky and your root hub actually supports 127 device descriptors. I got bitten recently because the newer intel chipsets only support 80 device descriptors.
Last I knew 127 was the most slaves you could have on a single usb root? hub.
Ooops Artenz and birdmun are correct on 127 device limit. I stand corrected.
I just had an awesome thought on this, They should start a program that calls for people to send them Pi Zeros in exchange for a final version of the board. Or an extreme discount!
Maybe it would be more interesting to have one physical ethernet port per Pi, then link them all up in an internal router?
ethernet on pi goes thru usb and that’s part of the problem ;(
via SPI ? (if not enough bandwidh needed)
ENC28J60 is a 10mbit part, not 100mbit, so would end up being slower than USB.
@MetalPhreak, Thanks, I stand corrected, ENC28J60 is a 10mbit part.
[Truth] below, states “The data throughput is not as critical as the matrix multiplication ability of the GPU”
There are other PHY that go upto 100Mb the real point here is: will SPI help?
The article already mentions that, “Future improvements aim to add a switch to the board and reduce this count to just 2 ports.”
You don’t need a router on the board, just a switch.
You don’t need a switch, just a hub.
You don’t need a hub, just a bucket of wishes because you can’t actually buy a Pi Zero.
That’s why they have only one.
(And I’m still waiting for mine!)
I managed to get two, but that’s because Pimoroni accidentally sent me another after my first one got lost in the mail. Surprisingly, they both managed to get delivered within a day of each other.
WOOT! I got 3. 1 in the magpi (unopened), 1 opened magpi, and my sister-in-law found one of the last ones at the local microcenter
Best of luck, finally managed to get one and for the life of me I can’t get the thing to reboot (via command prompt) more than like three times without borking the SD card somehow. Tried three SD cards and two power supplies that I’ve used with all of my other SBCs without any trouble.
Mulling over ordering a couple new SD cards to give it one last shot but at this point it’s probably less expensive to just eat the cost of the thing :/
This is a known problem with the Pi and some (quite a lot) SD cards. You are recommended to use a read-only rootfs.
supporting the boards by only the usb connector is asking for trouble.
this is part of the problem of the pi boards; they were never planned out well enough for mounting, rackmounting or concentrating them in a density box.
plus, the usb system is actually the sore spot of the whole design!
too bad beaglebone does not have a small board like this. they don’t have the elephant-in-the-room usb bug that has never been truly fixed (since its a design issue and can’t really BE fixed).
In their application, moving the data in and out works even with the suboptimal/poor/dire USB performance that is common to all the BroadCom chips used in the RPi products. The data throughput is not as critical as the matrix multiplication ability of the GPU. https://twitter.com/9_ties/status/677267898474225664 “8GFLOPS with $5 computer and $3 microSD means $1/GFLOPS.”, excluding the cost of their board. And that cost will now drop through the floor for them as they are going to make an improved PiZero Cluster Board and raise the money through a crowd funding campaign.
1 consumer GPU and a CPU will give you partitionable GPU access at far less than $/GFLOP so what’s the point
And what would be the $/watt ?
On a GeForce 980 you’d get around 37.3 GFlops/W. A Radeon R9 Fury X Will give you 30.3 GFlops/W
The Pi Zero can do 10GFlops/W
GeForce 980 (No RAM, No Motherboard, no CPU, no PSU, no HDD – add another $450 for that)
Power: 185 W
Cores (CUDA) 2048
clock Rate: 1126MHz (boost 1216MHz)
Memory speed 7GBPS
Performance: ~ 2555 (actual test) 4612 (theory) single-precision GFLOPS*
Performance: ~ 80 144 double-precision GFLOPS* (1/32 of the single-precision)
So after you add the hardware to get it to work about 2.5GFLOP per $ (actual) or 4.6 GFLOP per $ (theory)
Cost of running for 1 year at 10cents per KWH 0.1*365.25*24*(300/1000)W = $263 or 0.1029$/GFLOP
Total cost for 1 year: initial cost: $1000, running costs: $263
PiZero (no MicroSD card, add another $3 for that)
Cores: (QPU – Quad Processing Unit) 12
clock Rate: 1 GHz
RAM: 512 MB (shared with GPU)
Memory speed: ~1.2GBPS
Performance: 8 (actual test) 24 GFLOPs (theory)
So after you add the hardware to get it to work about 1GFLOP per $ (actual) or 3 GFLOP per $ (theory)
Cost of running for 1 year at 10cents per KWH 365.25*24*1/(3.75/1000)W = $3.29 or 0.4109$/GFLOP
Total cost for 1 year: initial cost: $8, running costs: $3.29
Yes it is about 4x less effcient, but the money involved is two orders of magnitude lower.
There’s a decent but small cluster case on Thingiverse I am trying to edit to make allowances. Look up Pi zero cluster case. The dude did 3 different ones. Im still learning CAD though so do t expect much from me.
Exactly: ” the elephant-in-the-room usb bug that has never been truly fixed”
The A+ has massive, massive issues with USB. Based on 30 A+’s passing through my hands, the chances of hitting problems with USB hubs is 50%, even Wifi dongles are hit and miss. Problems seem random, depend on the particular unit and time of day/lunar cycle… It’s something often complained about but – what **is** the problem? And why is it not discussed more?
So the Zero, as I understand it, has this lethal flaw as well? I had to design the A+ out of my project – it caused so much misery with its USB reliability problems I had to discard about 10 of the 30 units.
Are you saying you have no consistency with which hubs or wifi dongles will work? What do you think is causing this?
Yes, there’s lots of postings on this problem: some A+s refuse to work with some USB devices – like hubs, but also wifi dongles or even keyboards. But they then are fine with other other devices, even of the same type. And a hub that won’t work on one A+ will be fine on another, for instance. I tried about 30 A+’s bought from different places. Some A+s are very choosy on what they want on their USB port, others have no problem. It is known that (early?) A+s had mechanically poor USB connectors, but there’s something else. Of course the A+ has different electronics to drive the USB than the B+. That much I know. What the problem is exactly, I never found out. Not power. You see postings on lots of fora on this, but there’s not been a clear answer as to what is causing it. AFAIK.
A bad bin?
A bit like ordering LEDs from China. You can order guaranteed LEDs from Digikey and pay extra or order you LEDs from China where 20% of the LEDs are out of spec. Or in the case of those high powered LEDs, only 20% of the grid actually work. You pay about 20% of the cost for these defective parts and no mention of their status.
Maybe the Pi foundation worked out a deal with Broadcom where they’re being sold the bottom most bin. The stuff that barely works and gets sold on Aliexpress for crazy low prices. Would help explain the price point there.
Well, I had 50+ A+ units trough my hands and never had an issue with the USB, mechanical or software. Works fine with a cheap USB bluetooth dongles.
Did you ever try with USB hubs? That’s where they seem to stumble worst. I had one batch from Aliexpress – of all places – which were mostly fine, then two other batches from more – expensive? – sources, which had problems. That was when I got on this track of forum postings. A dozen of other people I know directly reported similar problems. I just never heard what was the root cause. I just hope te Zero is not afflicted by the same issue. It’s such a great board (when it becomes available, that is).
Yep, I have a few hubs lying around. All worked fine. I was always driving them trough the 5V on the GPIO port, not trough the micro usb, with ample power supply.
Seems there is good news: “like with the Raspberry Pi 2 and the Pi B+, USB traffic is reliable and there is no loss of USB packets. ” (source: http://www.yoctopuce.com/EN/article/creating-an-multimeter-with-a-raspberry-pi-zero).
“this is part of the problem of the pi boards; they were never planned out well enough for mounting, rackmounting or concentrating them in a density box.”
It’s almost like they were designed to be cheap and to sit on a school desk for children to use. What you think????
My iPhone is badly designed as it has no rack mounting holes………………………..
Actually, they just sit on a shelf collecting dust.
Doesn’t seem like a lot of schools are folding the Pi into their curriculum.
Damn it, I’ve been working on making one of these, with 15 USB-Ethernet adapters and a 16 port switch.
Putting a switch chip on board will help reduce part count significantly. By connecting the USB Ethernet MAC’s MII interface directly to the switch’s MII interface you can omit all the PHY chips and associated passives. There are 16+1 switch ICs out there so it’s just a case of doing the leg work.
The MACs used appear to have integrated PHY’s so this idea won’t work in this case!
It would still save you the cost of connectors and patch cables.
You could probably emulate multiple USB ethernet adaptors and a switch in an FPGA, so you could do it all in one chip. USB -> FPGA, rather than USB -> Ethernet -> Switch
USB is not the only available datapath. The RPi0 to ESP hack uses SDIO for Wifi.
Make SDIO to MII bridges in FPGA and connect that up to Ethernet switch. Strip down the driver and use it to dump raw Ethernet packet to SDIO.
on the FPGA side that should be called “PHY”, “PHY emulation” or reverse MAC. A quick search on OpenCores only turned up a discussion.
Some Ethernet switches support what is called a ” MAC-to-MAC connections” and let you bypass the need to emulate a PHY.
I can offer some basic knowledge of VHDL and Verilog and hands-on experience with MII to a KSZ8873MLL ethernet chip. A KSZ8873MML has not one but two 100MBit/s MII ports and would be a good starting point to explore SDIO-to-Ethernet interconnections.
I was able to buy 3 at barnes & noble (MagPi #40 magazine)
I went to Barnes and Noble only to find some asshole had bought all 3 they had.
Rspi zero makes me think of the One plus one phone…. by the time the suckers got it, it was already a second gen phone from any of the big manufacturers.
Kudos on the cluster though, i could use a 64 node one.
One per account. *evil laugh*
Can somebody please explain to me what this thing is supposed to do?
I only see 16 PIs, connected to 16 USB-to-Ethernet-converters, and 16 USB connectors.
Whats the advantage over an old 20$ thinclient with USB-hubs?
Why is everyone trying to fit a PI into products for all one is worth?
Why would i pay around 200$+ (just a rough estimate) for a board to connect 16 PIs together (another 80$), if i can get a whole f*cking NEW Computer/Laptop for the same Money?
And its most probably got more computing power, too!
Till now, i cant think about a sensible project to do with my PI.
I cant use it as NAS, because it lacks SATA ports. cant use it for small embedded projects, because its bootup-time. Peripherals not realtime capable. Needs to shut-down properly, otherwise Filesystem might get damaged. And i dont need another media player…
And then, the new PI zero doesnt even have Ethernet!
I wouldn’t even know how to USE this thing, even if i had an application!
Hello, voice of reason. I don’t know if you’re welcome here but I agree with your points.
It’s great for idling on IRC!
Srs tho, sensors with a need for a bit of compute appear to pair up well with it. The USB standard has great realtime support but it gets so little use that documentation is rare. Also DMA drivers are sabotaging rt-USB. No doubt stuff like Intel ME isn’t helping here.
With all the GPIO pins, why go through the USB at all? This seems like the perfect example of a problem to be solved with a non-standard communication back plane. I doubt the Pi has the ability to do fiber channel (yes, it can be over copper). And while it isn’t my field of computers, I’ve often wondered at the obsession of using USB/Ethernet/802.11bgnac comes from. It wasn’t that long ago that every big company had some different net supporting inter-chip communication for their large rack-based equipment; and rather than cook up a Pi-powered-network it seems the focus is “how to make the USB not suck”.
*rant does not apply to chips where the ethernet PHY is already built in: it makes sense to use what you have. But if USB is too crap, why not tie those GPIO to . . . a DSL SOC or something?
i have run r pi nas / sensor server for 3.5 years. i like low power things
I think it’s because once you encounter TTL, suddenly computers aren’t ‘turtles all the way down’ and that freaks certain people out. Maybe they are Java programmers who never learned the basics?
I came up with a tentative workaround.
It uses the HDMI port as a high speed network and an FPGA (thanks Jeri!) to route the hundreds of signals back through a USB interface allowing each node to communicate with all other nodes at close to the maximum possible speed.
Turns out that with some clever code it only needs a single e2prom to store the FakeDMI that emulates a connected monitor and you can use two of the four pairs as a single direction networking setup.
This also allows the GPGPU to be used as processing which quadruples the effective throughput of the system.
Moar info! link to the working solution? Please.
This could work although an FPGA might be slightly cheaper and also add the ability to isolate damaged nodes without compromising overall system integrity.
Also each Pi still has USB so if some of the channels are broken it can still work albeit slower than normal.
Hell, just use infrared or WDM over fiber and have it both ways, LEDs can be very good at distinguishing different wavelengths of detected light as I (re)discovered in 1999.
reliably detecting information carried by a single wavelength out of many is not that easy. most of the current telco stuff relies on AWGs to separate the passed wavelengths from the line, and lead them to the different ROSAs. and all receivers are wideband ones, so they see the whole spectrum, no matter if it’s C or DWDM. Without filters you just get noise if there are multiple wavelengths.
coherent wavelength detection is available as well, but only in the 100Gbps receivers (which in fact does 25Gbps DQPSK) and requires tuneable lasers. to my best knowledge this is the only way to get a single wavelength carrier out of the line without using passive filters.
so i’d guess this is not that simple to pull this off.
This is a great idea and would definely buy that board. And more Zeros.
Does anyone know why RPi Zero supplies are so damn low still?
It’s enough long from the release to make out more Zero’s but still all stocks are at Zero. Maybe that’s where the name comes? Not available?
Is this another “5$ PC” hoax? Maybe it’s not really possible to make 5 dollar/pound/whatever computer.
i too wonder about the stock.
i check in regular intervals, but still never find them in stock anywhere.
“Does anyone know why RPi Zero supplies are so damn low still?”
Because everyone wants one.
Pimoroni say they have over 64,000 requests for alerts on the Pi Zero stock.
Let’s use unsigned 16bit, we’ll never need more than that!
Broadcom made the chip in the pi zero. Avago bought Broadcom and is laying off Broadcom employees: https://www.thelayoff.com/broadcom
Not sure if the $5 raspberry pi zero is even sustainable, or just a loss leader, something akin to the netpliance i-opener.
The next cheapest thing is the $10 orange pi one:
“themselves buy developing a board” Not the only mistake, does anyone actually proof-read these anymore?
Your part of the writing team and you still haven’t fixed it..
I wonder if due to its speed the Beaglebone PRU could be used to achieve fast board to board communication without using USB or ethernet chips.
Muphry’s law…in action
“These are shown as the only think holding the Pi in place”
What about the RasPi Compute Module? Isn’t this what it’s made for?
Yes that is what it was made for, BUT:
Compute Module – US$30 (in batches of 100)
Raspberry Pi Zero US$5
How do they know it “packs a punch” if there’s only one board in it?
It is going to run a standard Beowulf cluster or something similar to that so it should scale up nearly linear, and since they have tested the performance ( https://twitter.com/9_ties/status/677267898474225664) 8 GFLOP from one board, from 16 GPU’s it should be very close to 128GFLOP’s.
It packs a “punch zero”.
That’s why there are no Zeros in stock…
Please be patient, the code is under development..
I am working on a way to get this to run using just the composite output and an external LM1881 for logic, etc.
Turns out separating phase sequential colours is not difficult if you have all the syncs and back porch etc available.
Also LEDs are not quite as bad as people think, for my FYP I actually implemented a dual mode LED/photodiode system which used polarized light reflecting off an LCD to send data. Got something like a 40% gain in ambient light rejection too
Any updates on this ?
Tell me please, where is the link i can get this awesome sht?
Please do not replaceethernet ports with ethernet hub/switch. It`s will be a “bottleneck” for future expanding !!! The best will be use old cisco stackable switches for future growing.
Exactly what I’ve been looking for! Cannot wait!!!
Any word if this thing will ever go into production?
I’m envisaging the RPiZeros are an array like they’re at least visually lending to the idea of being their own heat sink [for such an eventuality – with a fan to pull air through them as the only moving part if it is required] – quite tightly packed together and then just what comprises a practical solution – the area of a [minimum 10″] touchscreen [serving as one side of the casing] in either vertically or horizontally arranged RPis, the former requiring less of them, more slimline and less overall computational power – or the latter, quite a hefty device, and with either probably still requiring mains power are still highly portable for the processing capabilities achieved…
The long awaited yet to commercially materialise DMA1 [Dark Matter Audio] was by no means slimline or much flatter in any dimension, which certainly suggests people still go for performance without too much concern for aesthetics, so this could even be a high budget end project for RPi enthusiasts.
In so much, I’d like to discover the capabilities for those configurations, the closest proximity of one RPi to another for fitting maximum quantity clusters and to downsize the cabling and connections while determining if a panel for connecting peripherals would be necessary – given that ‘desirable’ is [even] a forgone conclusion.
Have fun waiting 3 years to purchase all 16 of your Pi Zeros
Please be kind and respectful to help make the comments section excellent. (Comment Policy)