$50 10Gbps Mesh Network Uses USB4

You want to build a cluster of computers, but you need a high-speed network fabric that can connect anything to anything. Big bucks, right? [Fang-Pen] developed a 10 Gbps full-mesh network using USB4 that cost him under $50. The first part of the post is about selecting a low-power mini PC, but if you skip down to the “Networking” section, you’ll find the details on the cluster.

The machines selected have two USB4 ports. In theory, you can transfer 40 Gbps on these ports. In reality, the cluster only hit 11 Gbps, but that’s still well above common Ethernet speeds. [Fang-Pen] has yet to determine why he isn’t getting even faster speeds.

Since Linux is Linux, there is a module for networking over Thunderbolt, so the rest is basically set up. There are, of course, some limitations. First, it is only fully connected because the cluster has three computers. More computers would need more USB4 ports or more hardware.

In addition, the standard says you can only count on full speed with cables 0.8 meters or shorter. However, that’s the 40 Gbps number. We wondered if a 2 m cable, rated at 20 Gbps, would have still managed 11 Gbps in this setup. A 10GBASE-T network, on the other hand, should allow 100-meter cables. But for a cluster of computers, do you really care?

We’d be interested to see this idea extended to more nodes. High-speed fabric can be useful in networked disk servers, parallel computing, and probably some other scenarios. We’ve seen 10G Ethernet on the Pi, although the PCI bus limited it to about 3.6 Gbps. For reference, we saw another three computer networks with 10GBASE-T done for about $130 with similar limitations.

10 Gigabit Ethernet For The Pi

When people like Bell and Marconi invented telephones and radios, you have to wonder who they talked to for testing. After all, they had the first device. [Jeff] had a similar problem. He got a 10 gigabit network card working with the Raspberry Pi Compute Module. But he didn’t have any other fast devices to talk to. Simple, right? Just get a router and another network card. [Jeff] thought so too, but as you can see in the video below, it wasn’t quite that easy.

Granted, some — but not all — of the hold-ups were self-inflicted. For example, doing some metalwork to get some gear put in a 19-inch rack. However, some of the problems were unavoidable, such as the router that has 10 Gbps ports, but not enough throughput to actually move traffic at that speed. Recabling was also a big task.

Continue reading “10 Gigabit Ethernet For The Pi”