PCIe Multiplier Expands Raspberry Pi 4 Possibilities

It probably goes without saying that hardware hackers were excited when the Raspberry Pi 4 was announced, but it wasn’t just because there was a new entry into everyone’s favorite line of Linux SBCs. The new Pi offered a number of compelling hardware upgrades, including an onboard PCI-Express interface. The only problem was that the PCIe interface was dedicated to the USB 3.0 controller; but that’s nothing a hot-air rework station couldn’t fix.

We’ve previously seen steady-handed hackers remove the USB 3.0 controller on the Pi 4 to connect various PCIe devices with somewhat mixed results, but [Colin Riley] has raised the bar by successfully getting a PCIe multiplier board working with the diminutive Linux computer. While there are still some software kinks to work out, the results are very promising and he already has  a few devices working.

Getting that first PCIe port added to the Pi 4 is already fairly well understood, so [Colin] just had to follow the example set by hackers such as [Tomasz Mloduchowski]. Sure enough, when he plugged the port multiplier board in (after a bit of what he refers to as “professional wiggling”), the appropriate entry showed up in lspci.

But there was a problem. While the port multiplier board was recognized by the kernel, nothing he plugged into it showed up. Checking the kernel logs, he found messages relating to bus conflicts, and one that seemed especially important: “devices behind bridge are unusable because [bus 02] cannot be assigned for them“. To make a long story short, it turns out that the Raspbian kernel is specifically configured to only allow a single PCI bus.

Fortunately, it’s an easy fix once you know what the problem is. Using the “Device Tree Compiler” tool, [Colin] was able to edit the Raspbian Device Tree file and change the PCI “bus-range” variable from <0x0 0x1> to <0x0 0xff>. From there, it was just a matter of plugging in different devices and seeing what works. Simple things such as USB controllers were no problem, but getting ARM Linux support for the NVIDIA GTX 1060 he tried will have to be a topic for another day.

[Thanks to Paulie for the tip.]

33 thoughts on “PCIe Multiplier Expands Raspberry Pi 4 Possibilities

      1. Those USB 3.0 ports are not USB 3.0 ports. The port multiplier cards are connected together with USB 3.0 cables and jacks, but they’re electrically wired for PCIe 1x. Plugging any USB 3.0 devices into them or vise-versa will let the magic smoke out and kill the port in the best case scenario.

      1. But it’s not a “CPU”, it’s a micro-controller, or, in other words, it’s an “almost everything” chip. To change “almost everything”, it’s usually easier to change “everything”.

      2. There was an old AMD project called Skybridge which would have had socketed ARM processors as part of it.

        The issue I see is that ARM cores are largely integrated on SOCs which are supposed to have everything integrated on them. Something like the Raspberry Pi Compute Module is one solution for modularity there there.

        As an enthusiast, I think it would be cool to see someone create an ARM chip that more heavily utilizes software enumerable buses like PCIe. But I also understand various reasons that doesn’t make a ton of sense.

        Still, I hope the community keeps making progress with PCIe on the Pi and we see more options open up (maybe even an inexpensive, well supported SOC with more PCIe lanes).

  1. arm drivers for AMD and Nvidia graphics cards would change the game. you could implement the new usb 4 spec on a future phone (or license a future less power hungry thunderbolt) and have it run an external graphics card and monitor. An ultra short range ghz transceiver could even be placed within a wireless charging coil so it connects and powers as soon as you set your phone down. One can only dream…

    1. Nvidia’s 710 works fine connected to our ARM virtual platforms via pcie-pasthrough, you have to modify the buildroot config for nouveau to persuade it that Aarch64 is a valid target, but then it builds and works just fine with Mesa. I’ve run the glx-gears test application and played GLTron on it. This is basically the same as the Pi setup if you’re using an Aarch64 Pi kernel.

    1. Pi4 Compute module in its present form factor wouldn’t have enough pins on the edge connector to break out PCIe. So edge connector plus an additional part for PCIe becomes harder to manufacture,& less robust in practice.
      I predict that first version won’t have PCIe, and a CM4+ may not be viable anyway.

    1. You would still need the memory card to boot. Then you would need to do a pcie/sata initialization routine that fetches the first 512 out of a defined SATA hard disk. After that, I think that the OS would need to be modified in order to be able to pull more data from the SATA drives, using this setup.

      I guess it would be much easier to pull the first 1 GB of data into RAM, and boot the OS from there. It would he much easier to implement a trivial driver inside the OS that could acknowledge the existence of a SATA controller board, inside a PCIe board.

      Most of this stuff gets done using the chipset inside the motherboard, but, since you got no chipset, you must implement this manually, and it would be much slower than the current motherboards approach.

    2. The easy solution would be to add a B-Key M.2 slot and use a PCIe drive, but it won’t be bootable, probably need drivers as well.

      Assuming Windows RT stays dead, Windows can’t and never will run in a usably on a Raspberry Pi. You unavoidably need a x86_64 board for that

    3. This should be far easier to add to the Pi4 than previous models, but right now it doesn’t look like it.
      Pi 3’s had the boot code in the BCM, and could only boot from sd, usb, gpio, and network.

      It looks like the Pi4 moved the boot loader into an eeprom though. The article I read said at the time it can only boot from sd right now but they are working on usb/net boot.
      But being in an eeprom means alternate boot methods can be created and flashed into it in the future.

      1. There is nothing wrong with trying to run windows 10 on an ARM architecture. Everybody is free to do the hacks that they want, and the use the Operating Systems that they feel necesary.

        Now, having said so, why the fuck would someone add PCIe expansion slots to a Raspi? Wouldn’t buying a generic ARM motherboard be cheaper than this?

    1. I think you answered your own question :)

      It would be kinda cool to hook up a 2 port network card to set up a little router/firewall, or hook up a bunch of sata cards and a bunch of slow hdds for big and relatively low power storage. Not that there aren’t off the shelf products that probably do it better, but hey! it’s fun!

  2. Raspberry Pi PCie connectors have a really long range of only 6 ft.

    2.1.4 Connect the cable to the Pi

    Plug the cable into a USB 3.0 port on the Raspberry Pi.

    Connect the wires from the Pi to the power/reset on the power supply on the PCIE (P8, U0, P9, P10, S1 )

    The cable should fit together snugly and the wires should be pointing towards one another.

    If you have problems getting the cable fitting around the connector (or plugging into other cables inside the Pi) this can be solved by turning off the power supply. Try not to turn the power supply off if this problem arises. Some of the cables on the PCIE may be difficult to get into the PCIE due to the tight fit.

  3. If anyone can get a Pcie to MXM adapter working, I’d be interested. I’m currently doing a old thinkpad laptop to pi conversion (I call it the ThinkPi hahaha) and something like this would be cool as hell (Although would destroy the power consumption lol)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.