Getting PCIe Working On The New Pi 5

After the Pi 4 released, a discovery was quickly made that the internals of the popular single-board computer use PCIe to communicate with each other. This wasn’t an accessible PCIe bus normally available in things like desktop computers for expansion cards, though; this seemed to be done entirely internally. But a few attempts were made to break out the PCIe capabilities and connect peripherals to it anyway, with varying levels of success. The new Pi 5 seems to have taken that idea to its logical conclusion and included a PCIe connector, and [George] is showing us a way to interface with this bus.

The bus requires the port to be enabled, but once that’s done it’s ready to be used. First, though, some support circuitry needs to be worked out which is why [George] is reverse engineering the system to see what’s going on under the hood. There are a few handshakes that happen before it will work with any peripherals, but with that out of the way a PCIe card can be connected. [George] removed the connector to solder wires to the board directly in order to connect a proper PCIe port allowing a variety of cards to be connected, in this case a wireless networking card and an old Firewire card. This specific build only allows Gen 1 speeds, but the bus itself supports faster connections in theory with better wiring and support circuitry.

While it might not be the prettiest solution, as [George] admits, it does a great job of showing the inner workings of this communication protocol and its use in the new, more powerful Raspberry Pi 5. This makes a lot of things more accessible, such as high-speed PCIe HATs allowing for a wide range of expansion for these popular single-board computers, which wouldn’t have been possible before. If you’re still stuck with a Pi 4, though, don’t despair. You can still access the PCIe bus on these older models but it’ll take a little bit more work.

Thanks to [CJay] for the tip!

36 thoughts on “Getting PCIe Working On The New Pi 5

  1. Definitely interesting but I’ll wait for an official expansion board personally. Jeff Geerling has done some tests with a test one that he was able to borrow from the Raspberry Pi folks and that can get up to gen 3 speeds. But this definitely falls into the realm of hacks.

      1. It really boils down to do the drivers do DMA, and do they assume they can do misaligned DMA? Many driver portability issues in Linux right now boil down to misaligned DMA requests. It used to be *much* worse on ARM boards, at one point adding a USB hub in the chain hanging off a host port could cause devices downstream of the hub to have their DMA buffers get misaligned due to quirks of how the embedded USB host controllers worked and cause severe performance loss or even system instability as misaligned DMA requests caused CPU exceptions which weren’t always handled properly. (Just ask anyone who used USB Ethernet downstream of a hub on a first generation BeagleBoard, Chumby 8″ or countless other devices at the time). There are still quite a few drivers that make no attempt to ensure source and destination DMA addresses are properly aligned, and either corrupt memory or cause exceptions and kernel panics. Now that we have PCIe slots on ARM hosts, the same kinds of issues will be found and hopefully fixed.

    1. And surpassed by other boards already supporting it out of the box. I sometime wonder whether the Raspberry Pi had slowly transitioned to a religion of some sort with people in lines waiting for the new model to be introduced or appear in stock again, while they could find more powerful and often cheaper alternatives right now.
      The “large community yadda yadda”? Folks, *you* are the community. Go spend a few bucks in something and start contributing to build one, or maybe take a look where it already exist, such as the Armbian and DietPi forums. Contrary to popular belief, often fueled by bad faith journalism, the RPi is not the only affordable Linux-capable board out there, it certainly is neither the most powerful nor the cheaper one, and other boards becoming unsupported after a few years is a myth debunked by the mere existence of projects such as Armbian and DietPi.

      1. The software support on the Pi is still top notch compared to many other boards. I’ve done an extensive search of single board computers for a project at work and have been sorely disappointed even by companies I thought were good at this like Olimex and Hard Kernel. Even basic things like onboard uart can lack documentation or software support on some of these boards. On the PIs most stuff just seems to work, or there’s already a hack for it.. This is my experience anyway.

      2. You forget that time is also valuable, even for a hobbyist who has limited time. I can buy a Raspberry pi (whenever they are in stock, rarely, but that should get better one day).
        I can install Octoprint on it, or just use an image. I can run klipper on it. Or home assistant. Or some NodeJS / NodeRed stuff.
        It’s true that I could use DietPi or Armbian, but since that has a smaller userbase it would mean more time needed to dig into things, read on how to set it up while I can get quick and easy answers for Raspberry pi problems.
        I don’t have time to contribute to these (for now).

      3. I think you’re really underestimating just how valuable the ability to just hit “flash” on an RPi image is from the Raspberry Pi imager for any one of dozens of different projects. I agree that the RPi is used in areas where it isn’t the best choice (old NUCs are often a better platform for self hosting server projects if you don’t need GPIO for instance), and a lot of projects do on paper support other SBCs, but you just aren’t going to get the same first class software support if you aren’t using the gold standard option, and I say that as someone who isn’t a particularly big fan of the RPi Foundation

      4. The nice thing about Pi is it actually being widely supported by trusted distros.

        I don’t want to download whatever version of sketchy Debian for some random board and hope everything is okay.

        With Pis I don’t have to do that

  2. The choice of that flex connector is really strange from signal integrity perspective. Sure, PCIe is a very forgiving and self-healing interface (as it repeats the packets when there is a checksum error), but really – why go that cheap an improper way.

    1. It’s not that bad, and you can actually reliably extend a PCIe bus very far over pretty simple cables as long as you maintain the differential pair and have adequate shielding. The pinout has grounds interleaved between signal pairs. I suspect the official cables will likely be wrapped with a shielding foil if it is determined to be required. Adding such on later isn’t hard to do at all, just scrape the resist off the ground plane on one end, attach self-adhesive copper foil wrapped around the cable.

      1. The wires need to maintain 100 to 85 ohms (gen1-2,3+) differentially, which is affected by ref ground distance, so random copper foil may or may not work. I’m surprised that it works as raw wires. There are also retry limits, which if they fail may not recover, depending on HW implementation. Flex cable would be best, but connector doesn’t show the N and P lines as being side by side. Lengths of P and N need to be time matched as well, but I forgot the specs. Look at most graphics cards, the PEX traces usually run surface as spaced pairs with distances matched.
        I didn’t see the TX inline caps, didn’t watch the whole vid yet.

    2. Because this isn’t meant to be a production grade implementation, they’re making a quick and dirty interposer so they can read off signals and do some protocol reverse engineering.

    1. Well, you can get a FPGA to do that.. or figure out where to source the RP1 chipset. I would be interested to hack that one – into the form of their private development PCIe card (but smaller, without those external power rails)

  3. Is this the end of Raspberry Pi/ARM?
    In my eyes, the Raspberry Pi 5 seems to be last of its kind, a last desperate uprising.

    The original idea of the Pi was to provide an affordable Single Board Computers (SBC) for the education sector. For IT class, for electronics etc. What happened to that?

    All in all, the Pi 5 reminds me more of a lower end Playstation, a Thin Client etc.

    It’s more of a wannabe PC than an embedded system.
    It has too much of desktop connectivity, rather than industrial i/o.

    The GPIO connector had barely improved, has it?
    Why are no optical links installed, say SPDIF, IrDA or something similar.

    Or something more modern, but that’s not related to home cinema/TV consumer stuff but real electronics?

    Something that goes back to educational sector?
    Something akin to an user port/cartridge port?
    As it had been done in the C64 days?

    The GPIO port as such was a good start here, but the ability to boot from a serial EPROM or parallel EPROM would be cool.

    A firmware update could help here, maybe, too.

    1. The new GPIO coprocessor improves and solidifies the GPIO functions & pinout. Not having dedicated optical SPDIF or IrDA hardware keeps the base board cheap and flexible in function, and leaves application specific functionality up for HATs to implement.

    2. The SPDIF era is somewhat over my friend.

      I do hold a decent sound system and projector, but modern media comes in codecs which are not going to work – eg. EAC3 wont like SPDIF, and not even Kodi is able to transcode this into AC3 realtime during playback (I have to do a offline remuxing of the video with ffmpeg to conver the audio track). So it seems my only way would be to upgrade from Logitech Z5500 to a discrete set of speakers with an expensive hdmi equipped receiver / amplifier, which will strip out the audio streams.

      If you lookup the NUC offerings, many of the last generations did lack the SPDIF as well, likely for these reasons. And the mainboards the same.. while up to certain generation I was happy to see a optical or coax out, new ones dropped these in favor of HDMI.

    3. I see the PI OS based hardware from the UK as a bridge between PC and embedded. If you intend to use it only as PC … skip it and buy a PC. No substitute. Spend upwards of $1000+ for a really decent system and stop complaining :) . But for $15 (Zeros) up to $80 (3,4,5s). I get a very useful ‘small’ form factor device that I can ssh into, vnc into, interface to displays/keyboards/mice/HATs/Cameras/Picos/networks/etc with a 40 pin GPIO header built in to boot. And that is all there out of the box. Easy peasy.

      Because of the user base, there is usually a module already written (or an example available) to do this or do that, so one can quickly build a project when the extra power is needed. Just don’t try to make it something it isn’t!!! Granted for a lot of my projects now, the lowly Pico, Pico W, and RP2040 boards handles them just fine. But when I want a little audio playback, camera work, or a hat that want to exploit, or setup a Pi-Hole, dns server, redis server, etc., grab a cheap RPI board and off and running.

      As for education, I don’t see RPI Foundation going away from that, it simply is getting better and more robust. And still ‘cheap’. Students can still write their reports, learn a programming language, blink an led, move a rebotic arm, search the web for solutions, write a little program, etc on an RPI-4 and RPI-5. It’s a all-in-one. You certainly don’t need a Ryzen 7950X for these applications nor would you have the GPIO to interface too if desired!! Also saves the school some money instead of feeding the beast with expensive Windows (should be Linux, but) based PCs or limited use Cromebooks every where.

      I don’t see what is not to like about the RPI eco system. It works.

      Now, if I could only get my hands on an RPI-5 ;) Ha! Still waiting for pre-order to show up across the Atlantic….. Can’t wait.

      1. I look at it as ‘expanding’ your market, and with more income, more development can happen. Which benefits everyone. Win win. And nothing wrong with pursuing the path to be independently wealthy along the way. Isn’t that why we pursue a career in the first place?

  4. While I’m all in for adding features to hardware that was not initially intended to have, I’ll do a hard pass when RPi is in question.
    There are other SBCs on market that hame several M.2 slots fro the start for far less money. Also, RPi decided to criple RPi5 when it comes to camera features so there is that….

    1. There are other American and European SBC boards out there? I’m only aware of RPI, BeagleBone, Arduino, Sparkfun, and Adafruit offerings for simple controllers and Linux based boards. I suppose I could be missing others, but not aware of ’em. And they all seem reasonably priced for what you get. Just ordered a couple Adafruit Metro boards today (RP2040 based).

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.