We’ve started designing a PCIe card last week, an adapter from M.2 E-key to E-key, that adds an extra link to the E-key slot it carries – useful for fully utilizing a few rare but fancy E-key cards. By now, the schematic is done, the component placement has been figured out, and we only need to route the differential pairs – should be simple, right? Buckle up.
Getting Diffpairs Done
PCIe needs TX pairs connected to RX on another end, like UART – and this is non-negotiable. Connectors will use host-side naming, and vice-versa. As the diagram demonstrates, we connect the socket’s TX to chip’s RX and vice-versa; if we ever get confused, the laptop schematic is there to help us make things clear. To sum up, we only need to flip the names on the link coming to the PCIe switch, since the PCIe switch acts as a device on the card; the two links from the switch go to the E-key socket, and for that socket’s purposes, the PCIe switch acts as a host.
While initially routing this board, I absolutely forgot about one more important thing for PCIe – series capacitors on every data pair, on the host TX side of the link. We need three capacitor pairs here – on TX of the PCIe switch uplink, and two pairs on TX side of the switch – again, naming is host-side. I only remembered this after having finished routing all the diffpairs, and, after a bit of deliberation, I decided that this is my chance to try 0201 capacitors. For that, I took the footprints from [Christoph]‘s wonderful project, called “Effect of moon phase on tombstoning” – with such a name, these footprints have got to be good.
We’ve talked about differential pair calculations before in one of the PCIe articles, and there was a demo video too! That said, let’s repeat the calculations on this one – I’ll show how to get from “PCB fab website information” to “proper width and clearance diffpairs”, with a few fun shortcuts. Our setup is, once again, having signals on outer layers, referenced to the ground layer right below them. I, sadly, don’t yet understand how to calculate differential impedance for signal layers sandwiched between two ground planes, which is to say – if there’s any commenters willing to share this knowledge, I’d appreciate your input tremendously! For now, I don’t see that there’d be a tangible benefit to such an arrangement, anyway.
Different Stackup Might Help
This time, I’m going with a 4-layer 0.8 mm stackup – otherwise the board won’t fit into an M.2 socket. According to the design rules, I can go down to 3.5 mil (0.09 mm) traces and spacings, as opposed to the usual 6 mil (0.16 mm) traces I’m used to when doing generic 2-layer boards. Initially, I’m choosing the 7628 stackup variant here – the main difference between stackups here is the prepreg thickness and the dielectric constant, which impacts minimum possible diffpair thickness and spacing.
I went by parameters on the JLCPCB stackup page, and took via parameters from the ordering page – you can put these parameters into the “File => Board Setup” window, in the “Net Classes” tab. After replacing the default parameters there with the 4-layer impedance controlled process ones – clearance, minimum trace width, via size and such, – we get some pretty lovely parameters that we can drop down to if there’s ever a tight spot, and an ability to do reasonably dense component placement.
Let’s aim for on-point 85 ohm differential impedance today – a great target wherever you can afford it. Again, traces on top, uninterrupted ground plane right under them, along the entire length of the pairs. For “7628” stackup, that means there’s 0.21 mm of material with 4.6 Er between the pairs and the ground – punch these two values into the calculator, leave copper thickness at 35 um (1oz copper), and we can play around with trace space and width values, down to our 0.09 mm limit – which leads us to a 0.225 mm width / 0.09 mm space option. This isn’t all that great space-wise, however.
You don’t have to stick with the default stackup, though! After little deliberation, I switched to the “3313” stackup – with 4.05 Er and 0.1 mm thick prepreg between top and middle layers. It seems to be a tad more expensive, but it does look a bit easier to route in the small amount of space that I have. This led me to 0.135 mm / 0.09 mm pairs, while keeping the same 85 ohms differential impedance. Now, all I need to do is to enter these parameters into the “Net Classes” table, and whenever I press ‘6’, I’ll immediately start drawing a 85 ohm impedance differential pair.
Diffpair Drawing
The IC is placed, the diffpair parameters are tuned – we can start drawing. Remember, flipping the PCIe pair polarity isn’t a crime, it’s an obligation to make routing easier on yourself. Somehow, in this design it wasn’t needed even once.
You do, however, need a bit of patience – KiCad’s differential pair drawing experience isn’t always smooth; if you’re having weird problems where one end of the pair doesn’t quite connect to another end, you’re not alone. For me, drawing these pairs was a tad infuriating. It’s conceivable that what I’m dealing with something commonly referred to as a skill issue – perhaps, there’s a setting I’m not noticing, as sometimes such “does not connect” problems are caused by DRC constraints. However, in that case, there will be a warning shown to you on the bar above.
Remember, whenever your diffpair crosses from top to bottom layer or vice-versa, it also changes reference layers, from In1 to In2 respectively – and you’ll want to add ground vias near the diffpair vias, so that the ground return current can travel along the pair, too. As usual, four vias are ideal, three are good, two are okay, and one is non-ideal but better than zero.
Once we’ve drawn diffpairs and added ground vias, there’s one last thing to take care of. While PCIe pairs don’t have to all be the same length, the two traces that each pair consists of, do have to be the same end to end. For that, you can use the differential pair skew correction tool – mapped to 9 on the keyboard. It will add a wiggle to the longer track, in the spot where a wiggle would fit best.
All The Layers
When drawing boards, you can often use one GND and one VCC internal layer – as technically, both VCC and GND can be used as reference planes for high-speed signals. However, since we’re crossing diffpairs between top and bottom layers, they would then have different references, and this is still unexplored territory for me – I’m not sure whether there’d be return current problems. I’ve started out with one 1.2 V and one GND internal plane, but afterwards, as I was sanity-checking my boards and writing this article, I talked with other engineers, re-read an interesting comment thread under the first PCIe diffpair article, and decided to switch to having both internal planes as GND.
When it comes to the outer layers where diffpairs go, you’ll want to remove the ground fill or move it away, so that the ground fill around the pair will affect the diffpair’s impedance – you are highly likely see a ground keepout if you take a look at any PCIe card you own. Here, I am going to route diffpairs on the outer layers, but I’m not not going to completely remove ground on these, using keepouts instead. First off, having ground is cool – it’s extra copper that can help dissipate heat from the switch, or the switching regulator, or both. Second thing is – JLCPCB has been weird about ground fills on dense boards recently, and I’ll want to avoid that.
How far should the keepout go, then? The 5L rule (five trace widths away) is a good one, or you can just keepout the entire area where you’re pulling the differential pairs. For us, 5L means 5 * 0.135 mm, or 0.685 mm distance between the pair/via and the ground fill. The 5L rule isn’t a hard limit – keep to it as much as possible, but don’t stress about having a few vias here and there. Remember, when wiring up PCIe, it’s important that the string is wet.
For power wires, use your thickest traces possible wherever you have space. Of course, you don’t need to pull a 2 mm track wherever you want to put 3.3 V, but having 0.6 mm or 1 mm tracks on a 3.3 V 1 A path is pretty ordinary – some will say it’s overkill, but if you have enough space, there’s hardly a benefit to not doing it. If you’re wondering what you can get away with, there are trace width calculators that will give you temperature increase and inductance values, but most of the time, making the trace thicker is a no-brainer.
After we wire up PCIe and power signals, there are a few things left. It might be tempting to route them on the inner ground layers – do your best to fight the temptation, however, as there’s usually a better way; having unbroken inner planes is widely accepted to be good mojo for diffpairs, and for signals in general. Instead, consider other nonstandard options that are less of a sin – for instance, it’s okay to remove solder pads from a footprint if the pad is unused, and that’s what I’m going to do to pull one of the PERST signals through the top layer.
Mistakes And Problems
In many aspects, this design hits the bullseye. It’s compact, it fits everything it should fit with space to spare, and the onboard power delivery is more than reasonable. Of course, there are a few possible hiccups I can foresee happening that I will check for once these boards are manufactured.
A PCIe switch functions at high speeds, so it makes sense that power consumption could be a bit harsh, and same would go for heat dissipation – indeed, you will see higher-port-count chips like these covered with glued-on heatsinks on Chinese adapters; My main concern is power consumption – a M.2 E-key card is expected to consume 2 A at most, and the peak power consumption of the switch, together with the inserted card itself, might exceed this budget for sure.
To solve that, I’m adding a separate power input, and drawing traces in a way that, if needed, it’s easy to separate the 1.2V regulator 3.3 V input from the E-key card power. My second concern about the power consumption, however, is thermal dissipation – there isn’t much thermal mass in this PCB, and the ground path isn’t too straightforward, so the chip could theoretically overheat under load. There wouldn’t really be place for a heatsink once the adapter’s plugged into a laptop, either – we’ll see just how much of a problem this is in practice.
Every board can be a small experiment, and this one’s definitely got a few things I’ve never tried before. I’ve added this board to my last PCB order, and once it arrives, I’ll share the end result with you all, one way or another. Until then, the files are on GitHub, and I hope that this board’s story gives you plenty of insights into designing with PCIe!
Both of these PCIe and m.2 article series have been excellent.
This has given some new motivation to revisit some old project ideas that involve one or the other.
Really good articles with professional level engineering. Thanks for sharing.
Maybe i’m missing something but if Mini-PCIe is one lane, why is a PCIe switch needed? Just use one lane from the m.2 port routed straight to the MiniPCIe. It’s a $2 adapter on eBay.
because it’s for cards that require two lanes.
Since m.2 supports up to 4 lanes in a x1, x2 or x4 config, you could just pipe 2 lanes thru to the mini-PCIe without the need for a PCIe switch at all…..
It’s mentioned in earlier posts of the series: this project is intended to make use of the WiFi card M.2 slot on some motherboards. Allegedly motherboard manufacturers often only route one x1 PCIe lane to such slots since WiFi doesn’t typically need more than one lane of bandwidth, and this project is similarly intended for lower bandwidth applications.
Ah, that makes more sense. I’ve never come across a mini-PCIe card that *required* 2 lanes considering mini-PCIe only has 1 lane in the standard to begin with.
These articles are why I keep up with HaD. Among the complaints of fluff pieces, there is some serious gold.
PCIE is new territory for me as well. However, I do have experience laying out high speed boards with mass production cost restrictions (resteicted fabrication features, layer counts, etc)
A tip – you can use your DC supply rail (citing VCC in the article above) as a reference plane for high speed routing if you are in a pinch as mentioned in the article. After all, a DC supply “is” a short circuit for an AC signal when performing analysis. However, nothing is quite that simple in the real world.
In practice you need to be extremely careful and remember the advice in the article above of ensuring a continuous reference plane along the high speed signal route. This means that any place where the reference plane transitions between GND and VCC the reference needs to be made continuous. This generally means having some capacitors between VCC and GND at those points which are sufficient to decouple the harmonic content of the high speed signal making the reference plane transition. It also means having enough vias connecting those capacitors to the planes to keep the inductance of the vias themselves low.
If you do it right it can be a great solution! If you miss an important detail, you’ll end up with stability issues with the interface, or worse – radiated emissions from the harmonics of the signals that you’ll fail FCC, CE, or other radiated emissions testing late in the design cycle. This’ll happen if the reference plane for the AC signal isn’t the path of least inductance/impedance you think it is and it finds one halfway across your circuit board.
Don’t let me discourage you. It’s fun to push the boundaries on things! Rules of thumb only keep you in completely safe territory. If you want to learn more, my favorite book on the subject is “High Speed Digital Design: A Handbook of Black Magic”. It’s a great read that really minimizes the math involved in favor of concepts.
So this is a good adapter to use the Google Coral Dual TPU in older/restrictive hardware that only brings a single lane to the NVME port?
Something to keep in mind is that the term “microstrip” refers to impedance-controlled signals on external layers (signal-over-ground), while “stripline” is used when the signals are on internal layers (ground-signal-ground, or other similar arrangements). Searching online for a differential pair stripline calculator should return plenty of options for calculating impedance of a differential pair routed on internal layers. Keep in mind you’ll need to keep ground planes above and below the signal pair (power planes can also work as an AC ground if your AC return currents are properly coupled between power and ground).
The mnemonic I use is that stripline contains the word “in” in it, just like StriplINe is IN the board
I believe it will add wiggles to the /shorter/ line. Also I was trying to figure out why the wiggles look weird to me. I think its because they use a very soft sawtooth on most commercial boards
I cannot believe I never checked the pin out and that 1 ‘finger’ has USB on it! Well 3.3v not 5. But a lot of USB stuff steps down to 3.3 anyway. Internal USB mod here I come?
Since there are I2C in A and B Key is there the possibility to connect stuff like GPIO expander and thermometer, then access them from Win or linux?
You mentioned that you have uploaded the files to github, but I don’t see a github link. Not that I will be able to do anything with it, but I am curious to see how the actual Kicad file looks like.
I made the mistake of getting the dual coral E key adapter only to realize that I don’t have a single computer with a dual lane E key slot. At the time, it was the only coral in stock on seeed, and I have managed to get a single tpu working all this time. I am very much invested in the final product and to see how this evolves.
Please share the schematic or at least an update on your progress! I want to learn from your design.
Yeah, more info please. There’s actually a significant market for these; many people use mini PCs as Home Assistant servers, and a Coral TPU card for AI processing in Frigate/whatever else. The Coral has two cores, but most of us can only access one. It’s been sort of just accepted. There was similar work here:
https://github.com/magic-blue-smoke/Dual-Edge-TPU-Adapter
But unfortunately, the A+E card which wed want doesn’t seem to be listed. Seriously – if you could sell this or at least offer schematics, you’d find thousands of people wanting in.