Bitten By PCB Defects

If you’ve ordered PCBs from Seeed Studio, ITead, DirtyPCB, or another board house in China, you’ve probably noticed that neat little 100% e-test option available on the order form. If you select this, the board house will throw your PCBs in a machine that will poke a pin in every pad to make sure all the connections are good. Less work for you, right? As [Andy] found out, not always. He was bitten by a manufacturing defect that sheds some light on what that e-test actually is, and the capabilities of what these Chinese board houses can do.

Most of [Andy]’s project have a lot of surface mount components, and when he receives his boards, he notices a few pin pricks on each and every pad. This is from a flying lead machine; a robotic continuity checker that makes sure all the relevant pads are electrically isolated from each other.

One of [Andy]’s recent projects is an entirely through-hole design. Apparently these robotic meters don’t test through-hole pads; it’s significantly harder to measure the continuity of a hole rather than a pad, apparently. After assembling one of these boards, he noticed a problem where one of the GPIOs was permanently high. The offending trace was found underneath a DIP socket, in precisely the worst possible place it could be.

If [Andy] had inspected the board beforehand, this problem would have been avoided. He says it was a relatively simple board with big traces and wide spaces and he didn’t think a manufacturing defect was possible. He was wrong, and now we have a warning. We thank him for that.

54 thoughts on “Bitten By PCB Defects

  1. I guess if you have the PCB space it could make sense to design test points next to all relevant holes or into the traces leading to those. That way it could be “e-tested” and as a benefit you’d get easier signal access for testing and debugging too.

      1. no, the e-test machine missed the point, dm8tbr is right and it was also my first reaction after reading this article, wanted to write snarky commend ..and was disappointed to find out first comment outsmarted me :( :)

  2. I’m too lazy to make ground fills and I think that’s good. No problem with this issue :) (I’ve ordered many boards from dirt)

    Maybe increasing the clearance would be enough?

    1. Not always.

      I’m going to preface this story by saying that I’ve ordered hundreds of boards from them, and their error rate is extremely low. I think I can still count them on one hand. My own error rate is probably an order of magnitude higher than theirs.

      I recently assembled a board I got in a batch from OSHPark a while ago and it failed to accept programming. On close inspection, there was a section of the board where the traces just faded away and disappeared. The clear space around them in the ground plane was still there – just no traces. It was as if some machine somewhere ran out of ink while laying down the mask for the copper on that side of the board. I missed it because it’s a surface mount board, and the defective side was the solder side.

      1. I have been doing this for years no i did not
        and modifying gerbers is automated and uses by many cheap houses to lie about design rules and hope that the increase in isolation size will not be noticed
        and apparently i’m not the only one check out the eevblog forum

        1. Translating from broken (possibly non-)english.

          I have been doing this for years, and no I do not [send bad gerbers]. Modifying gerbers [can be] automated and [is done] by many cheap houses to lie about design rules and hope that increases in isolation size will not be noticed. Apparently, I’m not the only one: Check out the EEVBlog forum.

          (It’s not terrible english: It’s just english that we all write when we’re upset or we spend too long on IRC. It’s okay, humans suck at communicating.)

          1. but yes i was also quite upset, i work with gerbers every day at work and getting blamed for somehow sending bad gerbers vs a cheap chinese manufacturing house known for shoddy work being blamed upsets me

        2. All good board houses (except when you tell them not to) will modify the gerbers to improve the odds of getting what you designed (the very cheap ones will just run with what you gave them because its quicker and easier). The reason is that the photoplots and etching distort shapes and widths of lines. Some of these can be compensated for by adjusting the photoplot masters that the boards are etched from. If they do it right you will get exactly what you designed.

          1. i agree,
            if i were running something thats what i’d do.

            BUUUT…
            i’d be VERY careful at saving the two DIFFERENT files
            and making sure to rename and backup all involved files!

            that way i could measure my machine’s error and apply a batch mod to the files for each day. why? machines wear out and finetuning is a hassle. measuring the error is much easier and batch-scripts can do the heavy lifting.

  3. Yes; we generally get what we pay for, and although the lure of a cheap-as-chips board is powerful (and I use them myself), this is a useful reminder that we don’t get what we don’t pay for. Board manufacturing prices have fallen dramatically over the years, but I live near a former manufacturing plant for high quality boards, so I am aware that quality always costs. Nevertheless; if you don’t want to make the board yourself (and it is more difficult now for complex dense boards, unless you mill the board), these low cost boards are useful, especially if you do a manual check before use (and I’m not saying I always bother…). It’s the end-user’s risk, ultimately. It’s just that with higher priced quality boards the manufacturer may lower the risk by testing connections fully.

  4. Board fabricators will edit your gerbers routinely to better suit their often proprietary systems. That per se is not a bad thing in most cases unless they are careless and mangle your design. They should have DRC tools to prevent this by analysing the plot files to detect geometry violations.

    But then again this is why the pros exist – pay 1000 dollars for an unstuffed 4-board panel prototype on a 5 day turn and you can fully expect no faults at all, with full netlist testing and impedance calcs.

  5. I object to the statement “it’s significantly harder to measure the continuity of a hole rather than a pad”. I used to work for a company that designed e-testing equipment and I know for a fact that this is not the case. Using larger head spring probes allows the equipment to test thru-hole points with ease – just use a conical spring probe with an OD larger than the diameter of the thru-hole. Testing the capacitance of each TP should instantly warn you of continuity to the ground/power plane.

    1. What this guy said. People that don’t think board houses can test through hole pads are pretty ignorant of the whole process.

      “The last time I took my car to get new tires they didn’t check my oil and my engine died. Mechanics must not be able to check oil any more! Why can’t mechanics check oil?!?!?!”

    1. Surprised you weren’t verbally (textually?) assaulted for offering a correction.

      A warning and a tip: the editors and readership don’t take kindly to corrections here.

  6. I actually wrote on my project log recently so what related: https://hackaday.io/project/4993-dual-channel-battery-chargeranalyzer/log/17291-keeping-up-with-the-times

    There was 1 defect on my recent PCB batch for a unrelated project. I have only used 2 PCB so far. They fixed the board with a ugly manual cut. I guess I should have eased off the ground pours with “isolate” parameter with their preferred spacing instead of the default with uses minimum spacing. I also have high res scans that shows the scratch marks of a flying probe on the pads.

    They have also fixed up some of the design rule check violations by trimming the gerber. I pointed those out in the mech layer as I couldn’t fix them easily. I did run into issues with their silkscreen alignment and random speckles.

    BTW eTest was done on my FPGA board as well. I was a bit conservative and used to their preferred 8 mil spacing as it was my first board. I have not spotted any fixes on the few I looked at. :)

    1. “I guess I should have eased off the ground pours with “isolate” parameter with their preferred spacing”

      Absolutely!

      Ground pours should be made with as much isolation as you can reasonably get, without screwing up the connectivity of the plane itself. Why? Because the ground pour has a ton of places it has to stay away from: every non-connected through hole or via. Probably more than any other signal on the board. And the ground pour isn’t just a single straight line – it generates round curves around everything, making the isolation even harder. But more importantly, the tight spacing almost certainly does *nothing* in most cases: little tiny strips of ground plane aren’t going to be that important most of the time.

      I’m also surprised to hear people say that they let the board manufacturer fix design rule check violations: I *never* send a board off that doesn’t pass the design rules that they specify. The few times I’ve missed something they’ve sent me an email back, and I fix it myself.

    2. Normally, I do normally increase the isolation because I do not like the extra capacitance loads. but this was for a different type of job.

      Given the lack of options in Eagle CAD for tweaking pad shapes, I would rather have the PCB vendor modify the gerber to shave off the few mils on the pad. I do have the soldering skill to pull things off. I am used to etching my own PCB, so even a poorly tweaked gerber is going to be better than my best homebrew board.

      As for really rush jobs, for me those do not work out in the long run. The extra time I take to review the design or even sit on a layout, procrastinate and change my minds a few time always work out better. I take time to do a proper design so that I get it done right (or at least 95%) the first time. I am the guy that get out of work at 5pm and finish projects on time.

  7. I run the price-comparison site http://PCBShopper.com and I’m surprised at how many of my users DO NOT choose the cheapest PCB manufacturer. I’ve speculated on some of the reasons: the American or European PCB manufacturer might deliver faster, or the user might feel a patriotic duty to buy from a manufacturer in his own country. But quality is also definitely an issue, whether perceived or real.

    1. I got my recent PCB batch from China and it shows in 7 days. It was about $14 for for 10 pieces of 10cmx10cm double side PCB + $20 shipping by DHL . The board was done in 5 days and the courier took another 2 days. See my project log in link from an earlier post. I even done a 0.4mm pitch CSB (Chip Scale BGA) on it.

      I submit a sub board to OSH shortly after the China and gotten a notification that my board came back from fab in the same amount of time. This is a backup for the sub board with the CSB part as I wasn’t sure about quality from China. Still waiting for it to show up in mail.

      The China board shop was faster and cheaper.

      1. If you have a rush job, you’ll probably end up buying from a company in your country, not from China. By “rush job” I mean you need it in 2, 3, 4, maybe 5 days max. So you need to pay for a shorter lead time and overnight shipping. PCBShopper.com has an option where you can specify how quickly you need your boards. If you say you need them in just a few days, the Chinese manufacturers disappear from the price results, because they just can’t meet your schedule.

      2. The extra few days after the mad dash to the finish line of sending in the board file is nice. It is like the clam before the storm when you get your boards back. I usually have enough background activities to keep myself occupied e.g. firmware, BOM, test plans, prep work for board bring up etc.

        “You rush a miracle man, you get rotten miracles.” – The Princess Bride

      3. I recently had the same experience with identical parameters, so I assume you were using Elecrow. Solid 7-day turnaround to Texas, quality was good, and they threw in three extra boards. I chose Shenzhen DHL pickup.

        http://www.elecrow.com/10pcs-2-layer-pcb-p-1175.html

        I’ve also used a US board house, Sierra Circuits. Quality was flawless and turnaround was fast and predictable. But to beat 7 calendar days, you’d pay a serious premium for both manufacturing and shipping.

  8. BTW, any decent company would re-do the boards for free if you pointed out a problem like this. I would expect the better-known companies in China would do so.

  9. I don’t have much experience, but I have used seeedstudio and dirtypcb, and I have not noticed the effects of testing, however I did have a design kicked back, they said “it wasn’t safe”, and they were right. The problem was OE on eaglecad. The only other problem I have seen is that I lifted a pad one time, but again this was OE (It works better when I don’t explode the components on the board). I think ‘you get what you pay’ for is right, but so far dirtypcb has been a great experience to learn without spending a fortune.

    1. Yeah, part of me wonders when I see the problems here if it isn’t a limitation of Eagle/KiCAD/etc. Ground pours are something that’s not exactly trivial to make sure you’re doing in such a way that’s easy for board vendors to actually produce. I’ve never had a manufacturing defect on any of the boards I’ve gotten from the low-cost vendors (Seeed/Elecrow/iTead), but I use Mentor’s PCB tools (through work, yay) which are obviously significantly higher cost.

      1. Eagle does a good job of following whatever design rules you tell it to use, and checking aganst them as requested. Two opportunities for operator negligence. Many board houses (Seeed, Itead, Elecrow) have Eagle DRU and CAM files to simplify gerber generation for their capabilities. SparkFun also offer their own design rules, with a nice tutorial:

        https://www.sparkfun.com/tutorials/115

        I’ve found the default Eagle design rules are often below the limits of the cheap board houses. So I start with the design rules from the manufacturer, and adjust upward until I have the most conservative settings for trace width/spacing, restring, etc. that still accomodate the requirements of my design. Then I run DRC at the end and review any clearance conflicts. That way there’s some room for success if the board house does a bad job.

        I’ve not used KiCad, but I’ll bet similar care over the design rules will also give good results.

        1. There’s a difference between something passing DRC and something being easily buildable, with plenty of margin. And that might depend on exactly *how* polygons are rasterized, how flexible the pad flash shapes are, etc. I’m not saying there is actually a difference, but it’s definitely possible.

        2. I like to find out what the board house can do on non-critical jobs that I can fix easily. So next time when I need to push them I know what they can do and what to avoid. For production stuff, I would go as conservative as possible to get the job done to optimize for yield. I found out that the board shop now has soldermask between fine pitch part as well as Chip Scale BGA that they previously didn’t. I also pushed myself too to find out I can prototype with the smaller packages.

          Without knowing the boundary, you are not taking full capability of the service.

      1. In order to increase output, it is generally important to enable the operator so they are well grounded. Depending on the desired output, electroshock therapy may work. Clearly I’m being productive at work.

  10. In my career as a PCB design I have seen a few things from board houses that were disasters. Boards that have had blisters, layers swapped (on an RF board), plating flaking off, holes not drilled, and plating that solder will not stick to. Almost all of these were from companies that were used to save a little money over our trusted board house.

    1st) All board houses make mistakes. The good ones make fewer mistakes and when they do they will compensate or make it up to you (cut a deal).

    2nd) Don’t expect quality from any place that just bids out the job. This is more important when dealing with boards from China as the people who make the boards don’t lose face when the boards are bad.

    3rd) just because it passes electrical test doesn’t mean it will survive assembly and reflow not to mention actual use in the field.

    4th) There is no substitute for inspection of boards before you put parts on them.

  11. As a preventive measure I suggest keeping more clearance between tracks/pads and copper pour. In this particular example it looks like the clearance was set to 10 mil … a bit too low. I normally put it at 14 mil for the copper pour.
    Same goes to the trace wight – make them wider. 14 mil seems to be a good value for a through hole or SOIC packages. Or even more, especially for your power rails.

    1. Word. When not pressed, I’ll do 12/12 traces/separation with 20 mil isolation on the plane fills. Why? I can toner-transfer this at home with ease, so I can quickly build a test using the actual files before ordering a large batch. With fat clearances and a known-good design, there’s not much left to go wrong.

      Down below 10/10, my homebrew boards get sketchy, and it’s off to the fab house without the safety proof, which always makes me slightly nervous, but if it passes DRC it’s gotta work every time, right?!?

      I’m not sure why you’d ever do copper pour isolation less than 20, really. If you need to reduce isolation to 10 mil to get the plane to hook up, that means that your plane polygon is only held together with 10 mil threads, and that probably should be a warning sign anyway. IMO, it’s better to patch the plane back together with some intentional, thick traces than to rely on plane pours with thin fingers.

  12. E-testing is just testing whether all the pad of a net are connected. AFAIK It will not test for short between different nets.

    To prevent this kind of errors is to use more clearance between traces and traces and poures. It is general a good idea not to use the absolute maximum capabilities of the board house, esp. the cheap chinese ones :D

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s