Fail of the Week: NASA Edition

There’s a reason we often use the phrase “It ain’t Rocket Science”. Because real rocket science IS difficult. It is dangerous and complicated, and a lot of things can and do go wrong, often with disastrous consequences. It is imperative that the lessons learned from past failures must be documented and disseminated to prevent future mishaps. This is much easier said than done. There’s a large number of agencies and laboratories working on multiple projects over long periods of time. Which is why NASA has set up NASA Lessons Learned — a central, online database of issues documented by contributors from within NASA as well as other organizations.

The system is managed by a steering committee consisting of members from all NASA centers. Public access is limited to a summary of the original driving event, lessons learned and recommendations. But even this information can be quite useful for common folks. For example, this lesson on Guidance for NASA Selection & Application of DC-DC Converters contains several bits of useful wisdom. Or this one about IC’s being damaged due to capacitor residual discharge during assembly. If you ever need to add a conformal coating to your hardware, check how Glass Cased Components Fractured as a Result of Shrinkage in Coating/Bonding Materials Applied in Excessive Amounts. Finally, something we have all experienced when working with polarized components — Reverse Polarity Concerns With Tantalum Capacitors. Here is a more specific Technical Note on polarized capacitors (pdf): Preventing Incorrect Installation of Polarized Capacitors.

Unfortunately, all of this body of past knowledge is sometimes still not enough to prevent problems. Case in point is a recently discovered issue on the ISS — a completely avoidable power supply mistake. Science payloads attach to the ISS via holders called the ExPRESS logistics carriers. These provide mechanical anchoring, electrical power and data links. Inside the carriers, the power supply meant to supply 28V to the payloads was found to have a few capacitors mounted the other way around. This has forced the payloads to use the 120V supply instead, requiring them to have an additional 120V to 28V converter retrofit. This means modifying the existing hardware and factoring in additional weight, volume, heat, cost and other issues when adding the extra converter. If you’d like to dig into the details, check out this article about NASA’s power supply fail.

Thanks to [Jarek] for tipping us about this.

OWL Insecure Internet of Energy Monitors

[Chet] bought an electricity monitor from OWL, specifically because it was open and easy to hack on at him within the confines of his home network. Yay! Unfortunately, it also appears to be easy to hack read outside of his home network too, due to what appears to be extraordinarily sloppy security practices.

The short version of the security vulnerability is that the OWL energy monitors seem to be sending out their data to servers at OWL, and this data is then accessible over plain HTTP (not HTTPS) and with the following API: http://beta.owlintuition.com/api/electricity/history_overview.php?user=&nowl=&clientdate=. Not so bad, right? They are requiring username and password, plus the ID number of the device. Maybe someone could intercept your request and read your meter remotely, because it’s not encrypting the transaction?

Nope. Much worse. [Chet] discovered that the username and password fields appear not to be checked, and the ID number is the device’s MAC address which makes is very easy to guess at other device IDs. [Chet] tried 256 MACs out, and got 122 responses with valid data. Oh my!

Take this as a friendly reminder and a cautionary tale. If you’re running any IoT devices, it’s probably worth listening to what they’re saying and noting to whom they’re saying it, because every time you send your data off to “the cloud” you’re trusting someone else to have done their homework. It is not a given that they will have.

Fail Of The Week: How I Killed The HiPot Tester

Have you ever wired up a piece of test equipment to a circuit or piece of equipment on your bench, only to have the dreaded magic smoke emerge when you apply power? [Steaky] did, and unfortunately for him the smoke was coming not from his circuit being tested but from a £2300 Clare H101 HiPot tester. His write-up details his search for the culprit, then looks at how the manufacturer might have protected the instrument.

[Steaky]’s employer uses the HiPot tester to check that adjacent circuits are adequately isolated from each other. A high voltage is put between the two circuits, and the leakage current between them is measured. A variety of tests are conducted on the same piece of equipment, and the task in hand was to produce a new version of a switch box with software control to swap between the different tests.

This particular instrument has a guard circuit, a pair of contacts that have to be closed before it will proceed. So the switch box incorporated a relay to close them, and wiring was made to connect to the guard socket. At first it was thought that the circuit might run at mains voltage, but when it was discovered to be only 5V the decision was made to use a 3.5mm jack on the switch box. Inadvertently this was left with its sleeve earthed, which had the effect of shorting out a DC to DC converter in the HiPot tester and releasing the smoke. Fortunately then the converter could be replaced and the machine brought back to life, but it left questions about the design of the internal circuit. What was in effect a logic level sense line was in fact connected to a low current power supply, and even the most rudimentary of protection circuitry could have saved the day.

We all stand warned to be vigilant for this kind of problem, and kudos to [Steaky] for both owning up to this particular fail and writing such a good analysis of it.

Our Fail Of The Week series has plenty to entertain the reader who is not of a nervous disposition. This isn’t the first fail to feature a suspect bit of connector wiring, not an unexpected short or even some magic smoke.

I Think I Failed. Yes, I Failed.

Down the rabbit hole you go.

In my particular case I am testing a new output matching transformer design for an audio preamplifier and using one of my go to driver circuit designs. Very stable, and very reliable. Wack it together and off you go to test and measurement land without a care in the world. This particular transformer is designed to be driven with a  class A amplifier operating at 48 volts in a pro audio setting where you turn the knobs with your pinky in the air sort of thing. Extra points if you can find some sort of long out of production parts to throw in there for audiophile cred, and I want some of that.

img_2857-2Lets use some cool retro transistors! I merrily go along for hours designing away. Carefully balancing the current of the long tailed pair input. Picking just the right collector power resistor and capacitor value to drive the transformer. Calculating the negative feedback circuit for proper low frequency cutoff and high frequency stability, and into the breadboard the parts go — jumper clips, meter probes, and test leads abound — a truly joyful event.

All of the voltages check out, frequency response is what you would expect, and a slight tweak to the feedback look brought everything right into happiness. Time to fire up the trusty old HP 334A Distortion Analyzer. Those old machines require you to calibrate the input circuit and the volt meter, tune a filter to the fundamental frequency you are applying to the device under test and step down to lower and lower orders of distortion levels until the meter happily sits somewhere in the middle of a range.

Most modern circuits in even cheap products just go right down to sub .1% total harmonic distortion levels without even a thought and I expected this to be much the same. The look of horror must have been pronounced on my face when the distortion level of my precious circuit was something more akin to a clock radio! A frantic search began. Was it a bad jumper, or a dirty lead in the breadboard, or an unseated component? Was my function generator in some state of disrepair? Is the Stephen King story Maximum Overdrive coming true and my bench is going to eat me alive? All distinct possibilities in this state of panic.

Continue reading “I Think I Failed. Yes, I Failed.”

Life on Contract: How to Fail at Contracting Regardless of Skill

I believe higher quality learning happens from sharing failure than from sharing stories of success. If you have set your mind to living on contract, I present this cheat sheet of some of the most simple and effective ways to muck it all up that have surprisingly little or nothing to do with your technical skill, knowledge, or even deliverables.

The previous installment of Life on Contract discussed how one might find clients as an engineering contractor or consultant while also taking a bit of time to pull apart the idea of whether life on contract is appropriate as opposed to, for example, bootstrapping a business instead. Assuming you are set on working as a contractor, let’s talk about what happens after you have found a prospective client (or perhaps more likely: after they have found you.)

WARNING: this article features an utter lack of success tips and tricks. Partly because those can be found in any seminar or business self-help book, but mostly because I do not have a foolproof recipe for success, and cheat codes to unlock easy mode still elude me. But I have witnessed (or committed) and reflected on many excellent ways to fail at contracting; or at the very least succeed in not being invited back.

Just because I won’t be sharing success stories doesn’t mean success has no learning value. Got a success story, or a better way to fail? Tell us about it in the comments!

Continue reading “Life on Contract: How to Fail at Contracting Regardless of Skill”

UK IT Specialist Unable to Boil Water, Make Tea

In our latest episode of “IoT-Schadenfreude Theater” we bring you the story of [Mark], a British man who can’t boil water. Or more specifically, a man who can’t integrate MQTT with Amazon Echo, or IFTTT with HomeKit.

Yes, yes. We all love to laugh at a technology in its infancy. It’s like when robots fall down: it’s a cheap shot and things will surely get better, right? Indeed, the Guardian has had its fun with this particular WiFi kettle before — they’re British and nothing is more important than a remote-controlled cuppa.

Every time we hear about one walled-garden protocol not speaking to another, and the resulting configuration mayhem that ensues, we can’t help think that [Mike] was right: home automation has a software problem. But that’s putting the blame on the technology. (We’re sure that [Mark] could have made the kettle work if he’d just applied a little Wireshark.)

Strongbad's VCR
Strongbad’s VCR

There’s another mismatch here — one of expectations about the users. A water kettle is an object that should be usable by grandmothers, and a complex networked device is clearly aimed at techies and early adopters. Combining the two is asking for trouble. Non-functioning IoT devices are the blinking 12:00 of our generation.

What do you think? Where’s the blame here? Poor design, bad software stack, stupid users, or failure of mega-corps to integrate their systems together? More importantly, how could we make it better?

Headline image:Fredy Velásquez Orozco, via Wikimedia Commons Thumbnail image: Markus Schweiss, also Wikimedia Commons.

How to Fail at Laser Cutting

Laser cutters are CNC power tools, which means an operator uploads a job digitally and then pushes START to let the machine do all the work while they lie back in a hammock sipping a margarita, occasionally leaping out in a panic because the sound coming from the machine changed slightly.

Like other power tools, laser cutters are built around doing one thing very well, but they require an operator’s full attention and support. The operator needs to handle all the other things that go on before, during, and after the job. It’s not too hard to get adequate results, but to get truly professional and repeatable ones takes work and experience and an attention to detail.

People often focus on success stories, but learning from failures is much more educational. In the spirit of exploring that idea, here are my favorite ways to fail at laser cutting and engraving. Not all of these are my own personal experience, but they are all someone’s personal experience.

Continue reading “How to Fail at Laser Cutting”