Cover Your Glass: A Lesson In Design Trades

Penn and Teller once had a show about “What is the best?” Engineers know that’s not a complete question. Think about a car. What makes the “best” car? It depends on why you want a car. For a race car driver, it might be that speed is the most important factor. A mom might value safety. Someone who commutes four hours a day might like a car that’s comfortable. A teenager wants something affordable.

If you think about it, though, it is even more complicated than that. For example, just about everyone wants a car that is safe. Reliability is pretty important, too. So the reality is, most people want a car that has multiple attributes. Worse still, they sometimes conflict; making one better will make some other ones worse. Mom wants a safe car, but not one that takes half a day to drive to the corner market. Nor does she want to pay a half million dollars for a safe car.

To the Moon

The more complex your system, the more considerations you have to deal with, and the more they affect each other. Tough design trades are what drive the most creative moments in engineering. For example, consider the Lunar Excursion Module (LEM) that carried astronauts from lunar orbit to the moon’s surface. There had never been anything quite like it, and engineers at Grumman had many challenges getting it to a working state.

lemThe model on the left is what people thought the LEM might look like in 1963. You might notice the four relatively large windows. Earlier artist’s renditions had even more glass, looking almost like a helicopter’s cockpit with intrepid astronauts sitting in front of high-tech instrument panels. The original concept vehicle had about 24 square feet of glass.

There are at least two problems with this picture. First, astronauts in space suits need a lot of room to sit down. The seats are also heavy, and every gram of mass on a spacecraft costs a hundred grams in fuel to get to the moon. The other problem is the glass. To hold the air pressure in and provide enough structure, you’d need very thick glass. And glass is surprisingly heavy. Besides that, glass admits a lot of solar radiation.

So the engineers kept shrinking the windows, but this required the seats to get closer to the window and tilt differently so that the astronauts could see out. But if the seats get too close to these smaller windows, it gets harder to maneuver into them while wearing a bulky 1960’s space suit.

There are quite a few design trades going here. The astronauts had to see to land. Remember that video technology wasn’t what it is today, and neither was electronic reliability. So while today you might put no windows in and rely on some redundant cameras, that wasn’t a reasonable option in those days. Some of the other trades involved weight, maintaining a pressurized environment, keeping the astronauts from bouncing around inside the vehicle, and managing radiation exposure.

apollo_lemAs the designed windows got smaller (in the end, they’d wind up with about 3 square feet of glass, down from 24), someone realized that in near-zero G, maybe the astronauts didn’t need to sit at all. If they stood up, they could all but press their noses to the glass and get excellent visibility. It took a little work to ensure they’d be safe standing during the landing, but it all worked out and the final LEM (see right) used the small windows and standing astronauts.

The net result was 90 pounds of weight savings, and a wider cone of vision for landing than most helicopters had at the time. By the way, it is hard to get scale from these pictures, but if you ever see a LEM in person, you’ll probably find it is larger than you think. The claustrophobic CM (command module) was much smaller for three astronauts, but it had different own design trades.

Making a Trade

It turns out that it’s just hard to make a decision with dozens of different factors, even when it isn’t an engineering problem. That’s why business schools teach operations research and multi-criteria decision making. There are several methods that attempt to convert a subjective gut feel approach into a rational mathematical approach.

I’m always suspicious of these processes because they all depend on the humans assigning values in the first place. If you aren’t paying attention, these methods make it look like the decision process is purely scientific. But as the old computer proverb says: Garbage in, garbage out. The fact that you guess at the input values to the algorithm mean that the output is a guess too.

Regardless, if you keep that in mind and try to be fair, some of these methods can be a useful way to organize your thinking. Just remember, you can usually game the numbers to get any answer you want out of these tools. Let’s take one for a spin.

cellSuppose you want to buy a cell phone. There are lots of choices, so you write down the criteria to make the decision. For example (in no particular order): price, weight, battery life, screen size, network type, front camera, rear camera, and storage. Of course, the assumption is that you’ll buy one phone, unlike the cab driver on the right who, apparently, couldn’t make up his mind.

Your list might be different, of course. One difficulty here is making sure these items are actually comparable. For example, CPU speed isn’t on my list. A phone with two brand X cores running at 1 GHz might be slower than a phone with four brand Y cores running at 900 MHz (or vice versa). So selecting on cores or clock speed probably doesn’t mean much unless all the choices have the same kind of CPU.

Working the List

Once you have your list, you need to assign weights to each one. Some criteria will be so important that they must be met, or a candidate is out. These are gating criteria. For example, I might need a phone that takes a GSM card, so any phones that have a network type of CDMA are out no matter what other positive attributes they might have.

Some gates are more conditional. It isn’t uncommon to have a budget. So phones with prices over $400 might be off the table. The other criteria get weights. You can do this in different ways, but the most common method sets weights so that they all add up to 1. Here’s how my table might look on a first pass:

Criterion Weight Hackaphone Makeaphone Orange jPhone Notes
Price 0.15 Must be < $500; less is better
Weight 0.1 Lighter is better
Battery 0.25 Rate against tested run time
Screen 0.2 Larger is better
Network N/A Must be GSM
Front Cam 0.1 More MP better
Rear Cam 0.1 More MP better
Storage 0.1 More is better

It is important to realize that the ratings for each phone (where the dashes are in the table above) aren’t the actual values. They are a relative rating (usually on a 1 to 10 scale). So the price for a $400 phone isn’t 400. It depends on how good the price is relative to the other alternatives. So if the Hackaphone is $400 and the other two phones are $495, you might assign a nine to the Hackaphone (since you wish it were cheaper) and five to the others. On the other hand, if the other two phones were $200, the Hackaphone might get a 2.

You could try to do this mathematically by determining the percentage of the budget that is left over and scaling it so the minimum score is 1. For example:

Int(10*(1-price/budget))+1  or    Int(10*(1-400/500))+1 = 3

That would leave the $495 phones rated at 1. You could also have computed the percent of the budget and assigned a negative weight. However, it isn’t always possible to mathematically scale things into numerical grades. For example, the reliability of a vendor is going to have to be a subjective number.

Once you fill in the grid with the scores, you multiply each entry by the row weight and then add up the columns. The highest score wins. Keep in mind, though, that since this is somewhat subjective, that a score of 78 and 79 might be a tie even though technically 79 is the winner.

Gaming the System

You can probably see how you could manipulate the system. If you wanted to be sure to get an iPhone, you could make FaceTime a gating criterion. Maybe it really is necessary, of course, but it could be that Skype or Google Hangouts would meet your needs just as well. Conversely, I could exclude the iPhone if I made it mandatory to have a micro USB charging port.

Even without gating criteria, it is easy to weight the scores to favor our gut-feeling choice, even if subconsciously. If I know my favorite phone has a high-resolution rear camera, and I weight that very heavily, it increases the odds my favorite camera will win. Unless your primary task is to take great pictures, that probably leads to a bad result. Again, garbage in, garbage out.

Still, if you need to pick between a bunch of transistors, a slew of op amps, or a raft of wireless boards, going through the above process at least organizes your thoughts and forces you to consider what is most important. Just don’t fall into the trap of thinking that it is infallible.

Back to the Moon

lunarThe U.S. space program in the 1960s was nothing short of amazing. The U.S. didn’t even launch a satellite until 1958. Alan Sheppard did a suborbital flight in 1961, and John Glenn did a few orbits in 1962. By 1969, two men walked on the surface of the moon. That was a lot of engineering in the space of about ten years. A lot of what they did, we might today consider hacks.

I’ve heard the stories about the LEM design for many years, but I found the best reference to them in a paper published by [Robert Smythe] at Grumman. There are quite a few lessons you can pick up from reading about what it took to get from nothing to a moon walk in ten years. You can find that and more in [David Mindell’s] Digital Apollo book, which is a great read. Even if you don’t want to read the book, [Mindell’s] sources are interesting reading. Another good source is NASA’s history pages.

The more you read about cases like the LEM, the more apparent it is that engineering is both a science and an art. Structured methods can help you think about a problem. But sometimes getting the best design takes a leap of creativity that you can’t encapsulate in a table of numbers or a math equation. You can use tools to make sure you cover all your bases or to help define the problem. You might need math or testing to prove out that crazy design idea (just like the LEM engineers tested the safety of the standing position). But the crazy–and often best–ideas don’t fall out of a process. They come from creative inspiration.

Cell phone taxi photo by Wesley Fryer (C BY-SA 2.0)

25 thoughts on “Cover Your Glass: A Lesson In Design Trades

  1. The movie Apollo 13 made the engineers at Grumman into people more interested in there jobs than the lives in jeopardy.
    What a crock that movie was. I can attest to the fact engineers for Grumman were in fact a major factor it went as well as it did.

  2. “But the crazy–and often best–ideas don’t fall out of a process. They come from creative inspiration” This is a key concept that has all but been lost over the last twenty years or so as process-worship began to infect the act of design. I understand the reasons – an attempt to keep costs down by limiting the number of failure, but in doing so almost a whole generation of young engineers were forced into a three-ring binder mentality at what should have been the most creative point in their careers. Not all companies or sectors succumbed but enough did that North America lost much of its edge in this regard.

      1. I would place it even earlier, circa 1982…
        High interest rates, and the IRS decided to charge taxes on inventory… because of the latter, many companies destroyed inventories of spare parts which sped obsolescence.

    1. Process-worship is a good name for it.

      It’s kind of funny, really. Job specialization is a huge part of what makes civilization possible. To accomplish anything interesting you need to get various different breeds of specialist together, and organize them in some way. Sometimes involving someone who’s specialized in designing organizational processes can be a tremendous boon.

      Sometimes, though, it all goes crazy. Folks lose sight of the big picture (or aren’t wired to see it in the first place) and you get bizarre cargo cult nonsense. This gets comically, sometimes disastrously exacerbated when some enterprising vulture adopts a particular process as a product to market to managers. If you haven’t experienced being on a software dev team that’s been sold on but not actually trained in or taught Agile, it’s quite a spectacle.

      Bonus points if the folks in charge are the sort who smugly believe themselves immune to marketing or above complex social/political dynamics.

  3. The US space program in the 1960s is a great example of what happens when engineers are left alone to accomplish their goal with a minimum of bureaucratic oversight and a large budget. The space program of the 1970s and later are what happens when those two get reversed.
    Just imagine what we could have accomplished by now…

  4. Phew i was worried about being the only who does extensive operational research before buying anything that cost 100 or 100 in recurring expenses. Anyway i do my math direcly on excel, and use sorted indexes to get the best. When comparing apple to oranges i use money as weight, as i ask myself “i’d be happier with an apple and 10€ or an orange?” And lower the money until the answer is “pff, whatever”.
    Phones are even more complicated, as i can repair them, so the choice is extended to used broken ones+repair cost.

  5. Another interesting thing about the LEM design process was they wanted to have two docking ports. One on top to connect to the CM for the coast phase to the Moon, and one on the front so it would be easier for the LEM pilot to see for docking on lunar orbit. That second port would add a lot of weight and be difficult to use as a door on the lunar surface.

    Then someone got the idea to replace the front docking port with a simple square door, which would be easier to get through with the bulky life support packs. That left the problem of being able to see for docking. The docking after TLI was the CSM maneuvering to link to the LEM atop the 3rd stage where the pilot could see the LEM. The solution was to put a window in the top of the LEM above the pilot station. That was after they’d gone through the process of shrinking the forward windows down. To make that top window work took a lot of rework of stuff for air pressure, thermal load, structural alterations etc. One tiny window addition led to a large amount of design changes, but it was worth it to make the LEM lighter.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.