Penn and Teller once had a show about “What is the best?” Engineers know that’s not a complete question. Think about a car. What makes the “best” car? It depends on why you want a car. For a race car driver, it might be that speed is the most important factor. A mom might value safety. Someone who commutes four hours a day might like a car that’s comfortable. A teenager wants something affordable.

If you think about it, though, it is even more complicated than that. For example, just about everyone wants a car that is safe. Reliability is pretty important, too. So the reality is, most people want a car that has multiple attributes. Worse still, they sometimes conflict; making one better will make some other ones worse. Mom wants a safe car, but not one that takes half a day to drive to the corner market. Nor does she want to pay a half million dollars for a safe car.

To the Moon

The more complex your system, the more considerations you have to deal with, and the more they affect each other. Tough design trades are what drive the most creative moments in engineering. For example, consider the Lunar Excursion Module (LEM) that carried astronauts from lunar orbit to the moon’s surface. There had never been anything quite like it, and engineers at Grumman had many challenges getting it to a working state.

The model on the left is what people thought the LEM might look like in 1963. You might notice the four relatively large windows. Earlier artist’s renditions had even more glass, looking almost like a helicopter’s cockpit with intrepid astronauts sitting in front of high-tech instrument panels. The original concept vehicle had about 24 square feet of glass.

There are at least two problems with this picture. First, astronauts in space suits need a lot of room to sit down. The seats are also heavy, and every gram of mass on a spacecraft costs a hundred grams in fuel to get to the moon. The other problem is the glass. To hold the air pressure in and provide enough structure, you’d need very thick glass. And glass is surprisingly heavy. Besides that, glass admits a lot of solar radiation.

So the engineers kept shrinking the windows, but this required the seats to get closer to the window and tilt differently so that the astronauts could see out. But if the seats get too close to these smaller windows, it gets harder to maneuver into them while wearing a bulky 1960’s space suit.

There are quite a few design trades going here. The astronauts had to see to land. Remember that video technology wasn’t what it is today, and neither was electronic reliability. So while today you might put no windows in and rely on some redundant cameras, that wasn’t a reasonable option in those days. Some of the other trades involved weight, maintaining a pressurized environment, keeping the astronauts from bouncing around inside the vehicle, and managing radiation exposure.

As the designed windows got smaller (in the end, they’d wind up with about 3 square feet of glass, down from 24), someone realized that in near-zero G, maybe the astronauts didn’t need to sit at all. If they stood up, they could all but press their noses to the glass and get excellent visibility. It took a little work to ensure they’d be safe standing during the landing, but it all worked out and the final LEM (see right) used the small windows and standing astronauts.

The net result was 90 pounds of weight savings, and a wider cone of vision for landing than most helicopters had at the time. By the way, it is hard to get scale from these pictures, but if you ever see a LEM in person, you’ll probably find it is larger than you think. The claustrophobic CM (command module) was much smaller for three astronauts, but it had different own design trades.

Making a Trade

It turns out that it’s just hard to make a decision with dozens of different factors, even when it isn’t an engineering problem. That’s why business schools teach operations research and multi-criteria decision making. There are several methods that attempt to convert a subjective gut feel approach into a rational mathematical approach.

I’m always suspicious of these processes because they all depend on the humans assigning values in the first place. If you aren’t paying attention, these methods make it look like the decision process is purely scientific. But as the old computer proverb says: Garbage in, garbage out. The fact that you guess at the input values to the algorithm mean that the output is a guess too.

Regardless, if you keep that in mind and try to be fair, some of these methods can be a useful way to organize your thinking. Just remember, you can usually game the numbers to get any answer you want out of these tools. Let’s take one for a spin.

Suppose you want to buy a cell phone. There are lots of choices, so you write down the criteria to make the decision. For example (in no particular order): price, weight, battery life, screen size, network type, front camera, rear camera, and storage. Of course, the assumption is that you’ll buy one phone, unlike the cab driver on the right who, apparently, couldn’t make up his mind.

Your list might be different, of course. One difficulty here is making sure these items are actually comparable. For example, CPU speed isn’t on my list. A phone with two brand X cores running at 1 GHz might be slower than a phone with four brand Y cores running at 900 MHz (or vice versa). So selecting on cores or clock speed probably doesn’t mean much unless all the choices have the same kind of CPU.

Working the List

Once you have your list, you need to assign weights to each one. Some criteria will be so important that they must be met, or a candidate is out. These are gating criteria. For example, I might need a phone that takes a GSM card, so any phones that have a network type of CDMA are out no matter what other positive attributes they might have.

Some gates are more conditional. It isn’t uncommon to have a budget. So phones with prices over $400 might be off the table. The other criteria get weights. You can do this in different ways, but the most common method sets weights so that they all add up to 1. Here’s how my table might look on a first pass:

It is important to realize that the ratings for each phone (where the dashes are in the table above) aren’t the actual values. They are a relative rating (usually on a 1 to 10 scale). So the price for a $400 phone isn’t 400. It depends on how good the price is relative to the other alternatives. So if the Hackaphone is $400 and the other two phones are $495, you might assign a nine to the Hackaphone (since you wish it were cheaper) and five to the others. On the other hand, if the other two phones were $200, the Hackaphone might get a 2.

You could try to do this mathematically by determining the percentage of the budget that is left over and scaling it so the minimum score is 1. For example:

Int(10*(1-price/budget))+1  or    Int(10*(1-400/500))+1 = 3

That would leave the $495 phones rated at 1. You could also have computed the percent of the budget and assigned a negative weight. However, it isn’t always possible to mathematically scale things into numerical grades. For example, the reliability of a vendor is going to have to be a subjective number.

Once you fill in the grid with the scores, you multiply each entry by the row weight and then add up the columns. The highest score wins. Keep in mind, though, that since this is somewhat subjective, that a score of 78 and 79 might be a tie even though technically 79 is the winner.

Gaming the System

You can probably see how you could manipulate the system. If you wanted to be sure to get an iPhone, you could make FaceTime a gating criterion. Maybe it really is necessary, of course, but it could be that Skype or Google Hangouts would meet your needs just as well. Conversely, I could exclude the iPhone if I made it mandatory to have a micro USB charging port.

Even without gating criteria, it is easy to weight the scores to favor our gut-feeling choice, even if subconsciously. If I know my favorite phone has a high-resolution rear camera, and I weight that very heavily, it increases the odds my favorite camera will win. Unless your primary task is to take great pictures, that probably leads to a bad result. Again, garbage in, garbage out.

Still, if you need to pick between a bunch of transistors, a slew of op amps, or a raft of wireless boards, going through the above process at least organizes your thoughts and forces you to consider what is most important. Just don’t fall into the trap of thinking that it is infallible.

Back to the Moon

The U.S. space program in the 1960s was nothing short of amazing. The U.S. didn’t even launch a satellite until 1958. Alan Sheppard did a suborbital flight in 1961, and John Glenn did a few orbits in 1962. By 1969, two men walked on the surface of the moon. That was a lot of engineering in the space of about ten years. A lot of what they did, we might today consider hacks.

I’ve heard the stories about the LEM design for many years, but I found the best reference to them in a paper published by [Robert Smythe] at Grumman. There are quite a few lessons you can pick up from reading about what it took to get from nothing to a moon walk in ten years. You can find that and more in [David Mindell’s] Digital Apollo book, which is a great read. Even if you don’t want to read the book, [Mindell’s] sources are interesting reading. Another good source is NASA’s history pages.

The more you read about cases like the LEM, the more apparent it is that engineering is both a science and an art. Structured methods can help you think about a problem. But sometimes getting the best design takes a leap of creativity that you can’t encapsulate in a table of numbers or a math equation. You can use tools to make sure you cover all your bases or to help define the problem. You might need math or testing to prove out that crazy design idea (just like the LEM engineers tested the safety of the standing position). But the crazy–and often best–ideas don’t fall out of a process. They come from creative inspiration.

Cell phone taxi photo by Wesley Fryer (C BY-SA 2.0)