If there was one book that describes what it means to be in the trenches of a cutting edge design, that book is The Soul Of a New Machine. Tracy Kidder’s Pulitzer prize-winning book has been an inspiration to thousands over the years.
Soul is the story of the creation of the Data General Eclipse MV/8000, code-named Eagle. Eagle was Data General’s first 32-bit minicomputer. If you’re not a retrocomputing aficionado, minicomputers were a major industry back in the 70’s and 80’s. Starting in 1964 with the Digital Equipment Corporation (DEC) PDP-8, minis provided a low-cost means for companies to get a computer. The only other option was a huge mainframe from companies like IBM. Minicomputers chugged along until the 1990s when microprocessor-based PCs and workstations passed them by. The market, and the industry evaporated.
Today, more than 30 years later, minicomputers are all but forgotten. Data General itself is long gone, purchased by EMC in 1999. DG’s mark on the landscape has all but been erased by the swiftly moving sands of technical progress. All except for the snapshot Kidder set down in Soul.
The technical side of designing a new computer is just one part of this book. The Soul of a New Machine is three stories: the story of the engineers, the story of the managers, and the story of the machine they built. For this reason, the book has found itself on the reading list of engineering schools and management institutes alike.
The thing that makes this book appeal to the masses is Kidder’s uncanny ability to explain incredibly complex topics in layman’s terms. He manages to explain the inner workings of a 32-bit CPU, all the way down to the level of microcode. He delves into Programmable Array Logic (PALs), forerunners of the CPLD and FPGA devices you read about on our pages today. PALs were a hot new technology back in the late 70’s. They allowed the Eagle team to make changes quickly — without pulling out their wire wrapping tools.
Kidder manages to explain these things in a way that doesn’t leave the average Joe scratching their head, yet doesn’t bore the technically savvy. If he ever decides to stop writing non-fiction, Tracy Kidder would have a career writing user manuals.
The Soul of a New Machine starts in a very unlikely place – on the deck of a sailing ship during a rough storm. The scene is our introduction to the star of the book – Tom West, a manager at Data General. West is multifaceted and enigmatic to say the least. A folk guitarist who was inspired to work on electronics by the Apollo program. He was a few years too late for NASA though. Eventually he found himself travelling the world building and adjusting incredibly accurate clocks at astronomical observatories for the Smithsonian. This meandering path eventually led him to DG, where he was hired as a computer engineer and quickly worked his way up the ranks.
Data General was already working on a 32-bit computer at this point. The company’s best and brightest engineers had started on “Fountainhead”, which was a no holds barred next generation machine. Disputes over taxes between Data General and the Massachusetts government caused the company to move the entire FHP team to North Carolina’s RTP. West headed up the group which designed Eclipse computers – the bread and butter 16-bit minicomputers which kept the lights on at Data General. This set the stage for a bitter inter-company rivalry, as well as fierce competition for resources. Several of West’s programs were cancelled before they ever got out the door. To put it bluntly, he was pissed.
West went to CEO Edson de Castro and asked what he wanted. The answer was simple: A 32-bit Eclipse without a mode bit.
Of Men and Mode bits
Any company has to deal with old products. In the computing world, the term is legacy. In the computing industry, it’s a particularly nasty problem. New generations of a computer usually need to be compatible with the older ones. Breaking compatibility would mean all the customers software has to be ported — a long and arduous job that opens the door for a competitor to swoop in.
How does a designer add new features, yet maintain compatibility with older software? The answer is often a mode bit. In “legacy mode” the new machine acts exactly like the old generation. Switch to “modern mode” and the true power of the new computer is unlocked. The problem is that future generation machines will need a different mode bit for compatibility with every generation that came before it. A (simplified) example of this is Intel’s x86 architecture. The 286 had to be compatible with the 8086, so protected mode was created. Fast forward to x86-64, and we’re up to 7 modes, switching between 16, 32, and 64-bit systems.
West couldn’t build a computer on his own. He pulled in lieutenants Carl Alsing and Ed Rasala. Together they recruited a team of 20-somethings, fresh out of college. The engineers fell into two groups – The Hardy Boys, who designed the hardware, and the microkids, who wrote the microcode.
Everyone on the project performed an informal ritual of “signing up” — a term coined by West. An engineer signing up for a project agrees to go above and beyond the call of duty to ensure the project succeeds. Saturdays, Sundays, 90+ hour weeks, whatever it took. All this came at the cost of the engineer’s personal life. Getting fresh college grads to sign up was easy.
West wasn’t a friendly manager. He didn’t say hello, or even acknowledge his new recruits in the hallways. One even nicknamed him “the prince of darkness”. It was all a form of mushroom management and clever (if immoral) manipulation to get the most out of the team. On the other hand, West insulated his team from the distractions of upper management. They toiled away in a basement hidden from view.
It’s with West and these engineers that Kidder spends much of the book. He became embedded, visiting the workers’ houses, meeting their families, looking at the hobbies and artwork they spent time on before the Eagle project. Learning their hopes, dreams, and what drove them to work as hard as they did.
This is after all, a book about history, so stop here if you don’t want any spoilers. Suffice it to say the team eventually did get the Eagle out the door. In a political move, West was shipped off to Data General Japan. The Eagle was broken up with many of them leaving the company to find new jobs. If any of them were expecting accolades and stock options from the company, they were slow in coming. Wired magazine checked in on the engineers back in 2000. Now, as the 20-somethings of 1978 approach retirement, you can find them on LinkedIn and other social media sites.
Once the book went to print, the team, especially West, became minor celebrities in the computer industry. Data General used this to their advantage, trotting West out whenever they could to sign books and promote new designs. He wasn’t exactly happy about that, but he stayed with the company until 1998, when he retired. West passed on in 2011. His obituary made the New York Times. His daughter, Jessamyn West, an accomplished librarian and MetaFilter staffer, posted a link to his Flickr page. While it doesn’t contain many photos of the Data General Eagle project, It does have shots of Tom’s fabled home workshop.
The story of the Eagle isn’t unique, in fact, nearly every company has a story that the staff reminisces about. One about a project where people put in crazy hours, and got the work done against all odds. The difference is that this project was put down in words everyone can understand and relate to. Tracy Kidder’s abilities can’t be overstated here.
The Soul of A New Machine had an impact on engineers at the time, and in the decades since. I found one example talking to the Hackaday writing staff: An engineer working on the Commodore 128 was told to remove the 128/64 mode bit. It almost worked, except for a bug in the font ROM which was brought to light by the popular koala paint program. [Bil Herd], lead hardware designer on the 128, had read the book too. He decided to keep a pseudo-mode bit handy, essentially a summation of all the things that were switched on in 64 mode. [Bil’s] insurance policy paid off with this bug. The 128/64 mode bit was used to switch between the original C64 font ROMs and the extended 128 ROM.
“If he ever decided to stop writing non-fiction, he’d have a career writing user manuals.”
Sick tech-pubs burn, Hackaday.
I must say I balked when editing this as well. But re-read what Adam wrote at the beginning of the paragraph. In that context this is a compliment — he doesn’t just have fiction chops, he has technical chops too and the ability to balance the two together.
I guess you could interpret that as saying tech pubs people write fiction. While that wasn’t my intention, I have met some tech pubs staff who live in a fictional world where everything works great and no one needs to troubleshoot.
Yeah you can’t have alpha geeks doing documentation, everything works out of sheer intimidation. You need a beta on there that everything goes wrong for and really appreciates the hard fought solution.
One of the best technology books I have ever read.
I really love the book. I have read it twice. The book is a really good look into an engineering team and how true hard work pays off. It is a great book for getting the spark going on a project you have slacked off on too.
Yes, it’s a pretty good read. It’s almost a cross between Arthur Haley in how it compares to the fictional work Wheels in structure and pacing, and Tom Wolfe for the nitty gritty non-fic realism.
A later work “House” is quite good also, giving you the insight into the making of a thing.
Read this when it first came out, great read. Don’t forget – https://en.wikipedia.org/wiki/The_Cuckoo%27s_Egg while we’re talking classics!
Quick aside: Data General programmers were paid (or assessed) by lines of code written, so every line ended with;;
Ah yes, the old semicolon counting system of SLOC. Some counting software will still count semis in comments.
Not true. I was on the team developing the system – I designed and implemented the “Ghost”.
It was very true that de Castro would approve the salaries of all the engineers on the team.
I read the book when it came out. About five years later (in 1986) I joined a start up company and was in a cubicle about five feet away from the office of Carl Alsing. A few months later I reread the book. I enjoyed the book the first time, but the second time really earned my respect. The way he described Carl really captured a lot about him, giving credibility to everything else in the book.
I remember two of the questions he asked me when I interviewed there. Problem 1: you have a disk which is half white, half black, and two photo-transistors. Where is the optimum place to situate the phototransistors to detect as quickly as possible if the disk is rotating clockwise or counter-clockwise, and then design a FSM to generate that information, then implement that fsm using TTL. Problem 2: you have software for an old computer and you need to run the software but you can’t find the computer anywhere. He didn’t say “emulate” but I knew what he was driving at and I said I’d write an emulator. That led to an extended dialog about estimating how long it would take to write, what factor slowdown I’d expect, how would I go about validating the emulator, etc.
That’s awesome Jim! Did he have any stories (that you can share) about things that didn’t make it in the book?
It was 30 years ago, so if he did I have forgotten them. If he did say anything it wasn’t memorable.
Problem number 1 reminded me of the Atari Tempest video game. The game had a rotary encoder wheel as one of the main player inputs. It had two phototransistor based sensors arranged for 90° offset square wave output. They connected both to a D flip-flop and the Q output of the flip-flop and one of the sensor outputs were the direction and clock inputs of a hex counter IC. The output of the hex counter was an input into the rest of the system (they actually used four of the inputs of one of the POKEY chips in this case).
It turns out that using a D flip-flop for that would cause one cycle of the detector to register the wrong direction when you transitioned from one direction to the other. Atari’s workaround for that was to simply make the rotation sensor far more sensitive than necessary and decimate it in software. The error was thereby simply lost in the noise.
‘The Mythical Man-Month: Essays on Software Engineering’ is a book on software engineering and project management by Fred Brooks, whose central theme is that “adding manpower to a late software project makes it later”.
The work was republished in an anniversary edition with four extra chapters in 1995 (ISBN 0-201-83595-9), including a reprint of the essay “No Silver Bullet” with commentary by the author.
“Soul of a New Machine” is one of my all-time favorite books, and the only one of those books that is nonfiction ;)
As an aside, if anyone can get me good photos of the Eclipse MV/8000 I’m quite interested… ;)
My first job out of school was working for the MV languages and tools group in RTP, working on Business BASIC and their PL/I compiler. I don’t ever remember being judged by SLOC or anything like that. Smart folks in RTP…. Many of the ex-FHP folks were still around. I learned a lot during the five years I worked there.
FHP could be a whole ‘nother article – I wish I could have included more here! The docs say it would have been twice as fast as MV/8000 in the whetstone benchmark. Lots of interesting architecture in that machine.
“Tracy Kidder’s abilities can’t be understated here.” Yes they can! I’m pretty sure that you meant to say that his abilities can’
t be OVERSTATED.
I did mean overstated. Fixed in the text. Thank you for catching that one Eric.
This book was on my dad’s shelf at home. I read it several times there. Years later I found a beat up old copy and bought it so I could read it again. I left it in a take-a-book-leave-a-book place and have regretted it ever since.
“Starting in 1964 with the Digital Equipment Corporation (DEC) PDP-8, minis provided a low-cost means for companies to get a computer.”
Ya know, given a name like “PDP-8”, one might suspect that minicomputers started a bit earlier. Like, maybe, with the PDP-5 (same instruction set). Or, perhaps, even the PDP-1. Maybe, just maybe, with Whirlwind.
They weren’t very mini, or very mass produced.
ding ding ding! The first 7 were important, but the straight 8 really kicked off the mini revolution.
LIke others here I’m old enough to have read TSOANM when it was a new bestseller. It remains one of the best books about the practice of building computers ever written.
I’d actually forgot that I’d read that book, thanks for reminding me. It is amazing how much memory comes back from a single trigger. I’ll put it in my kid’s Comp. Sci.extended reading list.
Along with http://www.goodreads.com/book/show/1101290.Dealers_of_Lightning and http://www.goodreads.com/book/show/56829.Hackers natch.
Oh and insanely great if you must, but you can stop reading halfway through.
Revolution in the valley – I always liked Andy’s inside view on the Mac story.
I discovered this book about 10 years ago in the library and loved it!
It’s quite the coincidence that Adam posted about this, I recently watched a documentary on Amazon Prime streaming about a chip design team which reminded me of “A Soul of a New Machine”:
Rise of the Centaur
https://www.youtube.com/watch?v=1ILqwOySIao
Here is the link to the documentary on Amazon Prime for those interested: https://www.amazon.com/Rise-Centaur-Glenn-Henry/dp/B01FSZU6FK
Yup, it was close to compulsory reading when you joined D.G., but it certainly wasn’t a chore. Great book!
Worked on DG Eclipse for the Geography Department at SUNY/Buffalo during the early ’80s.. It was sort of a charity gig as there was a PDP-11 in the Particle Physics lab which was my main focus. The Eclipse had its little air-conditioned room and a 300mb ‘washing machine’ mainframe disk drive. I think it had a RamTek frame buffer the display.. At any rate, relative to the DEC PDP, everything was a little foreign. At the end of the day DG was always the B Team. Selling machines for less money. The thing I most remember about is the front panel. They knew they had to had to have an appealing front panel yet, be different from DEC.. And this is what they came up with. I think it is still quite striking.
http://simh.trailing-edge.com/photos/eclipse-s130.jpg
Yah, that’s sexy, I’d wash my clothes in it any day of the week.
I remember being ‘forced’ to read it 30 years ago in EE school. Still got it here somewhere… I should dig it up and read it again now that I actually have time to enjoy reading.
I read this book while I was living it. I was a tech with the bigger pay-check.
First read it in Reader’s Digest
Finally got my hands on a used copy at the local flea market two years ago.
Even the abridged version was totally engrossing and inspiring.
Agree with the author. Outstanding book. Do yourself a favor and read it.
I bought it on the strength of this review and am half way through, great book! Thanks
A great read. It’s been a few years and many books since but what I remember best was the exciting tales of debugging the hardware through the long days and nights. Informative and hard to put down.
I read the book many many years back and highly recommend it to anyone. Very interesting on many levels
I read the book when it came out. Maybe a better description was I lived the book. I was out of college a few years and was working for a small computer company (which was actually mentioned in the book). I did a lot of the stuff covered in the book like working long hours to meet deadlines, lots of wire wrapping and wire wrap changes and even implementing PALs to optimize designs and minimize wiring changes for design fixes.
I read the book a couple of times later. Maybe it’s time to get it out again. Recommended.
Why does seemingly every project ever need so much overtime? I know the rule that everything takes longer to do than you expected, even taking the rule into account, but why don’t they take that into account? Can’t people just start a bit earlier? It’s like Reverse Mr Scott off Star Trek.
Working so long and so hard like that isn’t good for you, and actually takes years off people’s lives. There should be no need for it.
Because the dominant economic model is based on fear of being left alone to die. Makes people more conservative in their investment decisions. Makes it necessary to chip in if you want to go beyond what’s possible at the day. When the only thing you have is your health and mind, you chip in with those.
Somebody else is still walking away with all of the profits, though. Because they owned the building and could afford to pay for the manufacturing runs. Hardly fair if you ask me, but everybody else thinks it’s the best way to enrich the society so who am I to judge.
Some argue that it was also risky for them to give you a chance. You might have overestimated your ability and failed. As if those years you could have lost were nothing.
This book was required reading for my ‘Intro to Digital Design’ class in about 1982. I still have the paperback copy that I bought for that class. It was fascinating and inspired me as a young engineering student. I re-read it several years ago and will probably read it again.
Tracy Kidder is an incredible author and I’ve read a number of his other books. Highly recommend him.
Oh hey that is my dad! I am the kid who was 11 and went for bike rides in that book. Now I work at the Internet Archive keeping Open Library alive. My dad was an interesting guy, a sort of charismatic hermit who loved to fix things but wasn’t so great with people. Here’s an article I wrote about his house.
https://medium.com/message/deathhacks-b767903b7c15#.2a5wfep95
Tracy Kidder was at my dad’s memorial service and one of the things he said about the book was that it had been a very good thing for him (Tracy) but he had never been sure if it had been a good thing for my dad. Fun to read all this stuff here, so many decades later.