The most computationally intense part of an Apollo mission was the moon landing itself, requiring both real-time control and navigation of the Lunar Module (LM) through a sequence of programs known as the P60’s. Data from radar, inertial navigation, and optical data sighted-off by the LM commander himself were fed into the computer in what we’d call today ‘data fusion.’
The guy who wrote that code is Don Eyles and the next best thing to actually hanging out with Don is to read his book. Don’s book reads as if you are at a bar sitting across the table listening to his incredible life story. Its personal, hilarious, stressful, fascinating, and more importantly for those of us who are fans of Hackaday, it’s relatable.
Don epitomizes 1960’s counter-culture. He has been featured in Rolling Stone magazine, credited with saving Apollo 14 by a creative software hack to bypass the faulty abort switch on the LM, in the article entitled Don Eyles: Extra! Weird-Looking Freak Saves Apollo 14!. Don is also a well known photographer in Boston area. If you want to meet him show up to one of his exhibits.
I met up with Don recently at a talk he presented at the MIT Museum on his book: Sunburst and Luminary; An Apollo Memoir.
Don’s story begins as a young computer engineer in the late 60’s where he does his share of rambling but eventually lands a job at Doc Draper’s lab at MIT writing code for the Apollo Guidance Computer (AGC). The story intentionally starts out slow but it picks up rapidly in pace and in urgency as the time to Apollo 11 launch approaches, I found it hard to put this book down.
The First Fly-By-Wire
For those not familiar, the Apollo system was the first digital fly by wire. (An interesting book on this is Digital Apollo.) In addition to real-time flight control the computer was responsible for navigation. There were two Apollo Guidance Computer (AGC) systems; one on the command module and one on the lunar module. Below you can get an idea of the system design, and see the IO interfaces for the 1965 Lunar Guidance Computer — the guidance computer on found on the LM.
Earning the respect of his peers and supervisors Don was asked to code some of the toughest software, specifically he wrote the P60’s. These were moon landing programs, the most computationally intense part of the mission leaving only ~8% CPU margin. The spacecraft had to de-orbit itself and transition from a ballistic trajectory and to one more akin to a helicopter, then touch-down softly on the lunar surface.
Don’s code was put to the test during humanity’s first attempt to land on another world, Apollo 11, things did not go as planned… the computer crashed!
Don and his colleagues followed the landing with their program lists (Don gave us a tour of his Luminary list for Hackaday) from their office in Cambridge MA.
During the final phases of landing, program alarms 1201 and 1202 were going off, the DSKY display went blank at one point. Each time a program alarm occurred the computer re-started. Neil Armstrong was not happy, you can tell by the tone of his voice in the recorded radio traffic.
The team at Draper had not experienced a problem like this in the hardware simulators (there were three different LM hardware simulators, including one at Draper, one at Grumman on Long Island, and the LM mission simulator at NASA) It must have been a computer hardware failure, Don thought, they should immediately abort. But to everyone’s surprise Mission control replied quickly to each program alarm with a GO.
There’s a good reason for this, Don and his colleagues created a very robust software platform. During a re-start the AGC would automatically kill all unnecessary jobs and tasks and immediately start where it left off on just the critical tasks and jobs such as real-time thruster control, engine throttle control, and navigation. Thanks in part to the solid-state program memory restarts were quick and hardly noticed by the crew.
This scenario had been simulated by NASA and the solution was found empirically to be to go ahead with the mission. NASA at that time didn’t know exactly why or how the system continued to work in this scenario, they just knew that the system would continue to work regardless of the alarm in this phase of the landing.
The rest is history. Neil and Buzz went long, over-shooting their planned-for landing area ending up in the sea of tranquility, landing safely with seconds of fuel to spare.
As it turns out Don’s code was not the problem. The docking radar was left on and in a state where it was triggering computer interrupts consuming the slim computational margin. This problem was easily remedied for later missions.
A Life’s Story Interspersed with Technical History
Don dives into detail on how to program the AGC, explaining both the assembly language and the symbolic languages used. He describes how the multi-threaded operating system worked wherein there were multiple tasks and jobs being managed by the computer at the same time much like a modern computer system. Details on how code is developed with punch cards, simulated in slow-time with a mainframe, wired into solid-state rope memory, and tested rigorously in real-time simulators are all part of the story. Don even goes as far as describing specific functions, and the tricks used to compute 3D vector products in this fascinating deep dive into the Apollo Guidance Computer.
If any of this sounds interesting to you then pick up a copy of Don’s book or borrow it from your local library. As an electrical engineer I highly recommend it for a good summer read.
Eyles is also a photographer and an artist. One of his creations sits in the Fort Point Channel:
https://www.bostonglobe.com/lifestyle/names/2014/10/13/fort-point-artist-pyramid-hits-water/vPwIOcPDBtlIfh6CzXLvfK/story.html
Looks like the talk he gave at MIT is available here:
https://www.youtube.com/watch?v=eotnk1wVSB8
I’d love to buy this book if it was available in Kindle or some other electronic format. I’ve given up buying hardcopy books – they take up much space in my house.
The e-book vs paper is an interesting issue – I like the idea of the lack of space an ebook takes up but lending it to a friend is near on impossible – as a result I’ve gone back to buying books and been able to lend them out .
I thoroughly enjoyed Don’s book. As a NASA contractor, it’s refreshing to see a more sausage-making perspective on (part of) the Apollo program, with difficulties getting acknowledged. People are still terrible at ICDs.
Don goes into more detail on why the Apollo 11 landing was troubled by the restarts (the ICD didn’t specify the power phase for the docking radar). There’s a good Ars Technica article on it too.
Another case of “ensure system is in appropriate state prior to starting operation” situations.
Thanks for the nudge, this has been on my radar, and I finally pulled the trigger.
I’d also recommend “The Apollo Guidance Computer: Architecture and Operation” by Frank O’Brien.