Interplanetary probes were a constant in the tech news bulletins of the 1960s and 1970s. The Space Race was at its height, and alongside their manned flights the two superpowers sent unmanned missions throughout the Solar System. By the 1980s and early 1990s the Space Race had cooled down, the bean counters moved in, and aside from the spectacular images of the planets periodically arriving from the Voyager series of craft there were scant pickings for the deep space enthusiast.
The launch in late 1996 of the Mars Pathfinder mission with its Sojourner rover then was exciting news indeed. Before Spirit, the exceptionally long-lived Opportunity, and the relatively huge Curiosity rover (get a sense of scale from our recent tour of JPL), the little Sojourner operated on the surface of the planet for 85 days, and proved the technology for the rovers that followed.
In these days of constant online information we’d see every nuance of the operation as it happened, but those of us watching with interest in 1997 missed one of the mission’s dramas. Pathfinder’s lander suffered what is being written up today as the first bug on Mars. When the lander collected Martian weather data, its computer would crash.
Like many other spacecraft, the lander’s computer system ran the real-time OS VxWorks. Of the threads running on the craft, the weather thread was a low priority, while the more important task of servicing its information bus was a high priority one. The weather task would hog the resources, causing the operating system equivalent of an unholy row in our Martian outpost. A priority inversion bug, and one that had been spotted before launch but assigned a low priority.
You can’t walk up to a computer on another planet and swap out a few disks, so the Pathfinder team had to investigate the problem on their Earthbound replica of the lander. The fix involved executing some C code on an interpreter prompt on the spacecraft itself, something that would give most engineers an extremely anxious moment.
The write-up is an interesting read, it’s a translation from a Russian original that is linked within it. If the work of the JPL scientists and engineers interests you, this talk from the recent Hackaday superconference might be of interest.
[via Hacker News]
It would of been cool if they had to do a buffer overflow to load code to stream in the binary patch..
it would have been..*
I assume that picture is of the replica in the test bed? Wondering who took the picture. ;)
Cool story though.
No. Mars pathfinder mission consisted of two spacecraft: The “lander” and the “rover”. The lander was equipped with solar panels, scientific instruments and a very good stereo camera, so the pictures of the rover in Martian soil were taken by the lander
“Cool story though.” Yes, yes it is.
The Pathfinder base station took the picture. Sojourner didn’t travel far.
Actually, we don’t know how far it traveled.
Sojourner could only communicate to Earth through Pathfinder. It was pre-programmed to travel autonomously in ever-widening circles from Pathfinder. Since Sojourner was still operating when Pathfinder finally shut down, there’s a very good possibility Sojourner kept exploring after its communication line to Earth was cut. It could have lasted months before finally succumbing to the cold Martian winter.
Now let’s be sad for an anthropomorphized Martian rover
Aww… poor Sojourner!
Surely the first bug on Mars would have been much earlier, at the very least the update that bricked the Viking-1 lander. (See http://www.atmos.washington.edu/~mars/LFEM/lfemstep/lfemstep_slides/viking_documents/Pdf/JPL_Publication_82-107.pdf, page 28 / “3-12” in the PDF). Presumably there were other earlier bugs in the Viking software, and arguably one of the failed Soviet landings could count if the cause was known.
Interesting, Russian write-up on an American craft.
“The mission was at risk, but the powerful debugging functionality provided by the operating system, and professionalism of the programmers back on Earth (the guys did know their subject) enabled NASA to fix the bug in a short time.”
Now if we could only translate that back to earth.
Putin is constantly talking up the US, and then the US news simply ignores it and reports the opposite as it only exists in their minds.
Then it’s followed by a story how bad ‘fake news’ is and how all the others are doing fake news…
Define irony in a priority inversion bug being assigned a low priority…
Agreed
The article makes it sound like this story first surfaced today, which is patently untrue. The story has been available (indeed, I read it back in 2007) online since 1997:
http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/
Still, absolutely worth posting – it’s a great write-up and a fascinating story.
For more interesting stories check out Mike Deliman’s blog. Mike was the customer support engineer dedicated to supporting NASA JPL on VxWorks.
http://blogs.windriver.com/deliman/
Somewhere I read an incredibly succinct and detailed list of every issue the Opportunity rover had. I haven’t been able to find it since, but I thought it was very interesting. Some of the entries detailed how heaters were stuck on and prematurely fatigued components, having to muster all the power they could to move a stuck arm into a position where the rover could move. I wish I could find it again.
“The fix involved executing some C code on an interpreter prompt on the spacecraft itself”
Really? Because the site you linked to says different:
“No, we did not use the vxWorks shell to change the software (although the shell is usable on the spacecraft). The process of “patching” the software on the spacecraft is a specialized process. It involves sending the differences between what you have onboard and what you want (and have on Earth) to the spacecraft. Custom software on the spacecraft (with a whole bunch of validation) modifies the onboard copy. If you want more info you can send me email.
— Glenn Reeves, team leader of Mars Pathfinder software developer team”
*reads the next line*
Oh wait, I’m an idiot. Carry on..