A crewed mission to the International Space Station that was set to depart from Kennedy Space Center on Halloween has been pushed back at least several weeks as NASA and SpaceX investigate an issue with the company’s Merlin rocket engine. But the problem in question wasn’t actually discovered on the booster that’s slated to carry the four new crew members up to the orbiting outpost. This story starts back on October 2nd, when the computer aboard a Falcon 9 set to carry a next-generation GPS III satellite into orbit for the US Space Force shut down the engines with just two seconds to go before liftoff.
The fact that SpaceX and NASA have decided to push back the launch of a different Falcon 9 is a clear indication that the issue isn’t limited to just one specific booster, and must be a problem with the design or construction of the Merlin engine itself. While both entities have been relatively tight lipped about the current situation, a Tweet from CEO Elon Musk made just hours after the GPS III abort hinted the problem was with the engine’s gas generator:
As we’ve discussed previously, the Merlin is what’s known as an “open cycle” rocket engine. In this classical design, which dates back to the German V-2 of WWII, the exhaust from what’s essentially a smaller and less efficient rocket engine is used to spin a turbine and generate the power required to pump the propellants into the main combustion chamber. Higher than expected pressure in the gas generator could lead to a catastrophic failure of the turbine it drives, so it’s no surprise that the Falcon 9’s onboard systems determined an abort was in order.
Grounding an entire fleet of rockets because a potentially serious fault has been discovered in one of them is a rational precaution, and has been done many times before. Engineers need time to investigate the issue and determine if changes must be made on the rest of the vehicles before they can safely return to flight. But that’s where things get interesting in this case.
SpaceX hasn’t grounded their entire fleet of Falcon 9 rockets. In fact, the company has flown several of them since the October 2nd launch abort. So why are only some of these boosters stuck in their hangers, while others are continuing to fly their scheduled missions?
Certified Pre-Flown Boosters
Since 2018, an increasing number of SpaceX missions have utilized previously flown boosters. This is thanks to the introduction of the newest, and final, revision of the rocket known as Block 5. After gaining valuable flight experience with earlier versions of the booster, SpaceX engineers were able to identify the components that were the most heavily damaged during flight, reentry into the Earth’s atmosphere, and landing.
With these weak points addressed in Block 5, SpaceX believes the Falcon 9 should be able to fly at least 10 missions with only minor refurbishment. With occasional overhauls, each fuselage could potentially make as many as 100 flights, though it will be years or even decades before that theory can be put to the test. To date, no individual Falcon 9 has made more than six flights.
Out of the 18 successful launches that SpaceX has conducted so far in 2020, all but two of them have been on reused Block 5 boosters. Most of these rockets, and their engines, were originally constructed in 2018 or 2019. Newly manufactured boosters are generally only used at the customer’s specific request at this point, and the two entities that have consistently asked to fly on fresh rockets just so happen to be NASA and the Space Force.
Consequently, the rockets that were scheduled to launch the Crew Dragon Resilience on October 31st and the GPS III satellite Sacagawea on October 2nd were both built around the same time earlier this year. In fact they even have sequential serial numbers, referred to by SpaceX as B1061 and B1062, respectively.
In light of this, it becomes clear why SpaceX hasn’t grounded the entire fleet. We aren’t looking at an issue that impacts the Falcon 9 itself, merely the ones that have been manufactured in 2020.
Age Versus Experience
As the design of the Falcon 9 Block 5 and Merlin engine have been frozen since 2018 to facilitate human-rating the vehicle by NASA, we know no new or redesigned components have been introduced. But that doesn’t mean SpaceX couldn’t have received a bad batch of components from a supplier which found their way into B1061 and B1062. Such a scenario wouldn’t be without precedent.
In 2015, the CRS-7 mission ended with a complete loss of the vehicle just over two minutes after liftoff when a high-pressure helium bottle inside the Falcon 9’s second stage broke loose. An examination of the stainless steel eye bolts used to hold the bottles in place found that, despite being rated by the manufacturer for a load of 10,000 pounds, some failed at just 2,000 pounds. A later investigation by NASA determined that the onus for the mishap was ultimately on SpaceX as they failed to independently verify the manufacturers claims before flight.
While pre-flight testing is of course important, there’s no substitute for the real thing. SpaceX has always maintained that a rocket designed for reusability will be just as reliable, if not more so, than one designed for a single flight. With each subsequent flight and inspection, the company believes the overall reliability of the vehicle is improved.
Ancillary parts that appear to be wearing down are replaced, while core vehicle systems that have proven their reliability remain in place. SpaceX’s stated goal is to implement a maintenance schedule for their rockets not unlike what’s used with commercial aircraft, and eventually, be able to return an individual booster to flight within a few days or even hours.
In requesting a fresh rocket for their missions NASA and the Space Force believe they’re getting a more reliable vehicle, but this would appear to be at odds with the facts. To date, new and reused Falcon 9 Block 5 rockets have a cumulative 100% success rate. There’s no evidence to support the assertion that one is inherently more reliable than the other, whereas the current situation seems to indicate that the use of flight-proven boosters would have prevented delaying the Resilience and Sacagawea missions.
Test Like You Fly
Of course, a delay is preferable to a failure. When human lives are at stake, it’s always better to take the extra time to make sure everything is working perfectly. According to Kathy Lueders, Associate Administrator of NASA’s Human Exploration and Operations Mission Directorate, SpaceX is currently replacing one of the engines in the ISS-bound Falcon 9 that exhibited gas generator behavior similar to the October 2nd anomaly. They will also replace an engine in the next rocket that came off of the assembly line, serial number B1063, which is currently scheduled to launch a joint NASA/ESA Earth observation satellite before the end of the year.
Which brings up an interesting point. Using a standardized booster for all payloads, whether it’s cargo for the Space Station, communication satellites, or human occupants, allows for more opportunities to catch potential flaws than NASA ever had during the Space Shuttle era.
If the crewed mission wasn’t set to fly on a rocket that was virtually identical to the one carrying the Sacagawea GPS III satellite, the gas generator issue may not have been detected in time. While the Crew Dragon is ready and able to pull its human occupants to safety should the booster rocket fail, it’s not a contingency anyone is particularly eager to utilize.
Between post-flight examinations and maintaining the same vehicle configuration regardless of payload, the Falcon 9 is quickly working its way towards becoming one of history’s most reliable orbital boosters. Which is likely why NASA has decided to allow crew flights on flight-proven Falcon 9 boosters next year. The agreement specifies the agency will only allow astronauts to fly on lightly used rockets, but at least it’s a start.
Sounds like a gas problem
They should check the carb.
Adjusting the carbs is often the simple solution, and if it works great. However, in my experience there are a variety of causes that need to be taken into account such as vessel stress, fuel load, and often even the ambient temperature that can dramatically affect the situation. I would suggest high-protein and plenty of water until things normalise.
Have you taken into account that the forty-one manestically placed grouting brushes were arrranged to feed into therotor slip stream a mixture of high S-value phenyhydrobenzamine and 5 percent reminative tetraiodohexamine. Both these liquids have specific pericosities given by p=2.4 Cn where n is the iathecial evolute of retrograde temperature phase disposition and C is the Chomondeley’s annual grillage coefficient. Initially, n was measured with the aid of a metapolar pilfrometer, but up to the present date nothing has been found to equal the transcetental hopper dadoscope.
Instead, I would recommend Bromo-Seltzer.
> There’s no evidence to support the assertion that one is inherently more reliable than the other
Except the inevitable fact of metal fatigue. Vibrations and load cycles cause crack growth, which can only be detected by elaborate X-ray, magnetic, ultrasonic etc. testing of every square millimeter of the part that they re-use. This would cost a ton of money and time, so it is unlikely that SpaceX bothers with a full inspection.
The problem of a space rocket is that the violent shaking during launch is like fyling a commercial airliner for ten years in two minutes. If there is a beginning of a crack, things can go south very quickly. In other words, there is no guarantee that an engine or a critical load bearing part which survived one flight will survive another, since there is only a very narrow margin to spot the kind of fault that makes it fail.
It would be interesting to see what kind of a bath-tub curve it makes for rockets.
If there’s a manufacturing flaw, it will likely fail on the first go, so you could argue that the second launch is probably safer – but without inspecting all the parts, you can’t say that the first launch hasn’t opened up some microscopic crack that is just about to fail.
On the other hand, there could be microscopic cracks in metal castings already from manufacturing, which similarly can only be detected by elaborate investigation. But at least flying a lot of rockets (and most of them unmanned) will eventually point out parts that are prone to failure, and they can then be investigated more closely.
After all, fatigue inspections are a standard part of aircraft maintenance already.
Yes, but the inspections aren’t performed after every single flight, and the cracks in an airplane frame have time to grow visible before they become acutely dangerous.
That’s the issue that they faced with the Space Shuttle as well. There was so much that could go wrong that they had to spend weeks and months going through every detail, just looking at it and finding the things that might break. The Falcon is a simpler machine, but it doesn’t mean it has any less stuff to inspect. Meanwhile SpaceX is trying to push the turnaround time to days if not hours, which means they have to just wave it through without really checking anything. They barely have enough time to pull the thing apart and change a few gaskets.
And the people who actually fly on them, the astronauts, seem more than happy to climb aboard. Hell, I’d give body parts to fly on one myself. Nothing can be completely 100% safe, and the longer you chase that long tail the harder it becomes to actually get stuff done. Risk must be balanced.
That’s all fine, but if SpaceX wants to prove the economy of re-usable rockets, then the risk of failure must be brought down from the industry average for single-use rockets.
Otherwise they’re losing too many boosters too soon to save any money.
I suspect the number of people who really want to go to space all *that* much is low enough to just not even worry about reuse for humans. If an unmanned rocket blows up, nobody cares. It’s just cost of doing business, and probably still more eco friendly than trying to get by without satellites.
I get it that a lot of people are really passionate about making manned spaceflight accessible to more people… but it’s gonna be real uncomfortable, you probably won’t accomplish any science unless you actually have the training, and colonizing other planets is probably going to be straight up not a good time, without decades of tech advances.
Maybe we should be a little more realistic about how hard it would be to actually get large numbers of people into space safely.
And people who do want to go to space, should ask themselves how they feel about lockdowns and mask wearing. Much as most of the hacker/science crowd seems to be onboard, that doesn’t mean we *like* it. And in space, stepping out for a bit without you mask is gonna be *real* deadly.
> It’s just cost of doing business
Well, exactly.
If you give it an average 2.5% risk of failure, then the probability that your booster will not fly after the n-th flight is 1 – (1-0.025)^n where for n=10 your risk becomes 22%. Therefore one in five of your boosters are expected to fail by launch eleven – as opposed to 25 out of every 1000 single use boosters no matter how many times you launch them, because each is an independent case. Notice how the accumulation of risk changes because you’re using the same rocket multiple times. Of course this is not the exact same comparison, because some rockets do survive to 12 and more launches, so the calculation is incomplete at this point.
It’s like this. If you have 100 coins and you throw them on the table, you expect to see half of them coming heads up. But if you have one coin, and you start dropping it on the table a hundred times, trying to get heads to come up 50 times probably ends up on the third toss. If you then change the coin and try again, you may get to 50 heads with somewhat fewer coins than a hundred, with some probability.
The question is, how many fewer coins do you need? If you keep doing it forever, the answer is “no fewer coins” because the odds are still 50/50 and you’re simply gambling. On the short term you may win, but in the long term you will do no better.
That is exactly the gamble that Elon Musk is taking. If he had completed his studies instead of bailing out of school, he would have taken statistics and figured out that he’s going to lose in the end.
“If you give it an average 2.5% risk of failure, then the probability that your booster will not fly after the n-th flight is 1 – (1-0.025)^n where for n=10 your risk becomes 22%. Therefore one in five of your boosters are expected to fail by launch eleven – as opposed to 25 out of every 1000 single use boosters no matter how many times you launch them, because each is an independent case.”
ONLY if your rate of failure is perfectly and exclusively dependent on your number of flights.
If your re-usable booster design manages an equivalent/net “reliability after 10 flights” as a single-use booster’s single-flight reliability by incurring the additional costs of over-designed components (weight/fuel), recovery labor, disassembly, inspections (or over-designing to negate inspections), reassembly, etc. and yet still comes out to less than 10 times the cost of the disposable booster then you come ahead in the long run. For an EQUAL risk of failure across all ten flights of either design.
Then that’s your break-even point.
However, SpaceX has an additional problem: some of their customers need to launch more mass, or reach a higher orbit, which requires more fuel and expends the booster. The risk of losing the booster is not only because of technical faults. The whole recovery process is fraught with possibilities for mishaps, and getting the rocket to go up is just half of the story.
It is likely that spacex wiĺl implement preventive engine replacement based on condition monitoring as the falcon 9 fleet ages based off airline current rules, they just need more flights to build a robust statistic database on the merlin engines failure rate and modes. Eventually this will be ported to starship/superheavy and the raptors.
I’m pretty sure the original problem was traced to a defect in a supplier part.
The rocket assembly crew probably all had COVID-19 and weren’t feeling well that week.
It was just announced that a flake of lacquer was clogging a relief valve in the gas generator. It’s unclear whether this is a new issue, or has happened previously, and was just uncaught. The nature of the issue means that it is a first flight only problem, presumably because a full flight is enough to burn the lacquer out of the system.
https://arstechnica.com/science/2020/10/nasa-and-spacex-set-new-date-for-crew-launch-explain-merlin-engine-issue/
Red coating contaminates SpaceX rockets, delays crew launch
29 Oct 2020
https://phys.org/news/2020-10-red-coating-contaminates-spacex-rockets.html
The clogged holes were found after the aborted launch of a GPS satellite on Oct. 2. Two of those engines were contaminated with the bright red coating, which protects engine parts during cleaning.
SpaceX later found the same problem with two of the nine booster engines on the rocket that will launch four astronauts to the International Space Station. It will be SpaceX’s second launch of astronauts for NASA after a successful test flight earlier this year.
A company vice president, Hans Koenigsmann, said new procedures are being implemented to the prevent the problem.
The clogged vent holes—located in the engine gas generators—are just one-sixteenth of an inch (1.6 millimeters).
On-board computers detected the problem and halted the Oct. 2 countdown in the final few seconds. The contaminated engines could have been badly damaged if they had fired, Koenigsmann said.
The cleaning process is done by an outside company, which was not identified. The problem appears to have cropped up in just the last couple months, he noted, affecting brand new boosters but not the older ones that already have been recycled several times.