Better booster production

The retirement of the space shuttle fleet in 2011 afforded an opportunity for NASA and industry managers to take a fresh look at production practices. Justin Pancoast of Northrop Grumman explains how engineers expunged inefficiencies from production of the solid rocket boosters for NASA’s Space Launch System, the agency’s next human-rated launch vehicle.

When you’ve perfected the manufacturing of a product over decades for a government customer, it isn’t easy to make changes. In the aerospace industry in particular, no one wants to introduce risk by modifying processes, even when new materials become available and innovative technologies emerge.

When NASA retired the space shuttle fleet and transitioned work to the heavy lift Space Launch System, or SLS, it opened the door for Orbital ATK, now Northrop Grumman Innovation Systems, to make changes that could simultaneously improve the quality and cost of the large solid rocket boosters the company builds for SLS. Part of my job during that review in 2011 and 2012 was to help come up with changes in my work area, the insulation component work center where SLS boosters are now prepared. During the shuttle program, bare metal rocket motor cases and components began their production journey here following refurbishment after a previous space shuttle flight. Today, metal case surfaces are meticulously cleaned and prepared for exterior painting and internal case bonding operations. Erosion-resistant thermal insulation materials are applied to interior case surfaces to protect metal hardware from the heat generated by burning propellant. Following the autoclave curing of this insulation layer on the case, an adhesive liner is applied to act as glue between the insulation and propellant that will be cast in a different facility after liner application. In addition to insulated cases, this center is also responsible for fabrication of the flex bearing, which is a critical component of the booster that allows the thrust vector control system to move the nozzle for steering control.

Similar efforts took place concurrently in each of the other work centers — case refurbishment, non-destructive inspection, insulation, nozzle, mix-cast and final assembly — as well as in administrative support functions, such as supply chain management, finance and quality.
We followed six steps to make these changes.

Step 1: Involve the customer

When I started working with the other members of the insulation component work center team on this assignment, building a solid rocket booster was an orchestrated process that had developed and evolved over the course of the Space Shuttle Program. Throughout the program the original process had grown little by little to include additional checks and inspections introduced to correct and/or verify problem areas within the process. These additions were in response to non-conformances discovered at later stages of production or during post-flight inspections. There were many “obvious” (or so we thought) areas where we could reduce cycle times — the number of hours required to manufacture each part of the booster — by modifying or eliminating production steps and eliminating wasteful practices in general. However, we were concerned NASA might not be willing to accept those changes, as the existing processes were flight-proven over many years and had evolved with mission success in mind.

During our first several meetings with NASA as the prime contractor for the SLS booster element, the agency made it very clear that it was fully on board with nearly any change we could potentially bring forward, as long as we were able to develop a clear technical rationale for the change, and assurances that it would not diminish final quality or product performance and would offer a positive return on investment. We also had to get all proposed changes approved and implemented through established control procedures overseen by a joint NASA/Orbital ATK leadership team.

Alex Priskos, who was NASA’s SLS booster manager at the time, praised our company’s concerted effort to eliminate waste and drive down costs while managing risk. In a “thank you” video to employees in 2012, Priskos said NASA had “accepted the reality that we must embrace change rather than avoid it. We are embracing innovation both technically and in our management processes in order to be successful in these constrained budget environments. … Our shared goal is to deliver a safe, affordable and sustainable launch vehicle.”

Step 2: Map the baseline

Our next step was to map out the baseline process. We had dedicated conference rooms (designated as war rooms) where we would lay out a process flow diagram and post it on the wall to have a visual reference. Then we used Post-It notes of various colors to identify all of the process steps and laid them out in proper series and parallel flows, so we would see the interconnectedness and have the ability to move the process elements around as we worked through the modifications. In addition to mapping the flow, we estimated cycle times for each part of the process. The frustrating (and informative) part of this effort was that we had to revise the baseline process flow a number of times as we went through the planning documents to verify that we fully understood our baseline process. There ended up being a number of moves and holds in the process that, although we were aware of them, hadn’t been included in the original layout of the process. By the time we completed the baseline process layout, we had already identified a couple of moves we could potentially eliminate by combining operations in one station rather than moving a 3.7-meter-diameter steel cylinder weighing anywhere from 9,000 to 14,000 kilograms from station to station, thus reducing cycle time, improving safety, and reducing risk to the product.

Step 3: Have conversations with the customer about obvious waste

As we started to look at the process in more depth, we came up with more ideas about how to reduce the amount of time required for various steps. Sometimes these ideas were relatively simple. For instance, our baseline insulation process required a 55-minute dry time after cleaning with a specific solvent. Other work centers doing similar critical bonding operations successfully used a 30-minute dry time for the same solvent on a very similar material. By aligning our dry time with that used by other work centers, we could eliminate 25 minutes of process time on every step where that solvent was used. NASA asked for verification that the change would not impact bond integrity. We were able to point to the similarity with other work centers and develop bounding technical rationale for the change. As a result, NASA asked us to implement the change immediately. We found another easy refinement was in the standard practice of performing an alcohol wipe on bonding surfaces just prior to bonding. At first glance, this seems like a prudent precaution. However, the practice had crept over the years to the extent that operators were cleaning some parts multiple times even though they had not undergone any additional processing. We determined the alcohol wipes at these unprocessed stages were redundant and wasteful and could be eliminated.

Other process change recommendations were less straightforward but still relatively simple. As an example, over the years of the shuttle program, someone determined that a station used to apply a coating to the interior of the case segments would be more effective if operators introduced a heating capability to decrease dry times. That resulted in the need to let the coated part cool before operators could perform the next step in the process. We thought it might be possible to deliver ambient air to the station via what are normally heating ducts to “force cool” the part. After some testing we determined that the accelerated cooling did not introduce any changes to the initial coating. After hearing our justification, NASA allowed us to adopt the forced cooling option, saving nearly 24 hours of process time.

Another example was the procedure for touching up Alodine, which is a coating applied for corrosion resistance and surface preparation for bonding. As certain parts moved through the insulation component work center, operators would repair any scratches in the Alodine immediately, which introduced a delay into the process. The baseline process carried nearly 72 hours of process time to perform these touch-up operations while in the insulation component work center. Because the final assembly work center also had the requirement to perform any necessary Alodine repairs prior to final assembly operations, all Alodine operations in the insulation component work center were redundant. When we discussed the elimination of Alodine repairs in the insulation component work center, some team members hesitated to eliminate the repair of a corrosion control feature, but we determined that with the typical ambient conditions in Promontory, Utah, where we manufacture the solid rocket boosters, and the length of time involved in moving between facilities in Promontory, the risk was minimal and any necessary remediation was already in place as part of the touch-up procedures in the final assembly work center. NASA approved the elimination of Alodine touch-up in the insulation component work center, saving another 72 hours of process time.

We also scrutinized operations where segments sat idle, looking for additional opportunities to reduce cycle time. One such instance was the amount of time case insulation spent under vacuum prior to the curing process. Review of data on the boosters’ new insulation materials and some very basic testing demonstrated a 73-percent reduction in vacuum sit time was possible, effectively shaving one to two days from the overall processing timeline.

Step 4: Challenge norms and debate recommendations

Once we identified and completed the more obvious changes, we made additional efforts to target operations that were potentially more divisive. We examined inspection records from the Space Shuttle Program to determine how often inspectors found defects at various inspection points. After a detailed analysis, we identified several inspection points that had not identified any defects during the duration of the program — that’s more than 30 years and 330 boosters built (270 flight boosters and 50 static test motors). We deemed these inspection points unnecessary and recommended their elimination. A healthy debate with NASA representatives ensued, and after analysis of the risks involved, NASA agreed to the elimination of several of them. We did not eliminate all of the inspection points as the debates led to a consensus that a few inspection points were in place to screen for potential failure modes that while obviously unlikely were known and severe enough to warrant the inspection.

Similar to our examination of hardware inspections, we took a deep dive into raw material inspections to identify potentially wasteful practices. We found numerous instances where the material supplier and our own acceptance testing lab performed identical tests, duplicating efforts. In most instances this duplication was not an actual requirement, but a practice developed over time. Stopping this practice significantly improved lead time on the material’s availability, and as a bonus, the extra capacity created in the lab enabled the lab employees to streamline their operations. We made similar material enhancements by carefully reviewing shelf-lives and re-test data on well-characterized materials used over the decades. We assigned new shelf lives to materials that re-tested multiple times without failing, as long as data and aging mechanism analyses supported the change. This resulted in not only less frequent raw material orders, but also greater availability of existing material inventories.

Step 5: Predict results

Over the course of the efficiency effort, which lasted several months, the insulation component work center team recommended changes predicted to reduce the time required to process a center segment of the five-segment booster from 54 days to 24 days. We predicted similar savings for forward and aft segments. For the overall booster operation, 31 teams identified 308 changes to eliminate waste in the workflow. Those changes resulted in cutting 447 material moves from the workflow, which translated into a predicted overall cycle time reduction of 46 percent.

Step 6: Implement changes and track actual results

Orbital ATK concluded this waste elimination exercise in 2012, and we continue operating under these improved processes as part of Northrop Grumman. The company has manufactured seven five-segment solid rocket motors: three development test motors, two qualification motors, and the two boosters for the first flight of NASA’s Space Launch System. The flight boosters will be delivered to NASA’s Kennedy Space Center in just a few months to be integrated into the vehicle that will enable humans to return to deep space for the first time in more than 45 years.

Having completed the 35 segments for these seven boosters using the improved processes, Northrop Grumman has enough data to validate the quality of the boosters and the actual reduction in cycle time. Much of those data are proprietary, but I can share that the results of multiple static tests indicate the changes in no way compromise the quality of the boosters. In addition, the plant as a whole now performs the same work it did during the Space Shuttle Program, but with a workforce that is less than half the size.

NASA’s support of Northrop Grumman’s concerted waste elimination initiative seemed revolutionary at the time; little did we know the agency was ushering in a new operating model that would become a permanent part of human spaceflight. Thanks to prime contractors willing to make changes to eliminate waste, and program leaders willing to embrace change while managing risk, NASA can confidently and credibly use words like “affordable” and “sustainable” when talking about human missions to deep space. Affordability and sustainability are just the characteristics we need in a human spaceflight program whose purpose is to return humans to the moon and send the first human explorers to Mars.