Security and STS-107

Having contingency plans for many of the more troublesome possible events during launch was a hallmark of the Shuttle program. Developing these plans and training for them was ‘business as usual’ for us. So was hoping they would never be needed.

On-Pad abort (shutdown of the main engines just prior to T-0), the slidewire emergency egress system, Return to Launch Site Abort, Transoceanic Abort Landing, are four we had. Do you know which one was actually used?

One of the major changes in my job happened following the attacks on 9/11/01. Security for launches was never the same after that day.

Let’s go back to that time. Columbia’s launch was a mere sixteen months after 9/11, and only the seventh one after those attacks. Security around the country was transformed for special events and everyday life. Security for Shuttle launches went from an important but local event to one of national significance and regional impact. Declared a National Asset, the Shuttle “enjoyed” the same level of attention and support as the Super Bowl and presidential trips. For those of us trying to get real work done (launching it!), “enjoy” became the euphemism for enduring it while necessarily becoming part of it. The Launch Director became part of the security system, big time.

Obviously, I can’t say a lot about the specifics, but rest assured we were well protected. The posture for launches before 9/11 was probably about 10% of what we had after in terms of complexity and impact. Before, we cleared certain areas of KSC and CCAFS of visitors and employees to protect them from a launch accident. Afterward, we cleared much more area to also protect us from a possible attack. Ocean exclusion zones expanded, and we had many more assets to clear and secure the area. Lockdown of the land areas became more important and started earlier in the launch campaign. Clearing local and regional airspace was the most significant change, and the most difficult to plan and execute. Airports and airline routes were affected like never before.

The DOD stepped in, rightfully, and thankfully so. They had the knowledge, experience, and assets to pull it off. Integrating their operation and responsibilities into the launch countdown process was a tremendous amount of work, but was required. It was a contingency plan we hoped would never be needed.

Fourteen years ago this week we conducted our final launch security simulation with well over a dozen local, state, and federal agencies participating. We had a special area for their reps in the LCC with links back to their supporting staff. Dedicated communications capability from me to the DOD was tested one final time. Ground, sea, and air assets participated. A subset of the launch team was on hand as were a few shuttle program managers. A simulated airspace violation was declared and we reacted, and reacted well. The sim ended with Columbia in orbit. The plans were tweaked just a little then finalized. This greater team was ready for the real thing.

It was the fourth sim we held in preparation for STS-107. Why so much attention for this mission? The prior seven had less intrusive and expensive “tabletop sims” – usually no real air assets flying for instance. Well, in addition to the (now normal) practice of protecting the shuttle, Ilan Ramon was flying as the first Israeli astronaut, and our intelligence community friends were “concerned.” Security was expanded for STS-107 like no other launch before or since. Without details, we postulated all sorts of possibilities that could affect launch, and protected for all of them. It was a tremendously successful exercise of cooperation between so many agencies that shared a common goal.

Lots of work and expense? You bet. But it was most thorough protection against the unlikely that we ever did.

And it would pay off.

Next week: Launch Day

ksc-02pd-1938-crew-and-m113-tcdt
The STS-107 crew trains for an emergency evacuation with the M-113 armored personnel carrier during TCDT week. (NASA photo KSC-02PD-1938)

Columbia’s final flow

Last time I talked a little bit about Columbia’s final time at the launch pad and why it had to extend over the annual year-end holiday period. This time I’d like to take you back to those days 14 years ago and recall it in more detail.

The final Launch Team training exercise for STS-107 happened on Friday, December 6, 2002. It was the last chance for the whole team to practice together in an environment that simulated launch countdown conditions as closely as possible. It was also the ‘cert run’ for all members to be declared certified to conduct the actual launch in mid January. It went very well with the team solving numerous diabolical problems crafted by the training team to test our knowledge and ability to work under time constraints imposed by a countdown clock. And though the whole thing was conducted against a math model instead of the actual flight and ground hardware, it always felt like launch day itself, less the TV cameras and ties! We were ready. We took the weekend off.

We came back to work Monday morning December 9 and promptly rolled the vehicle out to the Pad. This eight-hour job was normally conducted at night to avoid daytime storms and lightning, but this time of year they weren’t a big threat so we rolled on first shift. I gave the GO to roll and the team did what they did best, getting the Columbia stack (Orbiter, ET, SRBs) on the Mobile Launch Platform to Pad A in a little less than 8 hours on top of the massive Crawler Transporter. Once there, the CT lowered the MLP onto six mounts at the pad and backed away, its job done.

The rest of that week was spent performing a ‘pad validation’ test wherein all systems between the stack and the pad are connected and wrung out. These include all power, communications, data, water, gasses, etc. During this two-day test we performed a confidence run on all three Auxiliary Power Units, as they were replaced in the Orbiter Processing Facility and needed to be tested “outside” and prior to launch day when they’d be needed to power the hydraulic system pumps. Also done this first week were all the preparations for loading the hypergolic fuels for the thrusters, Orbiter Maneuvering System engines, and topping off the APU tanks. In parallel with the hyper preps we conducted a test of the main propulsion system by pressurizing it with helium and looking for leaks. This test was developed after we experienced a series of troublesome leaks in the late 80s that delayed several missions.

The second full week at the pad was dominated by two tasks – loading the hypergolic fuels and conducting the practice launch with the astronauts participating. Hyper load took two full days as did the Terminal Countdown Demonstration Test, with the simulated T-0 at 1100 EST, Friday, December 20. Both went very well and it was great seeing Rick Husband and his crew one last time before launch itself. (I’ve described the traditional crew dinner during TCDT in another post, by the way.) We took both days of the weekend off.

Monday, December 23 was a day of preparing the vehicle and ground systems for the nine days we’d be home with our families before returning to work Thursday, January 2, 2003. But even though most of us were off those days a very dedicated skeleton crew cared for Columbia every day. Walkdowns, visual inspections, security sweeps were all done as a matter of course. We were all glad to see her in such great shape January 2, now just fourteen days away from launch.

columbia-rollout-ksc-02pd-1883-web-version
Columbia passes the Launch Control Center (right) on the way to Pad A for what would be her final launch. (NASA photo KSC-02PD-1883)

Launch fever

In an earlier posting, I talked about ‘schedule pressure’ and how it should have more correctly been referred to as ‘schedule awareness’. Awareness of the schedule contributed to good, productive decision-making. Pressure-based decisions could be shortsighted or worse, destructive.

What about decisions on launch day itself? Hundreds of subordinate schedules and decisions are about to pay off. Pressure? Awareness? Something else?

There’s a phenomenon in the launch business called “Launch Fever.” It, too, is very real. It is never productive and can be destructive. It’s the #1 thing on a Launch Director’s list to recognize and deal with. What is it? Why is it a big deal? What does a LD do about it?

Put yourself in the Shuttle Firing Room on launch day. A tremendous amount of work by thousands of people across the country is about to pay off. The vehicle is full of fuel. The astronauts are all strapped in, their hard work about to pay off big time. TV cameras are rolling. VIPs fill viewing areas, and tens of thousands of tourists line the causeways around the Cape area. Crew families await their loved ones’ achievement. Blue jeans have been replaced by suits and dresses on launch team members. Teams at other NASA and ISS centers are listening. KSC is in full launch mode with security tight on the ground, sea, and in the air. The final schedule achievement is about to be realized. We are about to launch the Space Shuttle. Everyone wants to launch. Our business is to launch. That’s Launch Awareness.

Enter a technical problem, weather issue, fouled Range, or any one of a thousand reasons to question whether we should launch or not. There’s a definite feeling, a palpable mood that sweeps over the collective team. For a while, good decision-making becomes secondary to the desire to launch.

Imagine that: Good decision-making becomes secondary to the desire to launch. Real? It absolutely can be. That’s Launch Fever.

What to do about it? After recognizing that it’s happening, which sometimes can be tricky in the first place, the #1 job of the Launch Director kicks in: Keep that Shuttle on the ground. As the last person to say “GO” it’s incumbent on the LD to have absolutely no reservations to launch. Zero. So even in the face of everyone else wanting to launch by saying GO, my job was to NOT launch the Shuttle unless and until I was ready.

Pressure? Only if you allowed it. Awareness of it was mandatory.

I was asked once what would have happened if on launch day I said NOGO in the face of everyone else being GO. Part of my training, and the training I gave each of my assistant LDs, goes like this. “That’s easy to answer. On that day we would not have launched. The astronauts and the vehicle would be on the ground and and they would be safe. NASA might have a new Launch Director the next day, but on that day we would have been safe.”

I never had to do put that training to use, as good decision-making always prevailed.

exp-nr-endeavor-launch-delayed-cnn-640x360
The final launch of shuttle Endeavour, STS-134, is scrubbed on April 29, 2011 due to problems with heaters on the vehicle’s auxiliary power units. President Obama and his family were on hand but unable to see the launch because of the scrub. (CNN photo)

Schedule pressure

I was asked in a press conference once if ‘schedule pressure’ had affected my decision for a launch. I wanted to throw the reporter out of the room, but I liked my job. Schedule pressure affecting a decision? Instead of a tirade, it was clear the reporter needed a little refresher in the basics of Project Management 101. Here it is:

Work scope. Budget. Schedule. Magic, huh?

Numerous times in the Shuttle program we were accused of letting the schedule affect some decision made. News flash—of course it did! What the reporter was really asking was is if the schedule made me make an unsafe decision. If he had actually asked me that way, then I would have thrown him out.

My job, and indeed the jobs of ALL the great people working so hard to put our astronauts into orbit, was to “get it done, safely.” No exceptions, no excuses for anything less. I found that the suggestion that it was otherwise to be appalling and personally offensive. But we heard it from time to time, and not just from the press. Indeed, to their defense, they had to ask it. Doesn’t make it any more friendly, but it was their job to push for answers. I much preferred the phrase ‘schedule awareness’, and that was how I answered the question. Schedules were just as real as money and scope. But none of the three trumped safety.

Of course we got the “how much did it cost?” question, and much more frequently than the safety question, fortunately. I always liked the cost question because that never drove my decisions, much to the dismay of our program office. My own view was that we had enough money to get the work done to the agreed upon schedule, and a little more. After all, the vast majority of our costs at KSC were labor, so we met the schedule based on the size of the workforce driven by allotted budget. Magic again. More money would have permitted more people working more shifts no doubt, but absent that, we got by with what we had.

Work scope was driven by essentially three factors. There were the normal systems tests, checkouts, maintenance, etc., that were required just to maintain a healthy, functioning program. There was scope, driven by failures and therefore some needed repair or replacement. And then there were system upgrades over the life of the program to modernize and improve safety margins. These could range from relatively minor enhancements, such as upgrades to the slidewire emergency egress system, to major flight hardware improvements, such as upgrades to the Space Shuttle Main Engines aimed at reducing the chances of a shutdown during ascent.

It was during these debates when scope growth was weighed against cost and schedule. And—virtually every upgrade decision made was based on an improvement to some safety margin, or it wouldn’t have made it to the table for discussion in the first place. Some ‘operability’ upgrades were accepted, but I would argue that these had their roots in improving safety by allowing better and/or easier user-to-system interaction.

Project Management 101. Simple. But in manned spaceflight, Safety was the overarching requirement—inescapable and thoroughly embraced. It wasn’t ‘the fourth factor’—it was the paramount factor, not often taught in the typical PM 101 course.

sts133-s-065
Mike Leinbach gives the Go for launch of Discovery  on mission STS-133, February 24, 2011 (NASA photo STS133-S-065)

Astronaut families on launch day

Put yourself in the role of an immediate family member of an astronaut on Launch Day. What do you do? What do you feel? Where are you? Will your wife, husband, or parent be OK?

Imagine the multitude of thoughts and emotions as the most important person in your life is about to put her or his life on the line, literally. Do you think about Challenger? Columbia? Apollo 1? Of course you do, but you don’t allow yourself to be consumed by them either.

If you’re the parent of a youngster, what do you do to assure your child that mom or dad will be fine? And how do you spend the final several hours of countdown waiting for liftoff?

I’ve spoken to lots of astronauts over the years and it comes down to “duty supersedes danger.” Easy to say as adults, not so much for kids.

So what did we do to help the families and their children on launch day? Several things evolved over time as good to do and they became standard protocol—a tradition, if you will. First though, a brief understanding of the timeline for launch might help.

The astronauts usually arrived at KSC three days before launch, as did their families. Being quarantined for about seven days before a mission necessitated the crew and families staying at different locations. The astronauts stayed on base at the Crew Quarters in the Operations and Checkout Building, about eight miles from the launch pad. The families would stay in local motels. Their paths crossed only in phone calls or other electronic ways. The next two days would see the crew get into final training for the mission and the families preparing for a very momentous day.

Launch day for the crew started approximately seven hours before launch and after fueling of the External Tank had begun. Breakfast and suit-up followed. By about three hours before launch, they’d be strapped in the Shuttle awaiting liftoff, as final systems checks continued from the Firing Room.

The immediate family members—wives, husbands, and children—would arrive at the Launch Control Center about four hours before launch and take up residence in my office on the fourth floor, just above the control rooms. Each family would have their own personal veteran astronaut assigned to them to explain what was happening, answer questions, etc. Protected from the prying eyes of the press in this private setting allowed them to be alone with each other and their thoughts.

But how do you entertain the youngsters for the long three to four hours of waiting? The ‘Kid Pic’ was born in the earliest days of the Shuttle Program as a fun way to help each child pass the time and think positive thoughts about what his or her parent was about to do.

The Kid Pic was hand drawn in full color by the astronauts’ children on a 3’ by 5’ white board in an adjoining but separate office. It was a great way to allow their parents even more privacy and occupy their children. Each drawing was unique, inspired by their pride in their parent and limitless imaginations. After launch it was framed and protected behind plexiglass and hung in the hallways of the LCC.

As the number of Kid Pics grew with each mission conducted, we began displaying them in other operational areas of KSC. Artistic prowess wasn’t required, and none were judged for it!

 

sts-107-kid-pic
The STS-107 Kid Pic, drawn by the children of Columbia‘s crew (photo by Jonathan Ward)

As liftoff approached, the families with kids in-tow would be escorted to the roof of the LCC to experience launch from a truly great vantage point. Once their loved ones were safely in orbit, their day of overwhelming pride and unbelievable stress was essentially over. They would return to their motel rooms or head directly back to Houston to watch the crew perform on behalf of the country as the mission unfolded.

Launch was a success, but true celebrations would have to wait for their reunions as families again on landing day.

view-from-lcc-roof
Panoramic view of LC-39 from the LCC roof (Photo by Jonathan Ward)

Manned spaceflight traditions

Traditions play an extremely important role in almost all aspects of life. Think of your work, your personal doings, your religion. Think about how your parents raised you, and how you raise(d) your kids. Think of politics—well, OK, don’t! Traditions are everywhere, and deservedly so.

Traditions in the American manned spaceflight business run deep. From those that were born in the Mercury program—like the ceremonial “Farewell” words from the capsule communicator to the astronaut on board the rocket—to the post-launch beans and cornbread of the Shuttle program, they are ways to celebrate, connect to the past, and look to the future.

Continuity. Celebration. Reverence.

Of all the traditions, my personal favorite was the pre-launch ‘quiet time’ shared by a lucky few at KSC with the crew that was just about ready to fly into space. We did this during TCDT Week. The Terminal Countdown Demonstration Test was, on the surface, a dress rehearsal for launch. The astronauts and the launch team would conduct an exercise as close to launch conditions as possible to prepare for the real thing. After all, no rookie astronaut or console operator should experience launch ‘cold’, right?

The three days leading up to the simulated launch was time to train in emergency systems, get technical and operational briefings, and have the ‘quiet time’, also known as the TCDT crew/management dinner. Always conducted at the famed “Beach House” on the Atlantic coast, it was a chance for the astronauts to get to know the folks who prepared their spacecraft and payloads, and vice versa. Of critical importance, and an intended consequence, was the opportunity for them to get to know and personally thank a fraction of those people whose work could directly affect their lives.

With a group of about fifteen folks at the dinner, representing the thousands of KSC workers, the Orbiter Test Conductor went from being “OTC” to being “Roberta.” The Director of Payload Processing was now known as “Steve.” And the Launch Director became “Mike.”

Just a name change? NO WAY. We went from ‘Cape guys’ to trusted colleagues and friends very quickly.

The meal itself was a tradition: barbecue chicken, beef brisket, smoked sausage, baked beans, cole slaw, potato salad, tossed green salad, buttered and toasted Italian bread, a dish of sliced jalapenos and other condiments, and brownies for desert. The wonderful ladies at crew quarters would prepare the food, the crew would provide the “refreshments”.

Talk was sometimes about the mission but mostly about families, colleges, hobbies, and the like. The time passed quickly overlooking the ocean and mementos of previous missions. It was by any measure a time for fellowship, some reflection, and certainly wishes for success.

(It should be noted that the astronauts routinely went around the KSC processing facilities thanking as many of the technicians, engineers, and office workers as possible as they were going about their business. Seeing a Blue-Suiter was much more common than rare.)

Many other traditions surrounded Shuttle’s TCDT Week, Launch Week, and Landing Day, and I’ll get to them in time. Many can be traced back to the 1960s when similar groups had similar interactions before manned missions.

We owe a tremendous amount to those that truly blazed the trail and left us with enduring and deeply meaningful traditions. I’m certain many will survive the transition to the next manned programs. They should.

(Hear Mike tell the story of the STS-107 Columbia crew dinner in this “Untold Story from the Rocket Ranch“.)

beach-house
The Beach House (NASA photo)

STS-107 and the ISS

In the early 2000’s the Space Shuttle program was deeply into the construction of the International Space Station, spurred on by America’s commitments to its sixteen partner agencies and countries. All components for the ISS, less those from Russia, were designed to be launched and serviced by the Shuttle. No other launch vehicle had its capacity or its capability. The ISS and Shuttle were tied at the hip, mutually dependent, fulfilling one of the early goals of the Shuttle program itself.

And, buoyed by the string of extremely successful missions following the Challenger accident in 1986 NASA was planning to extend the Shuttle program through 2020.

We at KSC were doing our advance planning to do major overhauls and upgrades of the four Shuttles. These Orbiter Modification and Development Periods (OMDP) were as important parts of the launch manifests as the missions themselves. Originally performed at the shuttle’s manufacturing site in Palmdale CA, they were now all to be done at KSC to save money and time.

Good decision, to be sure, but it had one unattractive impact on us. We had three Orbiter Processing Facilities (the hangars) but four Orbiters to deal with. That meant one Orbiter had to be at one of three other locations at any given time – in space, at the launch pad, or in the Vehicle Assembly Building. No way around it. (Had one landed at the Edwards AFB in CA it still would have been brought back to KSC after approximately a week.)

So how did this affect STS-107 and Columbia? With the emphasis on the ISS assembly missions, Columbia’s pure science mission was usually the one to be delayed when the higher priority ones needed something, anything. This included an OPF. Columbia was moved around KSC fairly often, always causing its scheduled launch date to be postponed.

STS-107’s launch date slipped thirteen times between when the mission was originally announced in 1997 and when it finally flew in January 2003. Flowliner cracks, wiring problems, and even unexpected problems with the Hubble Space Telescope de-prioritized STS-107’s science research flight and caused mission delays.

In response to Congress’ concern about delays and cost growth on the ISS, NASA committed to getting the Unity module (Node 2) to the ISS by the end of February 2004. That would essentially complete the installation of the US components that were on the critical path to bringing the Station up to its scientific potential.

It’s important to note that since Columbia was the heaviest Orbiter, it couldn’t launch to the ISS inclination without significant decrease in payload weight and giving her a new airlock. And NASA initially did not intend to fly her to the Station. But as of 2002, the delays caused by the issues mentioned above meant that Columbia needed to be pressed into service for ISS assembly if the February 2004 date was to be met.

Immediately following STS-107, Columbia was scheduled to go into an OMDP that would have seen modifications to lose weight to fly a single ISS assembly mission. And ironically, part of her weight loss would have included stripping out much of the MADS/OEX system—the “black box” that was so important in helping to pinpoint the accident cause in 107.

The interesting thing about launch slips or priority-induced effects on processing plans is that the KSC workers became used to them and it caused little more than temporary disappointment. It was simply part of the business.

Rick Husband and his crew had the same response. He and I talked about it once. Again, it was simply part of the business of space flight.

Eventually NASA settled on an achievable launch date for the 107 mission and we met it. Its outcome had nothing to do with the delays. Not a thing.

ksc-02pd-1196-columbia-under-tent-in-vab
Columbia under a tent in the VAB, awaiting space in an OPF (NASA Photo KSC-02PD-1196)