Three STS-107 Tribute Videos: Part 2

Today’s offering in our reflection on the fifteenth anniversary of the STS-107 accident is a music video tribute to the crew.

A Tribute to the Crew of Columbia: Sixteen Minutes from Home: Set to an original song by Kyle Breese, this video helps us reconcile our grief at the loss of the crew of STS-107 with the knowledge that the crew was doing what they loved best. We see footage of the crew enjoying themselves during their flight preparations and mission. At the end, the song reassures us that “at last you are truly home.”

crew on orbitWhile there are no graphic images of the accident in the video, please be aware that the content may provoke strong emotions.

Please feel free to share this video, but note that it is intended for noncommercial use only.

Tomorrow: Bringing Columbia Home: Recovery, Reconstruction, and Return to Flight

Three STS-107 Tribute Videos: Part 1

This month I assembled three videos to memorialize the crew of STS-107 and to recognize the sacrifices of those who fought so hard to bring Columbia home and prepare the shuttle fleet to fly again. These videos were first shown publicly at the “Columbia: Lessons and Legends of Recovery” event at Kennedy Space Center on January 26.

I understand that versions of these videos may be officially shared later on, but I wanted to give you the opportunity to see them this week as we reflect on the crew of STS-107 on this 15th anniversary of their loss.

I’ll share the videos one at a time for the next three days. Although I hope the videos stand on their own, I’ll give a little preview of what you’ll see.

The videos are not subtitled, so you’ll need to have your sound turned on when you watch them. Also, please bear in mind that making these videos was the first time I’d ever used iMovie, and I am far from being a professional video editor!

Today’s video is: STS-107 Preparation and Mission.

The crew of STS-107 discuss their excitement about the upcoming mission. Launch Director Mike Leinbach wishes the crew “Good luck and Godspeed,” and Columbia launches into a beautiful blue sky. We see some highlights of the crew’s on-orbit activities. The six-minute video fades to black with Mission Control’s calls to the crew during re-entry.

Tomorrow: A Tribute to the Crew of Columbia Mission STS-107.

Columbia ascent

Tropical Storm Ernesto and the Half Rollback

Every now and then, a little creativity goes a long way…….

As noted in my previous post, we were charged with protecting the Kennedy Space Center workforce and the hardware. But of course, we were in the business to launch the Shuttle, not stay on the ground indefinitely while maximizing the safety statistics. The basic tenet for all of us was “Get work done, safely.” Apply that to all aspects of Shuttle processing and launch. Meeting the manifest was important. Launching was important. Doing it safely was paramount.

Decisions that could affect the manifest required me to consult with the Shuttle Program managers at Johnson Space Center (JSC)—and for good reason. The “Program” was “theirs”, not “mine”, or KSC’s. So for major decisions like rolling back for storms two people needed to agree: the Launch Director (me, at KSC) and the Operations Manager (the JSC manager stationed at KSC). We conferred and jointly made those major manifest-impacting decisions.

Flashback to late August, 2006. Atlantis was on the pad preparing for STS-115’s assembly mission to the ISS. Atlantis was originally scheduled to launch on August 27. Our ground team made an unprecedented replacement of some parts on the shuttle with a week to go until launch, and we maintained the schedule. Then on August 25, one of the most powerful lightning bolts ever recorded at Kennedy Space Center hit the lightning mast at the top of the launch pad’s Fixed Service Structure. We needed at least 24 hours to assess the damage from that strike, moving the launch date to August 29.

That was all trouble enough. But meanwhile, a minor tropical weather disturbance had been forming in the Caribbean. Unimpressive—but like all such systems, we would monitor it in case the unexpected happened and it became a threat to us.

True to form, after days of watching the storm, now named Ernesto, its track and forward speed appeared likely to affect Florida’s Space Coast while Atlantis was still preparing for launch.

Decisions were coming, no doubt. We began the initial work to roll the shuttle back to the Vehicle Assembly Building as a precaution. This work could easily be reversed and still hold the launch date if the storm took an unexpected turn away from us. Discussions with the 45th Space Wing meteorologists were now being held every six hours to coincide with the official forecasts from the National Hurricane Center.

As Ernesto crossed western Cuba and headed into the Florida Straits, it grew to hurricane strength. The frequency of our calls went up to the rare every three hours, and then almost continuously. Ernesto was on a track to enter extreme south Florida and head up the eastern side of the peninsula, essentially right at us.

Meanwhile, Atlantis had a launch date that was sandwiched in between several other arriving and departing vehicles at the International Space Station, other Eastern Test Range operations, and other constraints that were beyond our control. If we didn’t launch before about the 12th of September (as I recall), we would have to stand down for another two weeks or so. Schedule awareness (not schedule pressure) was real.

The decision was coming. Stay at the pad and risk exceeding our wind limits and possible Shuttle damage? Or roll back to the safety of the VAB and miss the near term launch opportunity?

After numerous tense calls with the Launch Weather Officer, LeRoy Cain (the JSC Ops Manager) and I made the decision to play it safe and roll back. It was the morning of the day before the storm’s predicted arrival, approximately 40-44 hours hence. Rolling early enough to beat the winds was the game we needed to play.

But what if the storm changed course and became less of a threat? That became a real possibility after the wheels were already in motion.

As the ground operations team was rolling Atlantis off the pad for its eight-hour trip back to the VAB, the question came to me: Could we reverse the roll and return to the pad IF the storm really did weaken and veer off course, permitting us to stay at the pad?

The Crawler-Transporter (CT), built for Apollo, went in only one direction: forward. Forward toward the pad, and forward toward the VAB. What? The CT had two control stations—one on the east face, one on the west face. It was from either of these “cockpits” that the crew drove the vehicle. There was no way to turn that mammoth vehicle around on the crawlerway. To go the other way, you’d just stop, get out of the western control cab en route to the VAB, get into the eastern cab, and then start driving again, this time toward the pad.

I asked the Support Test Manager if his guys could do this. He went pale. Never before even contemplated, no procedures allowing it, the time required, etc., etc. all indicated a negative response was likely. But Bobby Briggs, being the best STM at KSC, said he’d “look into it.” His eventual answer was that if we decided before the CT was halfway to the VAB, his guys could do it.

Perfect answer.

Armed with that, I went to LeRoy Cain to see what he thought of the idea. He liked it immediately. If the storm veered away in the next four hours or so, we could stop and return to the pad.

For the sake of brevity, here’s the punchline. We did stop the roll and went back to the pad following the “final” call with the LWO, Kathy Winters. The storm would start to weaken coming up on its overland track, AND the track had it going a bit farther to our east. Atlantis would be on the best side of the storm.

As all of us watched the storm progress up the state from the safety of our homes. The favorable track was verified. It passed to the east of the pad by approximately 40 miles, as I recall. Winds remained within limits and no damage resulted.

We launched successfully Atlantis on September 9, 2006. Astronaut Brent Jett commanded the mission. He had been an integral part of the crew recovery effort after the Columbia accident.

As I look back on this achievement, I can’t help but think about this being just one example of team creativity and their can-do, will-do approach to all operational challenges. I can’t put into words how proud of them I was, and am to this day.

This ‘partial rollback’ was needed to do two things – protect the vehicle, and, because of the way things turned out, preserve a launch opportunity. Get work done, safely.

Damn, it was fun!

Hurricane Season and Rollbacks

June 1 thru November 30. The Atlantic Hurricane Season. As June 1 passed I was reminded how much ‘fun’ it was to experience.

From the earliest days of our manned spaceflight programs, the Launch Director was responsible for two things that outweighed all others, including launch itself. These were to protect the safety of the workforce, and to protect the flight and ground hardware. Obviously, these are everyone’s responsibility, and everyone took them very seriously. But when major processing decisions were required that had significant safety and/or processing implications, the LD made the final call.

Some of those decisions are well-known, the final decision to launch being the most obvious. Others were also the responsibility of the LD but not as visible. The decision to roll the vehicle out to the launch pad, establishing and enforcing the employee work time rules, and approving personnel exposure to launch pad hazards after external tank (ET) fueling are a few. The decision to roll the vehicle back to the Vehicle Assembly Building (VAB) from the launch pad was also given to the LD (along with a Shuttle Program rep that I’ll explain later).

What could drive us to roll back? Remember the woodpeckers and the damage they did to the ET that couldn’t be fully repaired at the pad? (That was STS-70, in June 1995.) How about the hydrogen leaks in the 1990s?

There was another event that could force us to bring the stack back to the VAB for safekeeping – threatening hurricanes.

If you recall, the vehicle spent about a month at the pad in preparation for launch. Unlike expendable rockets that can spend as little as one day at the pad, the Shuttle needed quite a bit more preparation time before launch. Payload installation, ordnance installations, hypergolic fueling, TCDT—those are just four of the numerous pre-flight jobs that had to be done at the pad. They were required for every mission regardless of the calendar.

Enter Hurricane Season. The Space Shuttle would spend that month of prep time on its seaside launch pad, less than ½ mile from the usually tranquil Atlantic Ocean. On occasion, that tranquility would be broken by the effects from tropical storms and hurricanes. If the Shuttle’s presence on the pad and tropical weather coincided, tough decisions would be required.

Protect the hardware. Protect the people.

The Shuttle program benefitted from the weather forecasts and advice from some of the best meteorologists in the world. The 45th Space Wing of the US Air Force provided weather support for us. Every day—not just during hurricane season, but every day—I would hold a 10-minute call with them for the daily forecast. Numerous processing groups would tie in to benefit from the information and how it might affect their work plans that day. If tropical systems began to form, the intensity on the calls would increase. If there were also a Shuttle at the pad, it would take on an additional air of importance and urgency. Daily calls would increase to twice per day, then every six hours, or even more as the threat got closer.

Ultimately, if the storm track, intensity, and speed combined to actually threaten KSC in the near future we would need to roll the Shuttle back to the VAB to ride out the storm. We could hunker down at the pad if the winds didn’t exceed 60 knots. But if the forecast had higher winds, we needed to get to the safety of the VAB, and this needed to be done early enough that allowed the workers time to get home to deal with their own, final storm preparations.

A lot went into those discussions, but for this entry it can be summarized as balancing the desire to stay at the pad and launch on time vs taking the protection option and delaying launch at least two weeks. But remember: the top priority was safety for the people and hardware. Launch schedules were secondary. But the balance needed to be struck, and it was the responsibility of the Launch Director and the Shuttle Program Manager (delegated to his KSC rep) to do it.

Why would it delay launch at least two weeks if we rolled back to the VAB? The three sets of required tasks, when combined, needed about two weeks to complete—rollback preparations including the roll itself, stay time in the VAB as the storm passed by, and then roll back out and re-perform pad operations undone in Step 1.

To get ready to roll back was at least two full days of activity—usually three—and was done with the storm bearing down on us and our families and homes. How long it took to prepare for rollback depended how close to launch we were when the decision to roll was made. The closer to launch meant more work had been performed that needed to be undone. The Payload Bay Doors needed to be closed after securing the payload. The aft compartment of the Orbiter needed to be closed with special doors, the side hatch closed, etc. Add to these relatively obvious preps things like hypergolic and ordnance systems securing, Pad-to-MLP disconnections (power, comm, gases, data lines, etc.), and you get the idea. The “final” disconnection was this contraption called the ‘9099 interface’ – a large bank of data and power lines on the side of the MLP. Once disconnected, all the work done to verify those systems were launch-ready became history—work that would have to be completely re-done once the Shuttle came back to the launch pad.

We had to roll the crawler out to the launch pad. The rollback itself was typically eight hours or so followed by rudimentary connections of the MLP to the VAB shore systems. Then the final workers could go home and shutter their houses.

Stay time on the VAB was storm-dependent of course, but let’s say three to four days until the “all clear” was declared to reopen KSC. If storm damage existed that would prevent normal work, add that repair time.

The workforce returned to work and got ready to roll back out to the pad. Two days minimum to get ready. Roll out was the same eight hours, and usually at night in the summer to avoid the afternoon thunderstorms. Once at the pad, you have to reconnect everything (reconnecting each interface means going through the complete set of checks to ensure that the connectors are properly mated again) and get back into the same posture as before the decision was made to ride out the storm in the VAB. Add back those three days or so. Then pick a new launch day, verifying that the new launch date doesn’t impact other planned launches at the Cape, other vehicles arriving at or departing the International Space Station, and so on.

All told, the decision to ride out a storm in the VAB meant an impact to the launch schedule of about two weeks.

No magic, just a lot of (necessary) work to protect the flight hardware.

Next week: Tropical Storm Ernesto and the “Half-Rollback”

The Hidden Cost of Reusability

All space flight providers want to control costs. From the traditional government and commercial enterprises to the new entrants, all want to keep costs as low as reasonably possible. Gone a long time ago—and forever—are the days of unlimited resources to get the job done. Bottom lines matter, and matter a lot.

It has long been argued that one way to keep costs reasonable is to reuse launch hardware. Even when adding the required refurbishment/reflight checks, repairs, component replacements, etc., to the equation it can be shown on paper to save money in the long run. The Shuttle was sold to Congress in part on this philosophy. And being the first reusable spacecraft they could know no different.

Nor could NASA fully know the long-term implications of reusability.

An additional fact is this: Adding astronauts to the equation requires changing ‘reasonably’ to safely. No argument there, right? But what are the implications of this change? Simply put, based on Shuttle experience, it requires lots more of the checks, repairs, component replacements, etc., in an attempt to make the re-flight as technically close to the first as possible. The newer the hardware, the less risk of failure. No argument there either.

These costs are easy to understand and accept. What about the posting’s title? Hidden? I have alluded to it in earlier postings and it’s not just a financial issue. It can cost a lot more than money.

Here it is:

With the desire to refly hardware that has (theoretically) performed well before, a tendency to rely on past performance when approving current flight rationale is given more weight than it should.

That tendency can either be seen overtly, or it can be latent. If obvious, it can be dealt with through open discussion and full debate of the issue at hand. If hidden, it can fester, grow, become a ‘new normal’—or worse, lead to disaster.

Examples in the Shuttle program are numerous. From those we recognized and dealt with—wire chafing, flow liner cracks, hydrogen leaks—to those that dealt us the most severe of consequences—foam shedding, O-ring scorching—we were challenged by the need to re-fly hardware, all with the overarching tenet of doing so safely.

Tragically, we didn’t always balance that correctly.

I sincerely hope today’s and tomorrow’s astronaut launch providers succeed 100% of the time. Recognizing when a ‘new normal’ is kicking in is part of the success criteria.

spacex-falcon-9-crs-9-landing
SpaceX Falcon 9 first stage returns to Earth after the CRS-9 launch. (Photo courtesy SpaceX)

Launch Countdown

Why was a Shuttle launch countdown three days long? The answer to this question has roots back to the ‘elders’.

Those of us that were part of the final years of the thirty-year operational life of the Space Transportation System inherited some truly thoughtful processes from those who came before us. Some processes were born as early as the Mercury program, nourished through the Gemini and Apollo days, and adapted to the Shuttle needs.

In architectural design, it’s called “form follows function.” Ditto for many of the ways we processed hardware and launched our friends into space.

Granted, some early concepts for a new adventure like this didn’t pan out. The good ones did, and they live on today in virtually all launch system designs. I was asked to design the launch team for the Constellation program in the 2005 timeframe. As part of that effort a small group was formed and we benchmarked numerous high-power teams looking for best practices (and worst!). It amazed us how closely other successful teams resembled ours. Why? Because it worked, and worked well. Period. Nuclear subs, unmanned rockets, ESA and Russian space programs, emergency response authorities, aircraft flight testing, and others, had teams very, very similar to ours. But I stray…

Three-day long launch countdowns? The basic reason was it was easier and more effective to control the myriad of tasks in those three days under one governing, integrated procedure than had they all been conducted as individual, stove-piped operations. Integrating all the tasks under one orchestra leader (the NASA Test Director) allowed for better command and control, and visibility for managers like me. The NTD would integrate and lead all the element Test Conductors, ensuring no conflicts, or overlaps, or omissions.

What tasks? In the simplest form they were these. On the first two days:

  1. Pad closeout and securing
  2. Fuel Cell cryogenic reactant loading
  3. Communication system activation
  4. Flight Crew equipment stowage
  5. Rotating Service Structure retraction to its launch position.

On day 3—Launch countdown day:

  1. External Tank fuel load
  2. Astronaut boarding
  3. Launch.

That’s basically it. Within these major tasks were hundreds of steps , but that was about it.

This basic design was created by my predecessors and not changed much at all for the 135 Shuttle flights. Norm Carlson, Apollo Launch Vehicle Test Conductor (the forerunner to the Shuttle NTD position) knew it would work. His compatriots knew it would work. We inherited it and it still worked.

Like many of the other processes we (the “late bloomers”) inherited, we enjoyed ones already tested and proven successful.

To Norm, Tharpe, Tribe, Horace, Page, Sieck, Fuller, Breakfield, and so many, many others—Thanks.

shuttle_countdown_sign
Launch countdown sign at Kennedy Space Center. (source: The Register)

Getting Ready for Launch Countdown

Fourteen years ago this week we were doing our final preparations on Columbia and the ground systems, getting ready to enter launch countdown (LCD).

With launch scheduled for January 16, the 3-day countdown was to begin Monday, 1/13. After dusting off vehicle and ground systems (and ourselves!) last week following our 9-day holiday period vacation, we were conducting several final tests that weren’t in the launch procedure but needed to be done as close to it as possible. On the schedule was installing the ordnance items, pressurizing the hypergolic system, checking out the space suits the crew would use if a spacewalk became necessary, doing some crew equipment and Spacehab installations, and closing the aft compartment of the Orbiter. Assuming this all went well, we’d have the 2-day weekend off just before LCD.

Ordnance installation and hypergolic fuel system pressurization were major hazardous operations requiring the pad to be cleared of non-essential personnel. And once done, they put the vehicle in a somewhat higher hazard level so we did them as late as practical. Access to the pad was more tightly controlled afterward. Dozens of explosives were used on the Shuttle and ground, mainly in separation sequences. The solid rocket boosters needed to be separated from the mobile launch platform, the SRBs from the external tank, the Orbiter from the ET, etc. All were done with ordnance firings. Installing them and verifying proper connections was an extremely precise skill. Our guys were the best at it.

Pressurizing the hypergolic propellants was necessarily done late in the pad flow to expose the least number of folks to that state following the pressurization. Hyper propellants—ones that ignite spontaneously when combined vs. needing an ignition source—were actually loaded well before the holidays. Hypers are stored at room temperatures, unlike the cryogenic liquid hydrogen and oxygen in the ET, but need to pressurized with nitrogen and helium gasses to get proper flow rates. Hypers were used in the thrusters, the auxiliary power units in the hydraulic system, and OMS engines.

Closing the aft compartment may not sound like a big deal, but it took several days to inspect each and every system, remove protective covers and access platforms, clean things up, and do other detailed work. It was an arduous process. It usually turned up a few items that needed to be addressed before we could have full confidence we were ready to go. We allotted 4½ days to fully inspect and prepare the multi-level aft compartment.

Checking out the space suits was fairly straightforward. Communications, power, and breathing systems were checked, as were all parts of the suits themselves. There were no scheduled EVAs in Columbia’s research mission, but we always provided two suits in case it became necessary. Every crew practiced an EVA to close and latch the payload bay doors if they wouldn’t close automatically. The Orbiter couldn’t reenter with those doors ajar!

We also did some “early stow” inside the Spacehab—experiments that weren’t time critical, different supplies, etc. Sounds easy—but remember the vehicle is vertical at the pad, and so is the Spacehab. Getting into it from the Crew Module was both tricky and ingenious. It involved a Rube Goldberg contraption consisting of tripods, hoists, wire rope, bosun’s chair, and nerve. The technician would be sitting in the chair, lowered down into the module via the wire and hoist system, dangling in free air all while installing stuff in lockers on the sidewalls. Easy, huh? In our jargon, the MVAK, the Module Vertical Access Kit, was neat to watch but a nightmare for the guys. But it was also the best thing we had to get the job done vertically. By the way, it would be left in place after using it this week. It would be used again during launch countdown to stow the late, time critical experiments – live worms, frozen samples, etc.

All this work went as scheduled so we had Saturday and Sunday off. Monday morning, however, we’d enter our Launch Countdown procedure and really act like launch was coming!

Next time, why does it take three days to launch the Shuttle?

mvak-diagram
Diagram of Module Vertical Access Kit (MVAK) operation used for a Spacelab module on STS-51B. KSC had two teams trained in MVAK operations. (NASA diagram)