Emerging Hazards in Commercial Aviation—Report 2: Ensuring Safety During Transformative Changes (2024)
Chapter: 5 Integrated Safety Management Across the Lifecycle and Across Organizations
5
Integrated Safety Management Across the Lifecycle and Across Organizations
The tasks of identifying, monitoring, understanding, and addressing emerging aviation safety risks cannot be fully achieved by only looking at separate aspects of the National Airspace System (NAS) and by applying sequential processes. The previous three chapters considered separately the details of the three aspects of Safety Management: safety risk management (Chapter 2), safety assurance (Chapter 3), and organization processes and culture (Chapter 4). However, safety is only effectively managed when these aspects of safety management are purposefully coordinated, and when they are integrated across the many organizations involved, the subject of this chapter.
First, the committee considered how safety now routinely spans multiple organizations at every stage of the lifecycle. During design, multiple different components from different suppliers may be assembled, and aspects of analysis may be contracted out to specialists; later design modifications may be conducted by yet other organizations (e.g., supplemental type certificates by third parties). During production, the “prime” manufacturer may outsource production of components, and supply chains may depend on parts and services from a fluid and evolving network of many third-party organizations. In such cases, safety requires more than just “quality assurance” by third-party suppliers to actively manage concerns with, for example, collectively modifying and evaluating component safety on the production line.
This need to coordinate multiple organizations also is important during the day-to-day operation: a “prime” organization such as a major air carrier may interoperate with many code-share and regional partners, and aircraft operators are supported by separate organizations providing safety-critical
functions such as cargo handling, maintenance, and dispatch. The committee learned of even more transformative changes in how multiple organizations may interact from an industry presenter at a workshop, who is proposing to create a third-party service provider that, even during the flight, provides autonomous aircraft with time-critical data and calculations such as revised flight plans. In cases such as this, sensing and decision making currently assigned to a pilot and onboard systems will be distributed via communication links and cloud computing to span onboard systems, a remote pilot, a central operations control, and support service organizations outside the aircraft operator’s organization.
An example of a gap between “prime” and supplier was in identified in the National Transportation Safety Board’s (NTSB’s) accident report for the 1996 accident of ValuJet 592,1 in which a contractor, SabreTech, delivered 144 expired chemical oxygen generators without the proper caps over the generators’ firing pins and improperly labeled in a manner interpreted by ValuJet workers that the canisters were empty and safe to transport. The canisters activated early in the flight and created an intense fire in the cargo hold, which ultimately burned through control cables, causing the aircraft to lose control and dive steeply into the Everglades. The NTSB determined that the probable causes of the accident included, first, SabreTech (for its overtly wrong actions) and, second, the failure of ValuJet to properly oversee its contract maintenance program, with the recommendation that “Part 121 air carriers’ maintenance functions receive the same level of Federal Aviation Administration (FAA) surveillance, regardless of whether those functions are performed in house or by a contract maintenance facility.”
This issue surfaced as a concern with established industry players. Even as the committee was examining it as a potential concern with new entrants in this study, very public displays of the impact of poorly coordinated outsourcing came to light with the Boeing 737 Max 9 door plugs. In this instance, significant components of the fuselage were manufactured elsewhere and shipped to the final assembly plant. Some of them arrived damaged; the repairs required removing critical bolts to access the damage, and safety required that these bolts be properly re-installed before the entire door plug was installed on the aircraft. In this transition between supplier and lead manufacturer, critical knowledge was dropped.
These concerns highlight that the prime organizations can outsource parts and labor, but they cannot outsource the safety risk these arrangements may bring to their product or operation. In all cases, risk may be added by any entity contributing to the product or operation if not systematically managed, and risk can be added by poor transfer of knowledge between the entities or poor coordination of their activities; either way, the
___________________
1 See https://www.ntsb.gov/investigations/AccidentReports/Reports/AAR9706.pdf.
resulting safety concern is an aspect of the assembled product or combined operation driven by the prime.
Safety risk management with out-sourcing in design and production raises several questions: how can the prime guarantee that delivered components were created according to its standards and integrate properly into the final product? How to guarantee that requisite knowledge (e.g., whether vital bolts were removed and/or replaced during the repair of an out-sourced component damaged in transit to the prime) is properly transferred between prime and subcontractor?
The other major components of safety management, safety assurance and organizational structures and culture, also face similar questions: How can a “prime” operator confirm that it is getting all the relevant data and insight from third-party organizations supporting its operations? Can and should it require the same data from these other organizations, for example, and how should these organizations be involved in analyzing and interpreting this data? If a “prime” works deliberately and pervasively to foster safety within its organization, what can and should it require of their third-party suppliers and service providers?
Finding 5-1: Safety cannot be regulated by examining only the safety management processes of a prime organization that involves third-party suppliers or service providers in support of their design, production or aircraft operation; likewise, it is not sufficient for the regulator to oversee each organization separately. Instead, the prime organization’s decision to involve others, including purchasing their products and services, requires a deliberate, layered approach to safety management that ensures all contributions together comply with their safety risk management and safety assurance processes, and are based on appropriate organizational processes and culture. The regulator has the role in overseeing that this layered safety management is implemented and continuously monitored and used to manage safety across all constituent organizations.
Recommendation 5-1: The Federal Aviation Administration Office of Aviation Safety should establish the personnel, mechanisms, and policies that enable oversight of effective layered safety management of an organization applying for certification, ensuring that this safety management also spans the contributions of those third parties whose products and services contribute to safety. This oversight must ensure that this layered safety management is not only implemented correctly at the time of initial certification but also continuously applied.
A second aspect of integrated safety management addresses how it is deliberately and effectively handled across the lifecycle—that is, safety risk management starting in the earliest stages of design and continuing through manufacturing and implementation, and then continuing into safety assurance during operation (including not only flight but also ongoing maintenance, modifications, and improvements). For many products such as aircraft, this lifecycle can span decades, involve substantial modification to the design and its use in new flight profiles and operations, and involve organizations that are uncoordinated (e.g., operating an aircraft manufactured decades before by a company that no longer exists). The previous chapters noted key touchpoints in safety management at distinct points the lifecycle, including:
- The safety assurance processes applied once a new technology or operation is implemented can be bolstered by knowing the key assumptions and reference points used earlier in safety risk management.
- Furthermore, safety assurance can challenge key assumptions and reference points upon which earlier safety risk management was based, triggering a reiteration of the safety risk processes, which then should inform an update for the plans for safety assurance.
- Throughout, effective safety risk management and safety assurance practices depend on organizational processes and culture.
- Likewise, ongoing evaluations during safety risk management and safety assurance can also be used to measure and inform continuous improvements to organizational processes and culture.
Finding 5-2: Often, the entities that seek to certify products, to certify personnel, and to operate and maintain aircraft are separate, uncoordinated organizations, and they may be active at very different points in time. Safety is bolstered when each entity, at the time of its activity, is expected to capture the data and knowledge that informs safety management by those reasonably expected to use the same technologies or operations after them.
Recommendation 5-2: The Federal Aviation Administration Office of Aviation Safety should identify and characterize the data and knowledge associated at each stage in designing, testing, maintaining, and operating aircraft that then can be useful to safety management later in the life of the product or operation. This data and knowledge should be required to be captured at the time, and later integrated into subsequent activities in support of both safety risk management and safety assurance.
The committee started researching this report out of a concern for safety management in the face of transformative changes in technologies and operations, often involving new entrants. However, the final drafting of this report is colored by recent events highlighting how safety amongst even the most established technologies and players in the industry can be vulnerable to any lapse in safety management. A key question for managing both established and emerging safety concerns the role of the regulator. Unlike the popular conception of FAA as the technical expert who provides final, definitive assessments of a product’s or person’s safety to fly, aviation is such an immense undertaking in scale, compounded by safety also requiring incredible attention to detail, that FAA cannot be expected to directly oversee all aspects of design, production, personnel, and operations. Instead, this chapter posits safety management as the process that FAA should ensure is properly occurring longitudinally across the life of products and operations, and also across organizations.
Finding 5-3: Recent events suggests that AVS may already be challenged to regulate all aspects of the NAS. Transformative changes will further pose challenges with managing new risks. To address both current and likely future technologies and operations will require adequate funding and staffing, and requires this staff has requisite technical expertise across the full spectrum of technologies. Furthermore, AVS staff must have the training and vision to oversee broad safety management processes spanning the life of a product (design, production, and operation), spanning multiple organizations, and considering the organization structures and culture needed in these organizations that they oversee.
Recommendation 5-3: The Federal Aviation Administration Office of Aviation Safety should evaluate its personnel requirements in light of the demands placed on the workforce in identifying and addressing both existing and emerging risks. Emphasis should be placed on expertise required to oversee and evaluate new and emerging technologies and operations, to oversee the transition from safety risk management to safety assurance as new technologies and operations are implemented, to support the maturation of safety culture within the industry organizations it oversees, and to ensure rigorous safety management processes within all the contributing organizations that impact aviation safety.
Henry Petroski, in evaluating the evolution—and failures—of technology in many domains, coined the phrase “to engineer is human” (Petroski, 1992). Even when all is properly regulated and evaluated in best faith to the extent suggested by human understanding, any first-time implementation
represents a new frontier in knowledge in which the unexpected can manifest subtly or suddenly and violently. In aviation, even reasonably small changes to otherwise-solid systems have a history of the unexpected occurring, such as the battery thermal runaways when Boeing aircraft transitioned to more powerful lithium-ion batteries to support increased use of electrical components across the aircraft.
Transformative changes in technology and operations reflect an even larger step-function change in the knowledge that is required to design systems and operations safely, and even to know what tests and monitoring to run. Each transformative change represents a step-change in knowledge—and each of these step changes may have gaps in this knowledge that are undetectable and inscrutable until later, with experience. Thus, it is important to constantly understand that, even after the best safety risk management process before implementation, something unexpected may happen once something new takes flight.
A seminal discussion of high reliability organizations noted:
Perhaps the most important distinguishing feature of high reliability organisations is their collective preoccupation with the possibility of failure. They expect to make errors and train their workforce to recognise and recover them. They continually rehearse familiar scenarios of failure and strive hard to imagine novel ones. Instead of isolating failures, they generalise them. Instead of making local repairs, they look for system reforms. (Reason, 2000)
Finding 5-4: The aviation industry, and FAA in all its roles, should remain vigilant for emerging safety risks as new technologies and operations are implemented—to detect the precursor before it manifests as an accident, to investigate unexpected behaviors and effects to characterize their safety and risk, and to be open-minded and prepared to seek new mitigations to newly identified risks.
REFERENCES
Petroski, H. 1992. To Engineer Is Human. Vintage.
Reason, J. 2000. Human error: Models and management. BMJ 320:768–770.
