6

Findings, Recommendations, and Organizational Roles and Resources

FINDINGS AND RECOMMENDATIONS

Recommendation. In-time Aviation Safety Management. The concept of real-time system-wide safety assurance should be approached in terms of an in-time aviation safety management system (IASMS) that continuously monitors the national airspace system, assesses the data that it has collected, and then either recommends or initiates safety assurance actions as necessary. Some elements of such a system would function in real time or close to real time, while other elements would search for risks by examining trends over a time frame of hours, days, or even longer.

Finding. Challenges. Successful development of an IASMS will require overcoming key technical and economic challenges:

• IASMS Concept of Operations and Risk Prioritization
  • IASMS Concept of Operations. A clear concept of operations (CONOPS) for an IASMS is needed to define the scope of such a system and to understand how it would work.
  • Identifying and Prioritizing Risks. Because the universe of all potential risks is large and each risk addressed adds some cost and complexity to the system, it will be important to have an approach and process to prioritize and focus on those risks that will have the most impact on system safety issues that fall within the scope of the IASMS.
  • National Airspace System Evolution. The capabilities of an IASMS will need to increase in sophistication as the NAS continues to evolve and improve, while also accommodating changes in conventional air traffic and new entrants, particularly with regard to the following:
    • Growth in air traffic,
    • Increased uncertainty from new entrants (e.g., UAS, on-demand mobility aircraft, and commercial space launch and reentry operations) and emergent risks,
    • Trust in increasingly autonomous UAS and associated traffic management systems,
    • Unauthorized UAS operations, and
    • Increasing pace of commercial space operations.

• System Monitoring
  • Data Completeness and Quality. Successful and efficient implementation of an IASMS requires identification, characterization, storage, and retrieval of the required data subject to availability, completeness, quality, and cost considerations.
  • Data Fusion. To accurately detect safety risks, an IASMS will need to correlate and synthesize data from heterogeneous data sources with different formats, timing, accuracy, and other characteristics.
  • Collecting Data on the Performance of Operators. Data regarding operator performance that are essential to achieving the full potential of the envisioned IASMS cannot be collected in a timely fashion or at all, in part because of privacy and related concerns.
• System Analytics
  • In-time Algorithms. Existing algorithms for identifying and predicting elevated risk states lack the ability to integrate the diversity of data sources of varying quality anticipated for an IASMS.
  • Emergent Risks. The complexity of the evolving NAS will result in anomalies with unknown root causes, making it hard to develop algorithms that analyze and predict the effects of emergent risks before accidents or incidents occur.
  • Computational Architectures. Existing computational architectures lack the ability to handle large volumes of heterogeneous data and dynamic analytics workflows, both of which are necessary to detect elevated risk states, to detect and characterize emergent risks, and to update the IASMS risk assessment algorithms.
• Mitigation and Implementation
  • In-time Mitigation Techniques. Existing mitigation techniques are limited in their ability to respond to many risks in the short time frame of interest to an IASMS.
  • Unintended Consequences of IASMS Actions. An IASMS could inject new risks into the NAS due to unintended consequences of actions that it recommends or initiates.
  • Trust in IASMS Safety Assurance Actions. The efficacy of an IASMS will be degraded if it is built without regard to the factors that influence operators’ trust in the system.
  • System Verification, Validation, and Certification. There is no accepted approach to verification and validation that leads to certification of a software system as complex as an IASMS, particularly if, as expected, the system includes adaptive, nondeterministic algorithms.
  • Operators’ Costs and Benefits. Operators’ perception of the cost-to-benefit ratio of an IASMS may be so high that it will impede its implementation.

Recommendation. National Research Agenda. Agencies and organizations in government, industry, and academia with an interest in developing an in-time aviation safety management system (IASMS) for the national airspace system (NAS) should execute a national research agenda focused on high-priority research projects in each of four areas, as follows:

• IASMS Concept of Operations and Risk Prioritization
  • IASMS Concept of Operations and National Airspace System Evolution. Develop a detailed concept of operations for an IASMS using a process that considers multiple possible system architectures, evaluates key trade-offs, and identifies system requirements.
  • Identifying and Prioritizing Risks. Develop processes to identify and prioritize risks that are relevant to an IASMS and that threaten the safety of the current and evolving NAS.
• System Monitoring
  • Data Fusion, Completeness, and Quality. Develop methods to automatically collect, fuse, store, and retrieve data from different sources and with different formats, timing, accuracy, and other characteristics.
  • Protecting Personally Identifiable Information. Develop methods of de-identifying and/or protecting sensitive data in a way that does not preclude effective data fusion.

• System Analytics
  • In-time Algorithms. Develop robust and reliable algorithms that can assess large volumes of heterogeneous data of varying quality to simultaneously identify and predict elevated risk states of many different types and that are fast enough to meet in-time requirements.
  • Emergent Risks. Develop approaches for continually mining historical data for detecting previously unknown anomalies and their evolution to characterize their emergent risks and to update the IASMS hazard detection algorithms.
  • Computational Architectures. Support the design of data repositories and computational architectures that support online detection of elevated risk states and offline analysis to detect and characterize emergent risks and to update the IASMS risk assessment algorithms.
• Mitigation and Implementation
  • In-time Mitigation Techniques. For the high-priority risks that fall within the scope of the IASMS CONOPS, identify those for which adequate mitigation techniques do not exist, and develop approaches and technologies necessary to implement timely mitigation.
  • Trust in IASMS Safety Assurance Actions. Identify factors specific to human trust in IASMS safety assurance actions.
  • System Verification, Validation, and Certification. Develop practical methods for verifying, validating, and certifying an IASMS.

Finding. Highest Priority Research Project. The research project on the IASMS Concept of Operations and National Airspace System Evolution is of the highest priority.

Chapter 2 describes a generic IASMS CONOPS (see Figure 2.1). A much more detailed CONOPS is necessary to guide the development of IASMS. The IASMS CONOPS research project is critical primarily because it will establish the framework upon which all other IASMS research is conducted. In addition, it would identify the near-term potential of IASMS research to enhance the safety of the NAS and to engender stakeholder support for and trust in an IASMS. It would also facilitate updates to the CONOPS as the NAS evolves. Developing the CONOPS will be extremely complex and time consuming because of the many factors to be considered and the difficulty of assessing the trade-offs among them, which include the following:¹

• System scope in terms of
  • Aircraft types, including new entrants
  • Data requirements
  • Known and emergent risks
  • Operations in different classes of airspace
  • Time scales for each functional element (monitor, assess, and mitigate) of the generic CONOPS
  • Users
• Ability to collect required data
• Architecture
• Costs and benefits
• Effectiveness
• Growth in air traffic
• Human performance limitations and human-machine roles
• NAS evolution
• System authority vis-à-vis human performance capabilities and limitations
• Technical capabilities
• Uncertainties associated with each functional element of the generic CONOPS
• Verification, validation, and certification

___________________

¹ System scope is listed first because it is the most important of the factors in the list. The other factors are listed alphabetically.

A key goal of this research project will be to understand the characteristics of an optimum IASMS and to thereby provide additional information for refining the list of key challenges and high-priority research projects. Many of the factors listed above are associated with other high-priority research projects identified in this report. Accordingly, the execution of this and many other research projects will likely proceed in an iterative fashion (1) as advances in one area support advances in other areas, (2) as more detailed information becomes available for various factors, and (3) as the ability to conduct complex trade-offs involving all of the factors matures.

ROLES AND RESOURCES

The allocation of organizational roles and resources associated with the development of an IASMS are similar in concept to the allocation of roles and resources described in the two prior reports in this series, each of which addresses the subject of one the six strategic thrusts established by NASA’s Aeronautics Research Mission Directorate.² In particular, each of the recommended research projects would rely on academia, industry, and government agencies to play the same role that they normally play in the development of new technologies and products. Academia would generally participate in the projects at lower levels of technology readiness. Industry would focus on more advanced research and product development. Government agencies would support research and development—internally and/or through contracts with academia and industry—consistent with the mission objectives of the organization and the desired nature of a given organization’s research portfolio in terms of risk, technical maturity, and economic potential. The FAA is leading the Next Generation Air Transportation System (NextGen) program, some elements of which pertain directly to the development of an IASMS.³ The FAA has the expertise and facilities to serve as a test bed for technologies developed elsewhere, and it would be directly engaged in the development of certification standards and methodologies to enable the introduction of IASMS elements into the ATM system. In addition, some ATM equipment operated by the FAA may need to be modified. NASA would contribute primarily by supporting basic and applied research to support advanced development of systems by industry and the FAA. The Department of Defense (DoD) would monitor any changes to the ATM system that could impact the operation of military aircraft in civil airspace. In addition, each of the research projects could be addressed by partnerships involving multiple organizations in the federal government, industry, academia, and other international government agencies. For example, several European and Asian governments are developing data analysis programs similar to the FAA’s Aviation Safety Information Analysis and Sharing (ASIAS) program (see Chapter 1). Both NASA and the FAA have existing cooperative research and development programs with their foreign counterparts. These could be expanded to share knowledge and the cost burden of new research and to maximize the benefit of unique capabilities by particular organizations.

In many cases, it would be beneficial to involve researchers with relevant expertise who might not have a history in addressing civil aviation issues. For example, state-of-the-art research and development related to algorithms for assessing complex data sets is not taking place in the context of civil aviation.

Executing all of the high-priority research projects described in this report would require significant resources. However, for many of the research projects substantial advances could be achieved using currently available resources, especially if those resources are aligned in accordance with the recommended high-priority research projects and if program planning and execution take maximum advantage of the synergies that exist among some of the research projects.

___________________

² National Research Council, 2014, Autonomy Research for Civil Aviation: Toward a New Era of Flight, The National Academies Press, Washington, D.C., and National Academies of Sciences, Engineering, and Medicine, 2016, Commercial Aircraft Propulsion and Energy Systems Research: Reducing Global Carbon Emissions, The National Academies Press, Washington, D.C.

³ The Next Generation Air Transportation System (NextGen) Airborne Collision Avoidance System X (ACAS X) system would replace and improve the capabilities of the Traffic Collision Avoidance System (TCAS). Elements of the ACAS X system would accommodate the special needs of unmanned aircraft systems (UAS) and low-performance general aviation aircraft that lack collision avoidance systems.