6

Conclusions

Safety management is never finished—it must be continually applied to identify and address emerging concerns in aviation safety. Furthermore, all aspects of safety analysis: the metrics and methods for identifying and addressing risks, the data gathered, the processes for analyzing data, and organizational structure and culture, must themselves be constantly reviewed and updated to address new challenges. This report focuses on transformative changes to technologies and operations, and the new entrants that often propose them, that cannot simply just apply current methods for safety management. “Transformative changes” are defined here as changes that are sufficiently novel that their impact on safety and risk cannot be extrapolated from current data and analysis methods. Such changes are highlighted in the potential systemic stressors noted in the first report with “new entrants” and “new technologies and operations.”

First, the committee examined how to identify and address emerging safety concerns with transformative changes to technology and operations. Second, the committee noted that, while extensive data-gathering activities support monitoring of current concerns, gaps exist in data and analysis methods for predicting new hazards that may emerge with transformative technologies and operations; similarly, methods for predicting and then continuing to monitor for safety concerns are vital to the safe implementation of transformative changes to technology and operations. Third, this report follows up on the committee’s previous finding that the Federal Aviation Administration (FAA) Office of Aviation Safety (AVS) assessment of its internal safety culture was, at the time of the first report, in a formative state requiring further development and review; this report extends its

discussion of safety culture to the question of how to foster and regulate safety cultures within the industry, particularly with new entrants.

This report frames its analysis using the established principles applied in aviation safety management. Safety management occurs across the entire lifecycle of technologies and concepts of operation, from their design, through their implementation via production, operations, and maintenance. Such safety management spans multiple processes, including (1) safety risk management processes before new technologies and operations are implemented, to identify hazards, assess their risk, and implement appropriate safeguards and mitigations, largely focusing on certification and regulatory approval before implementation; (2) safety assurance processes once new technologies and operations are implemented via ongoing data collection and analysis to support continuous improvement; (3) safety policy and objectives determining the high-level properties of the organization integrating a commitment to safety; and (4) safety promotion across all levels of the organization. Chapters 2 through 5 provide detailed findings and recommendations on these topics.

THE OVERARCHING FINDING: NEW CONCERNS ALWAYS EMERGE

This report reflects the committee’s tasking to identify, categorize and analyze emerging safety trends in air transportation, including offering advice to Congress, FAA, industry, and others on options for improving means for identifying, monitoring, understanding, and addressing emerging aviation safety risks. Throughout its activities, the committee relies on a definition of “emerging” that refers to something “becoming apparent or prominent.” Thus, the committee interprets emerging trends in safety to include both new hazards emerging via proposals for new technologies or operations, as well as current concerns that may be becoming prominent.

Even when all is properly regulated and evaluated in best faith to the extent suggested by human understanding, any first-time implementation represents a new frontier in knowledge in which the unexpected can manifest subtly or suddenly and violently. In aviation, even reasonably small changes to otherwise-solid systems have a history of the unexpected emerging. Transformative changes in technology and operations reflect an even larger step-function change in the knowledge that is required to design systems and operations safely, and even to know what tests and monitoring to run. Each transformative change represents a step-change in knowledge and each of these step changes may have gaps that are undetectable and inscrutable until they emerge later, with experience. Thus, it is important to constantly understand that, even after the best safety risk management

process before implementation, something unexpected may happen once something new takes flight.

Thus, the aviation industry, and FAA in all its roles, should remain vigilant for emerging safety risks as new technologies and operations are implemented—to detect the precursor before it manifests as an accident, to investigate unexpected behaviors and effects to characterize their safety and risk, and to be open-minded and prepared to seek new mitigations to newly identified risks. (F5-4)

GAPS IN THE STATE OF THE ART ON AVIATION SAFETY MANAGEMENT

Chapters 2 through 5 detailed a range of concepts in safety management. Some of their associated findings and recommendations highlight relatively straightforward actions that FAA and Congress should undertake, reflecting the state of the art. However, some findings and recommendations highlight where further research and knowledge is required. Commercial aviation is at the forefront of safety, to the extent that it cannot simply learn from other industries, or at least would need to purposefully tailor methods proposed in the scientific literature to apply them to aviation’s unique needs.

The discussion of safety risk management in Chapter 2 highlighted two such gaps. First, the call for increased use of performance-based standards (Finding 2-3 and Recommendation 2-3) requires conceptual understanding of all the functions that enable safe flight. This understanding must breakdown assumptions of who (or what) performs the function and where they are situated, how they are performed, and the types of technologies involved. Second, methods do not exist sufficient to rigorously and systematically design and assess transformative changes to general operating and flight rules that impact many stakeholders in the NAS (Finding 2-4 and Recommendation 2-4).

Likewise, the discussion of safety assurance in Chapter 3 highlighted the need for data analytics applicable to identifying emerging trends in aviation safety, learning from the rapidly evolving methods in many other domains. This includes not only methods for analyzing data but also methods for identifying the data collection mechanisms to build now into transformative technologies and operations to support safety assurance in the decades to come. Unfortunately, research and development of these methods is hindered by lack of representative data sets in the public domain, to the extent that the research communities in data mining, machine learning, and so on, have used other industries as their test cases.

The discussion of organizations in Chapter 4 highlighted how vital culture is to safety. With this comes the question “How to continually mature a safety culture?” This question has two components. First, what is the role of

a regulator, such as the FAA AVS, in overseeing an industry organization’s safety, and how do they do so? Second, what metrics are appropriate for an organization, and its regulator, to continually monitor and reflect upon, and what processes should be brought to bear for continual learning, adaptation, and maturation of its safety culture?

Finally, Chapter 5’s discussion of how to integrate safety across organizations and across all the stages in a product’s lifecycle highlight questions in the science underlying safety management. Current methods focus on one organization managing safety in their current operations: How do these transcend to multiple organizations, some interacting deliberately, and some picking up a product or joining in an operation years after it was implemented by others? The corollaries here include how risk can be managed across organizations, and what one organization needs to do right now (e.g., build in data collection mechanisms and capture knowledge) for later organizations to safely implement their developments.

These conceptual questions extend far beyond the research currently conducted by the FAA AVS. Many of them are sufficiently fundamental that research agencies such as the National Science Foundation may have a role, as may the Aeronautics Research Mission Directorate at NASA. Other related agencies, such as efforts to address safety with artificial intelligence and autonomy by defense research offices, also merit consideration. Thus, the role for the FAA AVS may be to create representative test cases and data sets for the broad national research enterprise to examine, in addition to, or instead of, attempting to single-handedly perform this research in-house.