II

Safety Analysis, Vulnerability Assessment, and the Design of Integrated Solutions

The second session of the conference covered safety analysis, vulnerability assessment, and the design of integrated solutions to address risks and vulnerabilities. Admiral James Ellis moderated a panel that consisted of Dr. Stephanie Morrow, Mr. Ricardo Moraes, and Dr. Jorge E. Sarkis.

The 2002 Davis-Besse Event and Safety Culture Policy at the U.S. Nuclear Regulatory Commission (NRC) – Stephanie Morrow, U.S. NRC

Dr. Morrow began the session with a presentation on the 2002 Davis-Besse event and safety culture policy at the U.S. Nuclear Regulatory Commission, including lessons learned and safety culture in the reactor oversight process.

The U.S. NRC was established as an independent agency in 1974 with a mission to ensure safe use of radioactive materials for civilian purposes, including nuclear power. It sought to protect public health and safety, promote the common defense and security, and protect the environment through licensing, inspections, and enforcement.

In 2002, massive corrosion was found in the reactor vessel head at the Davis-Besse Nuclear Power Station at Oak Harbor, Ohio. Its Babcock and Wilcox pressurized water reactor had a history of boric acid leakage. On March 6, 2002, a cavity was discovered in the reactor pressure vessel head adjacent to a control rod drive mechanism (CRDM) nozzle penetration. Corrosion was caused by boric acid leakage from CRDM nozzle cracks. The cavity extended though the base metal of the vessel head to the 3/8-inch stainless steel cladding on the inside of the head. The stainless steel cladding had not been designed to maintain the reactor coolant pressure boundary.

The lessons from this event highlighted the importance of safety culture through:

• Leadership safety values and actions: Davis-Besse had prioritized production over safety.

• Questioning attitude: There had been a shift in focus to justifying minimum standards.
• Decision making: There was a lack of conservative decision making or systematic safety analysis of decisions.
• Problem identification and resolution: Corrective actions addressed symptoms rather than causes.
• Continuous learning: Davis-Besse had a failure to integrate and apply operating experience to plant conditions.

The U.S. NRC now considers safety culture in the reactor oversight process (ROP), as introduced by a 2006 revision to the ROP. This revision gave U.S. NRC staff more opportunities to consider safety culture weaknesses before significant performance degradation occurs. It also instituted two processes for the ROP Action Matrix: (1) a process to determine the need to evaluate a licensee’s safety culture in the degraded cornerstone column of the ROP Action Matrix; and (2) a process to evaluate a licensee’s safety culture assessment and independently conduct an assessment in the multiple/repetitive cornerstone column of the ROP Action Matrix.

In a joint effort with the U.S. nuclear industry from 2011 to 2013, the U.S. NRC underwent a safety culture common language initiative, where they developed common terms for describing safety culture. These terms have been incorporated under the ROP cross-cutting areas.

The 2011 Safety Culture Policy Statement sets forth the U.S. NRC’s expectation that individuals and organizations performing regulated activities establish and maintain a positive safety culture commensurate with the safety and security significance of their actions and the nature and complexity of their organizations and functions.

Dr. Morrow also presented a definition of nuclear safety culture: The core values and behaviors resulting from a collective commitment by leaders and individuals to emphasize safety over competing goals to ensure protection of people and the environment. She maintained that safety and security are closely intertwined, and that licensees should emphasize the need for integration and balance to achieve both safety and security in their activities. In addition to the definition, she presented a table of safety culture traits (see Table 2-1).

Lastly, Dr. Morrow discussed outreach and education efforts to foster understanding of safety culture and disseminate good practices. Such efforts include interactions with licensees and external stakeholders, international involvement, conferences and training, educational tools (e.g., brochures, case studies, discussion of safety culture traits, posters, and support materials), and a safety culture website.¹

_____________________

¹Available at http://www.nrc.gov/about-nrc/safety-culture.html.

TABLE 2-1 Safety culture traits

New Sociotechnical Approaches for Safety and Vulnerability Assessment – Embraer experience –Ricardo Moraes, Embraer

Mr. Moraes described sociotechnical approaches to safety, drawing on his experience at Embraer. Different approaches to safety engineering are found in civil aviation, nuclear power, and defense. System theory, which was developed for biology and engineering, forms the basis of systems engineering and system safety. It focuses on systems taken as a whole, rather than their individual parts taken separately. Some properties can only be treated adequately in their entirety, taking into account all social and technical aspects, and these properties derive from relationships among the parts of the system. System theory is also concerned with two pairs of ideas: hierarchy and emergence, and communication and control. Failures are often system emergence, and these events raise questions of what the formal structure and functional interactions are, as well as how failure emerged.

Mr. Moraes presented a framework developed by Nancy Leveson known as System-Theoretic Accident Model and Processes (STAMP), which includes an entire sociotechnical system, component interaction error, software and systems design error, and human error. STAMP is a systems engineering, top-down approach to safety. It offers a more comprehensive view of causality, examining interrelationships rather than just linear cause-effect chains and going beyond

current models. It treats accidents as dynamic processes and looks at the processes behind events. Finally, STAMP includes organizational, social, and cultural aspects of risk (see Figure 2-1).

In comparison with traditional approaches, STAMP includes software and system design errors, human error and human decision making, and behavioral dynamics that change over time. Understanding why controls drift toward ineffectiveness over time enables an organization to detect that drift before accidents occur and, if possible, change its underlying factors. In sum, STAMP handles much more complex systems than traditional safety analysis approaches, Mr. Moraes said.

Embraer is evaluating whether STAMP is a viable methodology to be used as a complementary or alternative means to the current methodologies of the aerospace industry—particularly for highly integrated, complex, and software-based systems. STAMP is also now starting to address cybersecurity issues.

Mr. Moraes provided his definitions for the terms accident, hazard, and concept:

FIGURE 2-1 System-Theoretic Accident Model and Processes. SOURCE: Leveson model adapted from Morales presentation.

The requirements and constraints are derived from an analysis of the potential failure modes, dysfunctional interactions or unhandled environmental conditions in the controlled system that could lead to the hazard.

He then walked the audience through an application of STAMP to the use of landing gear in an aviation setting. Embraer is just starting this evaluation of STAMP, and the initial cases are very simple, but the results are promising, he said. The next step is to apply this methodology to fly-by-wire systems.

Finally, Mr. Moraes asked the group to consider how the software affects traditional safety methodologies, the increase of the integration and complexity of systems, and cybersecurity implications.

Threats Involving Nuclear and Radioactive Materials: Nuclear Forensic Capability within a National Nuclear Security Infrastructure – Jorge E. Sarkis, Instituto de Pesquisas Energéticas e Nucleares (Institute of Nuclear and Energy Research)

Dr. Sarkis presented on threats involving nuclear and radioactive materials, and the nuclear forensic capability within a national nuclear security infrastructure. The creation and maintenance of a nuclear safety system, he said, needs to be the responsibility of each state. Threats that involve nuclear material or radioactive materials are a collective safety issue that requires actions that many times depend on collaborations between nations. He emphasized the importance of collaboration with the International Atomic Energy Agency (IAEA) and other agencies that are dedicated to nuclear safety and exchange with countries and university research centers that have a greater experience in these areas.

Dr. Sarkis concluded that threats that involve radioactive or nuclear materials are not going to go away. Radioactive sources and nuclear materials are widely used, but in the hands of criminals they can become a threat to societies. To fight these threats effectively, we need to adopt preventive measures and train specialized personnel, exchange information, and collaborate with other nations. Very few countries have training courses and specialist information in nuclear forensics and response actions. Responders need to consider the legal aspects to preserve the evidence of the crime scene while allowing the sentencing and imprisonment of the culprits. He put forth the need to establish a nuclear forensic culture in the heart of the infrastructure of a nuclear safety system and program.

DISCUSSION

Admiral Ellis invited questions dealing with safety culture specifically related to nuclear power plants and the Davis-Besse accident. He began by asking how we can learn from the processes described with regard to aviation and forensic issues in terms of nuclear security.

Dr. Almeida wondered why the U.S. NRC decided to have a different definition and what the implication of having these two definitions will be.

Dr. Morrow explained that even at the time when the policy statement was being developed, a number of different definitions were in common use in the United States. For example, the Institute of Nuclear Power Operations (INPO) definition of safety culture was different from the IAEA’s. So a goal of developing a definition with the policy statement was to try to come to some consensus on a definition. In addition, Dr. Morrow mentioned that on expanding beyond nuclear power reactors, to include, for example, parties from the medical communities, there is a rich discussion of safety culture in terms of medicine in general. Different parties brought different definitions to the discussion and there was not a consensus. Therefore, the U.S. NRC needed to develop a definition that would appeal to all different licensees and certificate holders.

Admiral Ellis answered Dr. Almeida that there were two separate standards and discussions when it came to safety culture: the INPO approach, which had been embraced by the industry, and the regulator’s approach, which used different terms of reference. It was very confusing, and some facilities hired different consultants to work towards satisfying the self-regulatory model from INPO on the one hand, and the regulatory view of safety culture on the other. It was felt that a single, common point of reference and terms of reference and definitions were essential.

A participant noted that the 2007 TAM aircraft accident at Congonhas Airport was caused by human error. The pilot did not land or did not approach the landing strip in the right position. Landing in Congonhas is not easy, as the conditions of the runway are not optimal, and there have been two other incidents in Asia where the same problem was cited. This methodology can analyze the environment, behavior, and the chances of wrong behavior from the pilot if the pilot is not trained to act under these circumstances. She asked what different actions might be taken if we see within the analysis of these situations that there is a condition of the environment and pressure from such conditions that raise risks substantially.

Mr. Moraes commented on the methodology, explaining the idea of including many possible operational contexts under all foreseeable conditions. By analyzing the pilot with different scenarios the tool begins to capture the human element with all the different possible interactions with the environment and can achieve better insight into the human-machine interaction under a range of environmental conditions. Embraer deals with recommended practices based on context and behaviors, and it conducts research to understand where this methodology can yield the greatest insight.

Admiral Ellis, as an old fighter pilot, recognized that it is often easy to blame the accident on the pilots when, in fact, the system and conditions failed to put the pilot in a position that maximized the probability of success. There are technical elements involved such as instrumentation and training. There are pressures, especially economic, from the company, whether real or imagined. It is a complex situation, and he said that he always cringes when he hears the term

pilot error because the real determinant is the culture. Does pilot error give an honest assessment of all of the factors in play or does it put the blame on the person who was at the control panel or in the cockpit? That is an important piece in talking about individual accountability and responsibility. Being just and being accountable is not the same as blaming.

There was agreement that improving the system is constructive and useful. The goal is not to find root causes but to understand how to make these accidents not happen anymore. When we talk about Fukushima, for example, an analysis in terms of culture before the accident would have considered it a perfect cultural environment. A participant suggested implementing a System-Theoretic Process Analysis to try to understand interactions between components regardless of failure or errors, but noted that when there is an error, the interactions happen, regardless of whether it is due to a failure or not.

Admiral Ellis summarized key points from the panel discussion: Dr. Morrow described the challenges that come with systems that have been working well and normally for a long period of time. In the commercial nuclear industry, it is called the arrogance of excellence, where things have been done so well for so long that it is assumed it is as good as it can possibly be done. She commented as well about the importance of common definitions for safety culture and principles and described the efforts that the United States completed in 2013 to bring the two separate definitions and approaches of the regulator, on the one hand, and the industry, on the other, into a common language. The objective of harmonizing it with the IAEA still remains. She reminded us that, even though a system has been in operation for decades, there are still unknown unknowns, despite our impression that we know all the elements and all the aspects. U.S. industry thought, in this case, that it understood the corrosion mechanisms, and that while there was corrosion, it was not really important. It turned out to be very important. Continual reassessment and reevaluation of even longstanding and long-operating systems is of benefit and importance.

Mr. Moraes described systems as not just technical, but as socioeconomic, with all of the complexities and interactions that that requires, and the importance of examining the interrelationship of all of the factors, not just the technical or the mechanical. The participants talked more about that from a human standpoint and, most importantly, the piloting perspective. Mr. Moraes introduced software and cyber issues that are continually growing in their importance in our increasingly complex digital world. He asked how we should assess these issues and how we apply models that now have the ability to deal with them effectively in the safety and security context. He described a real-world model for risk and safety assessment that is under evaluation and may have some promise. If we can learn from other industries, it might have benefits in the nuclear world.

Finally, Dr. Sarkis talked in real terms about security and lapses or failures. In an accountability model for security issues, it is not just prevention that is important, but who stole it, where did it come from, and what were the sources. Despite the gargantuan size and complexity of the global nuclear indus-

try, including medical and other efforts, the number of incidents is very small. Thinking back to the infamous Pascal’s Wager, Blaise Pascal noted that the probability of an outcome is not the same as the consequences of an outcome. Just as the threat was global, the corrective actions and processes, up to and including the legal framework, need to be global as well. That is something that we can all help move forward, Admiral Ellis said.