CHAPTER 5

Research Needs for Improving the Nation’s Weather Readiness and Advancing Fundamental Social and Behavioral Science Knowledge

This chapter focuses on knowledge gaps and research needs, including a synthesis of prior calls for social and behavioral science (SBS) needs (Section 5.1), a discussion of the key characteristics that define different types of social and behavioral science research in the weather enterprise (Section 5.2), suggestions for the types of SBS studies that are needed going forward (Section 5.3), and building on all of this, suggestions for specific critical research topics for the coming years (Section 5.4).

Our focus in this discussion includes not only the immediate timeframe of forecasts and warnings for individual weather events, but also research to better understand vulnerabilities and strengthen preparedness well in advance of such events. It includes research that looks across the originators, communicators, and users of weather information and across all sectors of the weather enterprise. In addition, our focus is on broad-based research that advances fundamental knowledge and is potentially applicable across a variety of different contexts, in contrast, for instance, to marketing research focusing on development of a specific communication product for a specific audience.

5.1 PREVIOUSLY IDENTIFIED RESEARCH NEEDS

Many earlier reports have called for additional SBS-weather research. Our Committee devoted significant effort to reviewing a wide array of the previous documents and findings that discussed SBS research needs. Based on that review, we identify below areas in which there is broad agreement regarding critical knowledge gaps. This reflects the Committee’s detailed evaluation of many assessments that have been conducted over the past decade and beyond, including assessments made by prior National Academies committees and by national and international expert groups

convened to assess the state of research. In addition, we have drawn insights from published assessments of relevant SBS fields and theories, for example, related to team performance and team science (NRC, 2015), interdisciplinary research (NRC, 2005b), warnings (Thompson et al, 2017; Wogalter, 2006; Wogalter and Mayhorn, 2017), health risk messaging (e.g., Parrott, 2017), and risk and science communication (e.g., NASEM, 2017a). Many of these assessments illustrate both the advances in understanding that have resulted from research to date and the critical knowledge gaps that remain (e.g., Bean et al., 2015, on effective warning lengths). Our assessment identified the following topics:

• Efforts to explore and identify vulnerabilities and vulnerable populations, including simulations of risk area, populations’ levels of forewarning, access to shelters, access to warning information, and reduction of storm impacts (Lindell and Brooks, 2013); and studies of how these factors are associated with social and demographic characteristics of populations (Droegemeier et al., 2016).
• Expanded engagement with stakeholders and nontraditional research partners (NRC, 1996, 2003b; Sullivan, 2013) and development of participatory research processes (NRC, 2003b) in order to integrate needs into practices; engagement of stakeholders in order to identify priorities and values and their influence in decision making (NRC, 2005a); improved data collection from populations on many scales (NOAA Social Science Committee, 2015); and work that develops a deeper understanding of the intersections between forecast products and user needs (NRC, 2003b).
• Expansion of disciplinary research, such as economic analysis of tornado warning systems (Lindell and Brooks, 2013), reassessment of criteria for evaluating forecasts, and expansion of approaches to studying weather-related human behavior (Droegemeier et al., 2016).
• Forecaster- and forecasting-specific research needs, such as the effects of models on forecaster performance and the effects of peer pressure on forecaster behaviors (Carbin et al., 2013; Daipha, 2015; NRC, 2003b, 2007).
• Integrated warning system research (see Nigg, 1995; Sorensen, 2000) that addresses topics such as exploring consistencies and inconsistencies between warning channels (Lindell and Brooks, 2013), identifying appropriate roles for the public and private sectors in major storm forecasting and statements (NRC, 2003b, NOAA WP, 2015), and understanding how information networks come about and operate (NRC, 2007).
• Explorations of National Weather Service (NWS) practices and policies, such as false alarm calculations (Lindell and Brooks, 2013), the Asymmetric Penalty (NRC, 2012) (i.e., the preference for false alarms over missed alerts), confusion

• around watch and warning (Lindell and Brooks, 2013), alerting standards, and the implications of changes to these approaches.

• Explorations of specific technologies or forecast approaches in order to improve public understanding and/or to modify those products based on public needs. This includes, for instance, Forecasting a Continuum of Environmental Threats (FACETs), Impact Based Warnings, Warn on Forecast (Drogemeier et al., 2016), confirmation of tornado occurrence from dual-polarization tornado debris signatures, and effects of information and communications technologies such as internet and cell phones (Lindell and Brooks, 2013; Bean et al., 2015).
• Specific risk communication research topics, for example, better understanding the effects of message (especially warning) content on response (Bean et al., 2015; NWS, 2013); timing and pace of warnings (Lindell and Brooks, 2013), and how events intersect with the rhythm of daily life, such as behavioral differences between the week versus weekend, seasonality, and off season (Carbin et al., 2013); warning specificity, including geographic and temporal specificity (Lindell and Brooks, 2013; NRC, 1996); and studies on the use of fear, emotion, and drama in messaging (Lindell and Brooks, 2013).
• Exploration into how we measure and communicate uncertainty (NRC, 2012), with study of visual representations, numeracy, and public interpretation and understanding of different types of measures of uncertainty (NRC, 2003a), how to communicate uncertainty (Budescu et al., 2014; NWS, 2013), showing users the value of uncertainty information, and discovering how to help them incorporate it into their decisions (NRC, 2006a).
• Understanding of how social and personal characteristics influence warning behaviors, such as the effects of experience on risk perceptions and preparing for weather events, and how long those effects last (Lindell and Brooks, 2013).
• Calls for studies on public education and training to change culture and behavior related to hazardous weather preparedness (e.g., Does practicing severe weather drills and other in-school educational efforts decrease adverse outcomes?) (NRC, 2003b, 2005a, 2006a).

5.2 CRITICAL DEFINING FEATURES OF SBS RESEARCH IN THE WEATHER ENTERPRISE

Given the wide array of disciplines and perspectives encompassed within the social and behavioral sciences, the range of SBS research approaches that can be applied to the weather enterprise is highly diverse. This section offers some perspective on the range of approaches that a weather researcher might encounter. Just as a hydrologist and a meteorologist might understand flooding in different ways, so too do researchers

from different social and behavioral disciplines. The distinctions between disciplines matter because each reflects a different focus on the human condition. Furthermore, a disciplinary base provides a critical reference point for theories, concepts, and often for research methods. In this section, we discuss several of these distinguishing characteristics, and illustrate several key research design features. Examining the potential breadth of each of these key features helps us illustrate how extant and ongoing SBS-weather research efforts may be expanded on.

Advancing Science and Applications

When considering the range of research types it is common to distinguish between applied and basic research. SBS research that uses data, trends, and the resulting knowledge to inform a specific design problem or policy, or to adjust a specific practice, would generally be characterized as applied. Such projects focus on an immediate problem at hand. Basic social and behavioral research uses systematic, scientific methods and focuses on unlocking patterns in human perception, interpretation, and behavior. Though often viewed as opposites, the aim of advancing generalized, fundamental knowledge (basic research) and the aim of making practical, effective use of science and technology (applied research) intersect in what is called Pasteur’s quadrant (Stokes, 1997; Tushman and O’Reilly, 2007), a space where it is argued that applied concerns and basic science can and should influence each other (see Section 5.3, footnote 28).

Exploration and Testing in Science

Particularly in instances where little theory exists or prior research has not been conducted on a specific topic, social scientists will collect information and employ systematic processes to explore and identify patterns in the information. For example, behavioral researchers may collect and study records or self-reports of people using weather apps on their smartphones, to understand use patterns. Evaluating research questions and testing hypotheses about such patterns is the most common approach to building a body of scientific knowledge. Like other scientists, SBS researchers are expected to compare findings from their work to the extant record, provide insight into how they compare, and contribute to the accumulation of a scientific body of evidence. Independent validation of findings is essential for scientific confidence in the results, and replication or triangulation among studies can be important for evaluating validity.

Concepts and Theories

While a meteorologist may study atmospheric pressure as a way of understanding differences in weather conditions, a social scientist may study a factor like peer pressure as a way of understanding differences in human behaviors. Concepts such as risk perception, emotion, and economic resources are used by scientists to try to understand and predict social and behavioral patterns and variations (see, e.g., Babbie, 2007; Chaffee, 1991). For example, one might say that experiencing tornado damage makes one more likely to see that hazard as risky. While this general idea is intuitive, social science research will focus on understanding, defining, and measuring this concept, going beyond just the simple intuitive expectation that experience matters, to think more systemically about that relationship. For example, “hazard experience” as a social science concept includes many dimensions—e.g., being in a place that a hazard occurred, having home damage, being injured, having a major life disruption, and taking into account the number, the recency, and the intensity of each of those experiences. Well-defined concepts in a well-developed theory can succinctly explain broad patterns in perception or behavior. Some concepts and theories have been developed specifically to explain people’s interactions with weather, while others that have been developed to address other contexts or problems are (or could be) used to explain behavior related to weather.

New SBS theories are continually emerging to refine, replace, or augment SBS concepts and theories in extant literatures. Some types of theories are represented in at least a limited body of weather-related research, such as sociological theories of vulnerability (Tierney, 2014), psychological theories related to warning response and evacuation (Lindell and Perry, 2012), and economic theories concerning nonmarket valuation (Letson et al., 2007). Many other existing theories could be fruitfully explored further in the weather enterprise, such as psychological and behavioral economics theories of judgment and decision making (Kahneman, 2011; SBST, 2016), organizational and team performance theories (NRC, 2011, 2015; Stokols et al., 2008), network theories (Contractor and DeChurch, 2014), and theoretical models of behavior change (Prochaska and DiClemente, 1983).

The proliferation of SBS theories poses an ongoing challenge to the goal of advancing a cumulative science in the weather enterprise as well as in other domains. Yet, as the body of comparable studies increases, systematic reviews and meta-analyses become possible, which cannot only help advance the goals of the weather enterprise, but can also contribute to the advancement of the social and behavioral sciences at large.

Unit of Analysis and Observation

Saying that SBS researchers study “people” oversimplifies the range of what they actually study. SBS observations comprise social actions and experiences—from individual-level processes to societal-level changes—as well as interactions among these. SBS research can group people together in order to study the distinct characteristics of that group. For example, objects of study include families (e.g., Huang et al., 2016), organizations (Henderson, 2016), communities (Dynes, 1994, 2006), cultures (Oliver-Smith and Hoffman, 1999; Webb et al., 2000), and nations (McEntire, 2007).

For a given study, SBS researchers will carefully consider the implications of choices about the unit of observation and the unit of analysis. Sometimes they are the same—for example, collecting data from individuals to describe how they make decisions (Kunreuther and Michel-Kerjan, 2009) or their beliefs (Klockow et al., 2014) or values (Tierney, 2014). And sometimes they are different—for example, when researchers collect data from individuals in different roles within an organization to examine how the organization as a whole operates (Kendra and Wachtendorf, 2003), or collect data from individuals to examine how a system works (Bostrom et al., 2016). SBS researchers also study people’s interactions and experiences with the things around them, including tools, policies, rules, expectations, resources, symbols, mental models, and so forth (e.g., Barnes et al., 2007; Daipha, 2015), and how individuals, groups, and organizations interact in social systems.

While much SBS-weather research to date has focused on members of the public at the individual level, as illustrated above, SBS-weather research also concerns other actors and objects in the weather enterprise. Actors also include, for example, atmospheric science researchers, NOAA or FHWA management, public- and private-sector forecasters and broadcast meteorologists, a wide variety of public officials—for example, emergency managers, mayors, snowplow operators, and school superintendents (Donner, 2008; Sylves and Búzás, 2007)—and private-sector businesses (e.g., Craft, 1999). Objects include, for example, forecasts, computer software and hardware, budgets, offices and other infrastructure, and policies (e.g., Kunreuther and Michel-Kerjan, 2009). With the emergence of new research tools and greater access to big data, researchers are increasing their attention to behaviors at the system level, for example, agent-based modeling of evacuation behaviors (e.g., Chen et al., 2006; Widener et al., 2013) and social networking (Butts et al., 2012; Jones and Faas, 2017).

Data Collection and Study Design, Methods, Tools

Across different SBS disciplines, there is a range of methods one might employ for different purposes. For example, a study might focus on quantitative methods (such as survey questionnaires, experiments, large dataset analysis, GIS, or simulations using techniques such as agent-based modeling), or qualitative methods (such as interviews, content analysis, focus groups, and participant observations), or mixed methods that encompass both. Within each of these choices, multiple sub-types of studies can be developed. For instance, there are multiple types of experimental and quasi-experimental research designs (e.g., Shadish et al., 2002) and longitudinal analyses that model change and event occurrence (e.g., Singer and Willett, 2003). Surveys can be conducted by phone, internet, or mail, in person, or in mixed mode (e.g., Dillman et al., 2009). Interviews can be structured, semi-structured, or unstructured.

Even within these research types, there can be different approaches used, such as a mental models approach (Morgan et al., 2002), cognitive interviewing (Conrad and Blair, 1996; 2009), or Q methodology (Bouwman et al., 2012; Brown, 1993). Research designs can be cross-sectional (in which data are collected at a single point in time) or longitudinal (in which data are collected at multiple points over time), and they may have varying spatial dimensions (localized versus distributed, small versus large scale). Studies might also vary in their degree of interaction between researchers and the people, organizations, or systems being studied (i.e., is the study purely observational, or do study participants interact with researchers?). Analytical methods vary as well. For instance, studies vary in the extent to which they incorporate synthetic data. While imputing missing data in surveys is a familiar process for many survey researchers, more complex imputation processes, bootstrapping, and simulation techniques can also be used to synthesize datasets.

This section does not exhaustively address the range of options that SBS researchers employ, but it illustrates that SBS research encompasses a diverse toolbox of options, each with strengths and weaknesses. It is thus important to ensure that the right set of tools is used to address any given question, and to recognize that a wide variety of individual-, organizational-, and societal-level SBS research is potentially useful for the weather enterprise.

5.3 THE TYPES AND SCOPE OF SBS RESEARCH NEEDED

Using the general framework of research design characteristics discussed in the previous section as a foundation, and based on what we heard in discussions with researchers across the community, the Committee offers the following as guidance about the types and scope of research that is needed going forward:

• Research that is disciplinary, interdisciplinary within SBS, and interdisciplinary between SBS and atmospheric science and other fields, such as hydrology, engineering, and computational and information sciences;
• Research that is basic, applied, and developmental in nature;¹
• Research that is of different scopes and budget sizes to accommodate different research needs. For instance, larger grants are generally appropriate for interdisciplinary work or for particularly resource-intensive data collection and analysis (e.g., longitudinal studies, ethnographic studies, “big data” analysis). Smaller grants may be appropriate for entraining new scholars into the SBS-weather research arena, or for proof-of-concept studies;
• Research that examines a given topic or event from multiple methodological, disciplinary, conceptual, and sampling perspectives;
• Research that takes a given topic that’s been framed by the meteorological community and reframes it from a SBS perspective. For instance, the related topics of “false alarms,”“over-warning,” and “warning complacency” might be reframed as questions about people’s information access, interpretations, perceptions, responses, and experiences;
• Research that cuts across multiple events, populations, or time, in contrast to research that solves a very specific problem at hand;
• Research that systematically employs in-depth, naturalistic, and engaged methods to investigate the lived weather experiences of people, comparison of cases, and identification of similarity and differences across and within populations;
• Research that goes from “end-to-end,” e.g., that studies operational information that goes into and out of the forecast office; private businesses and media companies that access the information and transmit it to the public in

___________________

¹ Following the National Science Foundation definitions: Basic research: systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications toward processes or products in mind. Applied research: systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met. Development: systematic application of knowledge or understanding, directed toward the production of useful materials, devices, and systems or methods, including design, development, and improvement of prototypes and new processes to meet specific requirements.

• many forms; public officials and managers as mediators of forecast, preparedness, and response information; businesses and members of the public as end users; and

• Research that leverages and builds on methods, concepts, and theories from nonweather research, recognizing that hazardous weather is a subset of research on risks, hazards, and disasters. This should include past data collection instruments and datasets that can be mined for secondary analyses.

5.4 CRITICAL KNOWLEDGE GAPS

Synthesizing our analysis of prior calls for research in Section 5.1 and the wide array of ideas and perspectives about research needs presented to the Committee over the course of this study, including the information presented in prior chapters, we emphasize here three broad research areas that will be critical for the coming 5-10 years. We expect that research priorities will evolve over time as society and technologies change, and as meteorological and SBS understanding, practices, and capabilities advance.

Research looking at the weather enterprise as a system is needed to gain insights into how system-wide changes in forecast production and operations affect the quality and value of weather information, and to guide new decisions about weather enterprise operations at the system level. Research on risk assessments and responses and factors influencing these processes is needed because the growing emphasis on impact-based warnings and decision support increases pressures to understand different types of vulnerability and risk and how they interact and vary. Research focused on message design, delivery, interpretation, and use is needed, even though message design has long been a focus of weather research and there have been many recent advances (e.g., Bean et al., 2015; Sutton et al., 2015a,b). Much remains to learn in this realm because how people interpret messages is context-dependent, and the technological, social, and physical contexts for weather information are evolving rapidly (see Ch.4 of NASEM, 2017a). These three themes are described and illustrated with examples below.

Weather Enterprise System-Focused Research

Additional studies of weather information production, dissemination, and evaluation that would benefit the weather enterprise system include examinations of what information sources different people in the weather enterprise system use and trust; the timing of information production and dissemination; system effects on the design

of content, and on how information is used or influences others; the reach of information such as forecasts and warnings throughout the weather enterprise, and how this reach is affected by the information source, channel, and content. Studies of what messages get shared on social media such as Twitter illustrate this type of research and the lessons it can provide for forecast and warning systems (e.g., Sutton et al., 2015a). Examples of specific research questions include:

• How do we investigate “end-to-end” questions about how forecasters, media, emergency managers, private weather companies, and others create information and communicate with officials, local residents, and others—as well as how people access, interpret, and use this information? What are the critical systems of connections among individuals, groups, organizations in the weather enterprise, such as Integrated Warning Teams?
• What organizational and institutional changes would enhance collaboration across public, private, academic sectors in the weather enterprise?

A better understanding is needed of the ways forecasters access, interpret, and use the increasing proliferation of new observations, remote sensing (e.g., GOES-16), and numerical weather prediction information to assess and communicate weather hazards. For example:

• What global, synoptic, and storm-scale model guidance (including deterministic and ensemble-based guidance and associated verification information) do forecasters currently use and how, both for their own assessment of hazardous weather threats and for communicating hazardous weather threats to their core partners? How is this evolving over time, and what factors are driving this evolution?
• How can forecasters’ expertise be coupled with computer-based output in ways that enhance forecast effectiveness, trustworthiness, and acceptance? How might this vary for different hazardous weather scenarios, in different places, and at different scales?

Much remains to learn about team performance, organizational behavior, and focal activities in the weather enterprise, especially the activities and behaviors of forecasters and Weather Forecast Offices (WFOs). Recent research in this area (e.g., Anthony et al., 2014; Bass et al., 2011; Daipha, 2015; Demuth et al., 2012; Fine, 2007; Fine and Hallett, 2014; Heinselman et al., 2012; Hoffman et al., 2017; Morss et al., 2015; Wilson et al., 2016) suggests that the effects of rapid technological and management changes should be evaluated regularly. Further investigations are needed to help refine performance evaluation metrics and practices. Research questions that have

been raised previously but that remain incompletely answered in the current forecasting environment include, for example:

• What are the different ways that forecasters interact with other meteorologists and support staff within a given office, and how does this influence the forecast and warning process? How do the number and type of forecast office staff, the shift configurations, and other contextual factors influence forecast and warning processes?

Further study of the economic valuation of services and improvements in services provided by the weather enterprise is much needed (e.g., Freebairn and Zillman, 2002; Frei et al., 2014; Gunasekera, 2003; Letson et al., 2007). For example:

• How can one best assess the value of improved forecast information for different types of hazardous weather, different types of improvements, and different forecast parameters and timeframes? What are the most informative ways of measuring the impact of forecast improvements in helping to save lives? How do different users of forecasts perceive and value these impacts?

Risk Assessments and Responses and Factors Influencing These Processes

Research is needed to improve understanding of how people obtain hazardous weather information, including associated uncertainties; how they perceive their chances of being affected and harmed by a weather threat; their beliefs about their abilities to reduce those risks; their behavioral and emotional responses; and what factors influence all of those processes. Such research should address distinctions in the information needs across various populations, including the especially vulnerable (such as the poor, illiterate, innumerate, disabled, very young, aged, non-English speaking, tourists, or hospitalized) and key professional groups (see, for example, the discussion of transportation management professionals in Chapter 4). Some open research questions include:

• How can the weather enterprise improve the timing, content, and channels of hazardous weather information to better reach, and enhance the utility of information for vulnerable populations, or populations with special or professional needs that are not well served by current systems? When information alone is insufficient to protect vulnerable and special needs populations, how can the weather enterprise effectively inform and support other types of interventions, such as engineering or planning interventions?

For understanding the needs, opportunities, and capacities of professionals with regard to weather information and decision making:

• Are there new types of information or products, or enhancements of existing products, that NWS could provide to better support the weather communications efforts of broadcast meteorologists, private-sector forecast providers, emergency managers, transportation managers, and others in the weather enterprise?

For understanding the needs, opportunities, and capacities of different populations with regard to weather information and decision making:

• How might community- or neighborhood-based weather hazard communication and participation (i.e., crowdsourcing or other processes that focus on exchanging information with key community leaders, gatekeepers, and actors) enhance group-level adaptive capacity to enhance resilience and reduce harm?
• What factors moderate people’s interest in, access to, interpretation of, and responses to weather information (emotional or protective factors, social norming, isolation, connectedness)? What is the role of factors such as physical context and vulnerabilities, socio-demographic characteristics, prior experiences, worldviews, personality traits, cultural background, and personal values?
• How does scientific evidence about the effectiveness of protective actions (e.g., the safest structural locations in which to take shelter from a tornado) match people’s perceptions of those protective actions?

Message Design, Delivery, Interpretation, and Use

Advancing the effectiveness of the weather enterprise requires a better understanding of how message content and weather information provision practices influence people’s interest in, understanding of, and response to weather services and products. Much remains to learn about how to most effectively tailor forecasts, warnings, and protective action recommendations to the needs of specific types of users. As highlighted in the studies summarized in Section 5.1, specific issues needing further study include warning specificity, how communication technologies interact with message design, and how best to design impact-based warnings. Those designing and producing weather forecasts, warnings, and other decision support information need a better understanding of the perceptions and uses of uncertainty information by different types of audiences (such as state DOTs, drivers, public works directors, and vulnerable populations), and of how these different groups in turn communicate such uncertainties to others. Examples of specific research questions include:

• What are effective approaches to communicating forecast uncertainty information so that it is understandable and actionable for different users, at different temporal and spatial scales, for different weather hazards, including both slower-onset, spatially diffuse hazards such as hurricanes and winter storms and rapid-onset, spatially localized hazards such as flash floods and tornadoes?
• How do operational forecasters balance the requirements for consistency in messaging (e.g., use of nomenclature or designations like “hurricane”) with the needs for flexibility to best suit different geographical and cultural contexts?
• What are the social impacts of, and the best strategies to manage, heavy media attention on major weather events, including the potential for and concerns about “media over-hype”?
• How do false alarms and misses associated with high-impact weather events such as tornados affect people’s perceptions and behaviors when hazardous weather threatens?
• How are new communication and information technologies (including the proliferation of different sources, content, and channels of weather information, along with improvements in meteorological understanding and forecasting) changing how people access, interpret, and respond to weather information? What effects are social media and information crowdsourcing having on people’s weather comprehension, preparedness, risk perception, and response?

This page intentionally left blank.