B

Summaries of Research Results from DHS-Supported Principal Investigators

The committee asked each of the principal investigators funded by the Department of Homeland Security Science and Technology Directorate to conduct research on emergency alerts to provide a brief summary of their work. These brief statements are included below. Although the committee reviewed these statements, these individually authored papers do not necessarily reflect the views of the committee.

PUBLIC RESPONSE COGNITIVE MODELING OF THE IMPACT OF WIRELESS EMERGENCY ALERTS

Courtney D. Corley, Ph.D., Senior Data Scientist and Team Lead, Pacific Northwest National Laboratory

Wireless Emergency Alerts (WEAs) are a critical mitigation measure employed during emergencies to inform and keep the public safe. Research on WEAs and disasters conducted by the Pacific Northwest National Laboratory (PNNL) and Advanced Brain Monitoring (ABM) has found that individuals perceive the threat of floods differently than other types of disasters on a physiological level within the frontal lobes of the brain. This difference occurs both when subjects are told they are about to watch a video about floods and when they are watching or reading alerts about floods. The perceived urgency of floods also appears to be more sensitive to the personality characteristics of individuals than during other types of disasters.

Methodology

The PNNL and ABM effort collected 20-channel electroencephalography (EEG) data from 51 subjects as part of an experiment to evaluate the ways in which people perceive different kinds of disasters, and their response to different types of social media content related to disasters. Subjects were presented with a series of 50 WEA and Twitter messages collected from each of five types of disasters (blizzard, flood, gas leak, hurricane, and tornado) for a total of 250 messages, and asked after reading each if they would share that message over their own personal social network. These messages were a combination of those shared by actual Twitter users and disaster alerts sent by news stations and other emergency alert services at the time of the disaster. Prior to exposure to a disaster-specific set of messages, subjects were told what type of disaster they were about to view, and then shown a contextual news broadcast related to that type of disaster. All subjects were exposed to the same 50 WEA and Twitter messages for each disaster, but the order in which the disasters were presented was changed randomly each time.

Subjects responding to Wireless Emergency Alerts and social media messages were more predisposed to share WEA and disaster tweets expressing a dismissive sentiment (i.e., a message that advocates or expresses intent to ignore a disaster alert) about floods than they were other types of disasters. Analysis of EEG data from the subjects during the period of time when they were deciding if they would share a given message with their peers over a social network suggests that this decision-making process occurs primarily within the frontal lobe. This is significant because it aligns with other published research postulating the frontal lobes are essential for all aspects of decision-making and play an important role in many higher cognitive functions.¹ Subjects in our study typically had higher levels of brain activity when deciding to share a message as compared to when deciding not to share a message, suggesting a more deliberative thought process. Brain activity changes were especially pronounced when subjects were choosing to share disaster alerts. Older subjects (age 50+) were significantly more likely to share messages of all types with their social network than younger subjects. Overall, all subjects were highly responsive to all types of disaster messages (WEAs and tweets) and shared them a majority of the time.

___________________

¹ D.A. Pizzagalli, R.J. Sherwood, J.B. Henriques, and R.J. Davidson, 2005, Frontal brain asymmetry and reward responsiveness: A source localization study, Psychological Science 16(10):805-813.

Video Response

Our research suggests that subjects have different brain responses toward different types of disasters that are inversely correlated with the volume of danger perceived. During the subject trials, all subjects were shown a context video (news coverage of the specific disaster) immediately prior to responding to WEAs and tweets associated with that disaster. Subject brain activity during these videos was analyzed and compared across disasters types to assess how the subjects perceived the disasters. Previous research by Dennis et al. (2010), exploring the impact of emotional film clips, discovered that subjects with higher levels of electro cortical activity in the frontal lobes were less affected by the stimulus, and the influence of the stimulus was shorter lived.² This is consistent with what we observed. Our analysis also found that subjects with the highest levels of activity during the video stimulus were also those who were less likely to share informative WEAs and tweets about the disaster with their peers. The subject’s brain activity prior to the presentation of the context videos was also examined. Before the beginning of context videos, subjects were presented with a message explaining that they were about to see a video and tweets about a particular type of disaster. We observe that users have an immediate change in physiological disposition. Their response, therefore, is not shaped by the particulars of the video itself, but only their immediate, visceral disposition toward that type of disaster. This analysis suggests that subjects’ response to floods is due to their fundamental perceptions of the dangers of floods, and not the specifics of the scenario. Conversely, upon being told they were about to view a video about tornados, subjects showed unusually high attention—a stark contrast to the response seen toward floods.

Results

Overall, the WEAs tested proved to be highly effective across all disaster types and when compared to other social messages, the WEAs were among the most shared by the test subjects. However, even when subjects chose to share these alerts, the EEG responses to flash-flood specific alerts were distinct from other disasters. When shown context videos for each type of disaster, and particularly for floods, subjects with the least levels of attention and engagement during the video stimulus were also those who were less likely to share informative tweets about the disaster with their peers. Additionally, subjects more frequently shared

___________________

² T.A. Dennis and B. Solomon, 2010, Frontal EEG and emotion regulation: Electrocortical activity in response to emotional film clips is associated with reduced mood induction and attention interference effects, Biological Psychology 85(3):456-464.

messages expressing a dismissive sentiment (i.e., a message that advocates or expresses intent to ignore a disaster alert) regarding floods than they were other types of disasters. These responses appeared to be the most exaggerated when among subjects with the least depressive personality types.

Together, this research suggests that the subjects perceived the threat or urgency posed by a flash flood quite differently than other disasters on a physiological level. The response also appears to occur almost instantaneously, suggesting that the response is perhaps reflexive or develops over their lifetime. This response appears to manifest itself in the form of subjects both appearing less mentally engaged with the news coverage of floods, as well as an increased willingness to ignore or actively dismiss the associated weather alerts. There are limitations with the study conducted that are detailed within this report.

Conclusions

The PNNL and ABM team have one primary recommendation and one secondary recommendation for the use of WEAs coming from this research.

Conclusion 1. When compared to tornado, hurricane, gas leak, and blizzard WEAs, flood WEAs are systematically perceived differently in our study group. This leads the PNNL team to suggest that additional attention be directed at communicating the risk of floods to citizens. For example,

• The WEA could focus on stating specific and direct action for recipients.
• Various formulations of WEA could be disseminated specifically for floods as a special case.
• Although geo-targeting of WEA was not in the scope of the PNNL study, providing citizens with location relevant information may further encourage action.
• Education stressing the seriousness or severity of floods and other similarly dismissed disasters might help reduce the public’s flippant response to the alerts.
• Users act as a megaphone for disaster alerts in other instances, amplifying the exposure of the alert by repeating its information, particularly for tornados. The WEA program could identify methods to better harness this effect for perpetuating the flood alerts.
• Further understanding is needed of how citizens perceive the risk of disasters in specific regions or of certain cultures. PNNL noted social

• media users dismissing specific types of disaster alerts (floods) based on Southern California flash floods. Citizens in other types of disasters in other locations (i.e., Southern United States as opposed to the Northeastern United States) might treat hurricane warnings with similar disregard because they are more common.

Conclusion 2. The results of this study in combination with several recently published reports support the validity of specific neural responses to various types of communications, narratives, and messaging that can accurately predict human behavior in response to these communications. We recommend the implementation of platform technology to routinely screen emergency message form and content using neurophysiological, cognitive, and other measures to add to a database acquired for comparisons and data modeling. This approach would include developing a database of responses from a diversity of people representative of the US population demographics and regions. Data would be uploaded via a cloud-based portal that is easily accessible with a PC and Internet access.

WEA MESSAGES: IMPACT ON PHYSIOLOGICAL, EMOTIONAL, COGNITIVE AND BEHAVIORAL RESPONSES

Deborah Glik, David Eisenman, Kerri Johnson, Mike Prelip, Armen Arevian, Andrea Martel, UCLA Fielding School of Public Health, Geffen Medical School, and Department of Communication

Effective alerts and warnings for disasters protect people and save lives. Over the past decade as mobile communication technologies have become ubiquitous, disaster and emergency messages sent to end users directly have emerged as promising new practices. In particular, short message service (SMS) text message formats have emerged as a modality that is both practical and popular as the majority of the public now use smartphones. In regard to Department of Homeland Security (DHS) wireless emergency alerts (WEAs) these messages are pushed out through commercial mobile carriers to customers who are located geographically near the hazard, and newer smartphones are “WEA enabled.”

While the WEA system and other SMS or text based warning systems and messages are coming online rapidly in governmental agencies, universities, and other organizational settings, research about how these systems work has been sparse. We do not have adequate data about how people act and react when they receive WEA messages in real time. This information is key to designing messages that work with the current technology as well as take account of typical human responses to threat messages, otherwise known as the stress response or the “fight or flight” reaction.

The major goal of this research is to test how short WEA disaster-warning messages are processed by recipients. In a series of laboratory experiments in which participants received simulated warning messages on a smartphone, we measured psychophysiological, emotional, cognitive and behavioral responses of recipients. We conducted experiments on a young (18- to 26-year-old) audience who are part of the wired generation, assuming they are adept in regard to mobile device use and literacy, hence representing an audience who should be most likely able to process and use these messages to inform subsequent disaster response actions. Major issues addressed in this study include the following:

• The impact of receiving simulated WEA messages on psychophysiological arousal
• Relative effectiveness of different WEA message lengths (90, 160, 280 characters) and message content
• Behaviors observed among recipients of initial simulated WEA messages
• The role of personal characteristics, emotions, cognitions, and perceptions among recipients of WEA messages
• How physiological arousal, emotions, perception, and behavior interface with the text message and current mobile device technology
• Difference in response received in a social rather than a solitary context

Study methods included a series of social psychological experiments. The study sample was comprised of undergraduate and graduate students between the ages of 18 and 26 who were attending a large urban university. Once recruited into the study, recipients came to a laboratory and were connected with MindWare technology that monitors physiological functioning. They then received WEA messages on a mobile device with either an active shooter or explosion scenario on their campus. Physiological measures were comprised of skin conductance, cardiac activity, and arterial pressure. Personal characteristics, emotions cognitions, perceptions, and behaviors were measured using surveys, observations, and qualitative interviews.

Major conclusions include:

• WEA SMS text messages do have a significant impact on physiological arousal, emotional response, cognitive processing, and behavior.
• The most reliable physiological indicator was SCR (Skin Conductance Response).
• A message of 160 characters is more impactful than a 90-character message, but there is no clear gain with messages that are 280 characters.

• The most effective message length is the amount of characters that can fit onto the mobile device screen of the recipient in the first alert notification. In new phones this is 160 characters; in older phones it is 90 characters.
• Because of the stress response most recipients only read a few words of text before enacting a more general scan of their immediate environment and most do not go into the application itself.
• Moreover, recipients only remember a few key words. Important words or guidance must be articulated at the beginning of the message not buried at the end and must be specific.
• Many recipients clicked off the messages and many did not believe the messages were credible.
• When people were in groups they were more likely to talk to one another about the messages after they received them.
• However, a larger number of people in all study conditions actually did nothing when they received the messages, and people did not click on the embedded URL.

Other conclusions include the following:

• Short concise concrete images and messages of 160 characters are sufficient.
• Use social marketing campaigns and education to teach people what the DHS WEA technology is and how to use it.
• Build a brand that people trust.
• Create pre-event messages templates that are up-to-date and accessible.
• Build capacity for other organizations to build successful disaster and alert warning systems.

RESULTS OF AN INTEGRATED APPROACH TO GEO-TARGET AT-RISK COMMUNITIES AND DEPLOY EFFECTIVE CRISIS COMMUNICATION APPROACHES

Bandana Kar, University of Southern Mississippi

Research Focus

A warning system, essential for risk communication, is comprised of three main components—a detection subsystem (that focuses on detecting and/or predicting location and time of a hazard event), emergency management subsystem (that focuses on determining the threat posed by a hazard, and the necessity to formulate and disseminate alert and

warning messages to the public who are at risk from the hazard), and a public response subsystem (PRS) (that focuses on public receipt and understanding of messages, and their responses in the form of preparatory actions).^3,4,5 A number of technologies are used to alert at-risk populations of impending disasters, which include outdoor sirens, Tone Alert Radio (TAR), televisions, and Wireless Emergency Alerts (WEA) that are disseminated via cellular networks to cell phones and other mobile devices as text messages.^6,7 These technologies follow a hierarchical approach such that alerts and warnings are delivered from recognized sources (e.g., National Weather Service) to emergency responders and ultimately to at-risk populations. By contrast, social media that allows creation of social networks and is horizontally integrated across society has become a popular source of risk information before, during, and after an emergency event.^8,9

Although significant advances have been made to accurately predict a hazard and its impacts, and disseminate risk information to the public, the effectiveness of existing communication technologies is still a research challenge, specifically, from the perspective of the public response subsystem. The PRS is influenced by two main factors: (i) warning messages—message content and style, message source, message delivery technologies; and (ii) message recipient characteristics—social and psychological characteristics of the public. This project focused on examining the effectiveness of the PRS by undertaking the following tasks in the culturally diverse and heavily populated Mississippi Gulf Coast: (i) determine the coverage area of existing alert and warning technologies; (ii) determine perception of the local emergency management agency (EMA) personnel of available warning technologies; (iii) examine the impact of socioeconomic and cultural characteristics of the public, and message format and content on public response to warning technologies and alert messages; and (iv) examine

___________________

³ J.H. Sorensen, B. Vogt, and D.S. Mileti, 1987, Evacuation: An Assessment of Planning and Research, Oak Ridge, TN: Oak Ridge National Laboratory.

^4, J.T. Grabill and W.M. Simmons, 1998, Toward a critical rhetoric of risk communication: Producing citizens and the role of technical communicators, Technical Communication Quarterly 7(4):415-441.

⁵ National Research Council, 2012, Disaster Resilience: A National Imperative, Washington, DC: The National Academies Press.

⁶ J.H. Sorensen, 2000, Hazard warning systems: Review of 20 years of progress, Natural Hazards Review 1(2):119-125.

⁷ FEMA, “Integrated Public Alert & Warning,” last update August 30, 2017, https://www.fema.gov/integrated-public-alert-warning-system.

⁸ B.R. Lindsay, 2011, Social Media and Disasters: Current Uses, Future Options, and Policy Considerations (R41987), Washington, DC: Congressional Research Service.

⁹ B.F. Liu, L. Austin, and Y. Jin, 2011, How publics respond to crisis communication strategies: The interplay of information form and source, Public Relations Review 34(4):345-353.

the role of public participation in message preparation on public response to messages.

Methodology

This study was conducted in the three Gulf Coast counties of South Mississippi (Hancock, Harrison, and Jackson). Other than being the landfall location for Hurricane Katrina, the Mississippi Gulf Coast is susceptible to tropical cyclones and coastal flooding events. The high-risk areas of the coastal counties are heavily populated, consist of culturally diverse communities, including Anglo-Americans, African-Americans, and Vietnamese and Hispanic immigrants, and are home to vulnerable population groups, such as immigrant communities composed of older-generation individuals with limited knowledge of English. By conducting this study in a diverse ethnic setting, the purpose was to identify factors influencing use of warning technologies so that policies can be formulated to increase public response to alerts.

A combination of spatial and statistical techniques using both primary and secondary data sets were implemented for each task. Spatial data sets, such as census boundary and socioeconomic information, transportation and hydrologic networks, coast boundary, digital elevation models (DEMs), land use/cover data, and location and spatial coverage data about alert and warning devices were used to model spatial distribution of vulnerable populations, identify physical risk areas, and determine spatial coverage provided by existing alert technologies. Participatory and action-oriented ethnographic surveys were administered in English, Spanish, and Vietnamese to individuals and EMA personnel to collect primary data about the usability and performance of available warning technologies, format and content of messages, and public perceptions and responses to alert messages and technologies.

Research Results

Task 1: Currently, the EMA personnel primarily use the following four technologies to disseminate alert and warning messages to the residents of the Mississippi Gulf Coast - Siren, Radio/NOAA Weather Radio, TV/National Weather Service (NWS), and WEA/Cell Phones. While sirens are located only in two cities—Bay St. Louis (Hancock County) and Gulfport-Biloxi (Harrison County), and provide barely 1.46% of spatial coverage, each of the remaining three technologies provide more than 90% spatial coverage. In fact, cell phone was found to provide 99.4% spatial coverage although the coverage is influenced by spatial variability of signal strength. Despite limited geographic coverage, sirens are still used as their

availability is a criterion in the Community Rating System of the National Flood Insurance Program.

Task 2: According to the EMA personnel, in addition to the technologies identified in task 1, Reverse 911, sirens, and posters/pictures are also used to disseminate warnings and alerts. The EMA personnel indicated that TV/NWS, Radio/NOAA Weather Radio, WEA/Cell Phones, and Reverse 911 allow dissemination of updated and accurate warning and alert messages, and therefore, have been found to be more effective in communicating risk information to the residents of the three study counties during a hazard event. Because 90% of the time warnings and alerts are disseminated in English, sirens and posters/pictures are the only technologies available to reach out to the Vietnamese and Hispanic residents with limited English knowledge. However, these technologies fail to provide frequent updated messages to residents.

The EMA personnel also indicated their preference to use social media, specifically Facebook, to disseminate messages during a hazard event. Because social media sites allow public participation and broader outreach, EMA personnel consider this technology to be more effective in motivating the residents to take preparatory actions in response to alert messages. However, due to lack of skilled personnel and the possibility of generating rumors, EMA personnel are reluctant to use social media.

Task 3: Analysis of physical risk revealed that the three counties are occupied by moderate to high risk zones susceptible to coastal flooding. These risk zones are occupied by major urban areas, densely populated, and have a high concentration of socioculturally vulnerable groups.

Analyses of the coastal residents’ responses to a household survey revealed that the residents use the technologies identified by the EMA personnel to receive alerts except for posters/pictures that are used by a small group of the population, specifically, the Vietnamese residents. Respondents indicated that they trust information received from their family/friends followed by the information received from authorities via TV, Radio, and WEA/cell phones. Despite limited use, survey participants indicated that they trust sirens more than social media sites. Family and friends were also found to be more effective in encouraging residents to take positive actions as opposed to messages received from authorities. For instance, a smaller, yet sizable, number of respondents indicated that their decision to evacuate will depend upon the actions of family/friends or on their own perception of risk rather than the alerts and warnings received from official sources. However, analyses of survey responses

revealed the overall willingness of the residents to follow official evacuation notices.

Almost half of the respondents (130 of 275 survey participants i.e., 47.3%), which included a majority of respondents from Asian and Hispanic ethnicity, expressed their wish to receive alert and warning messages in languages other than English. These individuals indicated that they use social media and family/friends to receive risk information about hazard events, wanted local EMAs to use social media to disseminate alert messages during hazard events, and believed that social media would allow them to receive messages in different languages. The survey respondents indicated that WEA message should include the following information: nature of the disaster, impact zone, time frame and duration of the disaster, recommended actions, evacuation routes, when to take action, shelter location, how to obtain additional information, and a map of evacuation routes, shelters, and nearby hospitals.

Task 4: There was a strong agreement (about 60%) among survey respondents regarding (i) their willingness to collaborate with local EMAs in disseminating alerts and warnings; and (ii) their inclination to take positive actions in response to alert and warning messages if they were involved in message dissemination and preparation. These respondents also revealed that they had never participated in message dissemination and preparation with the local EMAs.

Conclusion

The culturally and ethnically diverse communities of the Mississippi Gulf Coast trust and rely on alert and warning messages received from traditional media using conventional technologies. However, these at-risk communities also use social media sites to communicate with families and friends, and have indicated that they would like EMAs to use social media to disseminate risk information. Despite considerable progress in the development and implementation of WEA, its usage remains varied across communities of the study area. Although a majority of respondents claimed to have known about WEA prior to this research, a sizable minority did not, which suggests that more focus is needed to increase awareness about WEA to specific segments of the population. Hispanic and Vietnamese residents tend to use social media during a hazard event for risk information due to the possibility of receiving alerts and warnings in languages other than English. These population groups want EMAs to disseminate alerts in other languages.

Research Gaps and Future Directions

Future research should examine (i) ideal message content for maximum effectiveness, (ii) why some individuals/socioeconomic groups are more inclined to heed messages than others and/or participate in message dissemination than others, (iii) how language barrier affects response to messages, (iv) how EMAs can maximize social media to disseminate messages, (v) to what extent current social media usage provides accurate information to be effectively used by the public, and (vi) how social media usage is impacted by different types of hazards. Studies should also investigate (i) ways to increase public participation in message dissemination process, and in generating relevant and reliable data and information to be used by EMAs during response and recovery activities in near real-time, and (ii) why some individuals and/or socioeconomic groups do not wish to participate in message dissemination nor to take appropriate actions in response to valid alerts and warnings. Actions should also be taken to increase public knowledge of WEA messages, EMAs’ social media presence, and ways to disseminate messages to individuals who do not understand English.

COMPREHENSIVE TESTING OF IMMINENT THREAT PUBLIC MESSAGES FOR MOBILE DEVICES

Brooke Liu (PI), Hamilton Bean (Co-PI), Michele Wood (Co-PI), Dennis Mileti (consultant), Jeannette Sutton (researcher), and Stephanie Madden (graduate research assistant) National Consortium for the Study of Terrorism and Responses to Terrorism

This project sought to determine the optimized message contents of WEA messages. The project compared first-alert WEAs to 140-character and 1,380-character messages. The project also tested 280-character messages should the length of future WEAs be expanded. Research methods were 50 think-out-loud interviews, 13 focus groups, 16 experiments, and 1 community event survey. Hazards examined were active shooter, nuclear explosion, tornado, tsunami, and flash flood. The research resulted in the following conclusions:

• Message content order. A different order for the content contained in 90-character WEAs may improve public response outcomes. WEAs currently use the following order: hazard, location, time, guidance, and source. An alternative order had an advantage in improving the public outcomes tested: source, guidance, hazard, location, and time. For 280-character messages, moving the source to the start of a WEA was optimal; however, the optimal order for all WEA content seems to depend on message length.

• Relative importance of content elements. For 90-character messages, guidance and hazard message content elements played key roles compared to other message content elements (location, time, and source) in facilitating the sense making outcomes of interpretation (understanding, believing, and deciding) and personalization. They also reduced milling (causing delay in taking a protective action). For 280-character messages, the message elements of guidance (what to do and how to do it) and time until impact (how much time people have to take the recommended action) play major roles.
• Expanding message content. Should it become possible to expand the length of WEAs in the future, it would be most important to expand the content areas communicating the hazard (what happened), guidance (what to do about it), and time until impact (when to do it).
• Familiarity with the WEA service. Continued outreach and education about the WEA service may help to speed the rate at which members of the public read and respond to WEAs. Survey findings suggest that some people who receive WEAs do not read them immediately. Survey findings further suggest that, when received and read, WEA messages can be effective at reaching and motivating immediate protective action taking.
• Understanding of acronyms. The public may have little or no understanding of many of the acronyms used in WEAs. Hence, consideration should be given to modifying the system to discontinue the use of acronyms, educate the public about their meaning, or increase the message length to allow for full text descriptions rather than acronyms. There may be unique exceptions (e.g., NWS)
• How to best express time. The way WEAs express time may confuse the public. Currently, WEAs express time by stating when the message expires so that such messages do not persist in perpetuity. However, expressing time this way is confusing and potentially life-threatening. If time is expressed in WEA messages with language about the time a message expires, consideration also should be given to communicating the time a message “begins” to reduce public confusion.
• How to best express location. Given the 90-character limit of current WEAs, the phrase “in this area” does not effectively work to communicate who is and who is not located within the risk area. For example, more than a quarter of WEA message recipients from our community event survey did not think that the message was meant for them. Furthermore, each WEA disseminated message that states “in this area” but does not apply to the individual receiving the message may train message receivers that the phrase “in this area” may not apply to them.
• Understanding of alert and warning concepts. The public may not understand basic alert and warning concepts. Messages should not rely on the assumption that the public understands terms such as shelter,

• evacuate, and proceed to higher ground. For example, survey respondents who reported receiving a WEA message and hearing outdoor warning sirens ranged widely in what they thought proceed to higher ground meant.
• Map inclusion. More research is needed on how maps impact public understanding of warnings and differences among hazards. High information map inclusion (specifying the areas affected, areas not affected, and the receiver’s location) in 90-character messages had a statistically significant and positive effect on public response outcomes including interpretation and personalization in the case of active shooter and radiological hazards. Inclusion of a low- information map (specifying the areas affected and not affected, but not the receiver’s location) had the opposite effect. However, in the experiments testing the 280-character messages for tsunamis none of the map elements tested had a statistically significant effect on message outcomes and research participants varied widely in their reactions to the tested maps.
• Inclusion of a hyperlinks. Consideration should be given to including a hyperlink in warning messages of any length. Findings from our community event survey indicated that those who received a message with a hyperlink had a shorter delay (i.e., less milling) before beginning to check media compared to those who did not receive a message with a hyperlink. Delay before avoiding flood areas also was shorter for those who received one or more messages containing a link (compared to those who did not). Focus group research confirmed that including hyperlinks that display additional information may be a useful strategy for expanding the number of characters available for crafting WEA messages, though not all participants supported adding hyperlinks.
• Message source. More research is needed on the impact of message source. For 90-character WEAs, findings suggested that local sources may work best except for well-known federal sources like NOAA/NWS. For 280-character messages results indicated that participants had significantly higher levels of message understanding and message deciding when the message came from NOAA than when the message came from a county emergency management agency.
• Generalizing across hazard types. Like shorter messages that are 90 and 140 characters; 280-character messages likely do not contain sufficient information to overcome people’s pre-alert and warning event perceptions of different hazards based on personal experience, perceived risk, and knowledge, which may or may not match the event they face. Hence, like 90- and 140-character messages, 280-character messages offer less to help effectively manage public protective action-taking than messages that are 1,380 characters.

PUBLIC RESPONSE TO ALERTS AND WARNINGS: OPTIMIZING THE ABILITY OF MESSAGE RECEIPT BY PEOPLE WITH DISABILITIES

Helena Mitchell, Center for Advanced Communications Policy, Georgia Institute of Technology

The Center for Advanced Communications Policy (CACP) at the Georgia Institute of Technology conducted research and development activities to gain a better understanding of how people with disabilities respond to Wireless Emergency Alert (WEA) messages. While these needs were proactively considered during the development of WEA, even today, barriers to access remain. It is critical that impediments to emergency message access be identified, explored, and diminished as the number of people with disabilities and those aging into disabilities increases every day—quickly expanding to nearly 20% of the population. CACP, in support of the DHS/S&T mission and DHS Long Range Broad Agency Announcement 12-7, undertook a number of tasks to address some of these barriers. Project researchers hypothesized that greater awareness and exposure to WEA alerts would increase trust and appropriateness of individual responses to alerts. CACP conducted a national online survey to test the hypothesis and collected data on the availability, awareness, and accessibility of WEAs. The objective was to increase understanding of how to optimize WEA messages, and the devices on which they are received, in a way that encouraged appropriate protective actions. Additionally, numerous development efforts were undertaken, guided by the technical and engineering team. The prototype development was informed by market and user needs analyses. Tasks included testing of WEA-capable handsets, focus groups on experiences with vibration and light features, constructing the prototype, subsequent refinements regarding vibration and light features, and finally the proposed vibration rating (V-rating) scale. The development activity included creating an architectural design for six prototype “handsets” and conducting usability tests with the target population to determine the optimal vibration strengths and the utility of adding a light signal to increase WEA message recognition.

To distinguish the vibration, light and sound qualities within the project, CACP developed the following prototype signals for evaluation:

• WEA Sound. Although evaluating the WEA Sound attention signal was not initially part of the proposed research, we included it in the prototype design to be inclusive of all current and prospective WEA attention signals.
• WEA Light. Adding a WEA light cadence had the ability to increase response time to WEA messages for certain populations. For

• participants who were hard of hearing, WEA Light had the quickest response time.
• WEA All. For participants that had low vision the WEA All (i.e., light, sound, and vibration) received the quickest response time over any of the vibration signals alone. Therefore, we concluded that simultaneously activating all notification signals: sound, vibration, and light would increase the likelihood of timely receipt of WEA messages.
• V-Rating. The backbone of this project was to create a vibration rating scale that would be valuable to the FCC, industry, and consumers. The cost to manufacturers to implement this V-rating should be low. Several measurements are already required by the FCC to sell a product and testing the vibration strength of their device could be done and documented at the same time.

Top Conclusions

CACP’s research yielded the following conclusions:

• People with prior knowledge of WEA were more likely to take immediate action, less likely to be unsure of what action to take, and less likely to make judgments on whether the emergency alert applied to them. Unfortunately, survey respondents without a disability were twice as likely to report having heard of WEA than those respondents with a disability.
• The survey results showed that 98% of the respondents owned a mobile phone. Regarding wireless only households, 40% of respondents with disabilities reported they did not have a landline phone, suggesting that for those respondents, mobile devices are an important communications tool. Therefore, it is imperative that WEA messages and the devices on which they are displayed be optimized for accessibility with respect to the attention signals and message content.
• Our initial proposed research concerning the vibration and light intended only to target people who were deaf and hard of hearing. However, focus group research incorporating diverse groups of people with sensory disabilities found that people with vision disabilities sometimes relied on the sound and visual attention signals as well.
• Assistive technology device alerting mechanisms and WEA-capable cell phones have a wide range of amplitude, frequency, cadence, and duty cycles in their sound, vibration, and light signals. This inconsistency could impact the user’s perception of the WEA vibration cadence (i.e., delayed recognition or none at all). A vibration scale (V-Rating) should be implemented for an end-user to determine the compatibility of a wireless device with their abilities and, therefore, optimize their receipt of WEA

• messages. Publishing the V-Rating scale would help manufacturers promote their devices that fall into the range of user needs.
• Data concerning the vibration, sound, and light attention signals indicated a strong need for manufacturers to improve handset effectiveness for both people with and without disabilities. Consequently, if mobile phone manufacturers design handsets with the capability to adjust the strength of the vibration and sound and to include a light feature, both they and the consumer would benefit.

Research Gaps

As WEA has only been active since April 2012, to truly understand its effectiveness, more empirical and case studies need to be conducted. Longitudinal national surveys are still needed. Following are additional research gaps and areas that need further exploration:

• Validate and expand the V-rating scale. It is unclear as to why subjects responded faster to the low and mid-range vibration setting. The vibration motor used in the prototype vibrated at a different frequency for the stronger setting, which could have impacted detection. Further study of the vibration frequency could ensure optimal vibration settings. A frequency scale could then be incorporated into the V-Rating.
• The majority of survey respondents keep their phone in a case, and a small percentage reported that the case affects the phone’s signaling to its user. Almost half the respondents reported they “did not know” if the notification signal was affected. More research on this topic is needed to determine the impact.
• The majority of devices that are WEA-capable are smartphones, and a significant number of these have multiple accessibility features. As a result, phone accessibility is diminishing as a barrier to receipt of messages. Nonetheless, efforts should be focused on ensuring that WEA-capable devices are developed to have full out-of-the-box accessibility as findings showed that only 8% were fully accessible.
• Explore aspects of message content to determine how to make WEAs clear and actionable (i.e., encourage more individuals to take immediate protective actions, rather than spend time verifying information contained in the alert).

Future Research

Continued research is needed to inform NG-WEA regulatory rule-makings on accessibility. This research should include:

• Identifying a range of light cadences for WEA messages;
• American Sign Language (ASL) video translation of WEA messages; and
• Exploring technical, and user experience requirements for the integration of symbology and other non-text media into WEA messages.
• Developing and evaluating technology and policy options for the integration of wearable technology into the WEA/IPAWS environment. Wearable technologies are a growing market, and according to our 2015 survey results, both people with and without disabilities have adopted its use at the same rate (14%).
• Determining whether people would be amenable to a feature that turns on the device when a WEA message is detected. Respondents with a disability are significantly less likely than respondents without a disability to keep their phones powered on while sleeping, resulting in missing WEA messages that might be sent during sleeping hours.

OPPORTUNITIES, OPTIONS, AND ENHANCEMENTS FOR THE WIRELESS EMERGENCY ALERTING SERVICE¹⁰

Martin Griss, Hakan Erdogmus, and Bob Iannucci CyLab Mobility Research Center, Carnegie Mellon University, Department of Electrical and Computer Engineering

Research Activity

We consider the essential purpose of the Wireless Emergency Alerting system to be the delivery of the right alert messages to the right recipients at the right time via mobile phones subject to (perceived or real) constraints imposed by the cellular networks. The primary goals of our research are to gain insight into WEA adoption and acceptance issues, in particular with respect to perceived poor public response to alert messages, and to develop and test strategies for overcoming these issues through incremental extension of the WEA service architecture. Our studies involved

• A comprehensive Alert Originator Requirements Study (AORS) to build our understanding of how alert originators (AOs) view and use the WEA service,

___________________

¹⁰ This research was sponsored by the Department of Homeland Security, Science and Technology Directorate, First Responders Group (Award HSHQDC-14-C-B0016), to advance the WEA Research, Development, Test and Evaluation Program. The full report is available at the DHS web site [1]. Three summary papers are also available [2–4].

• Exploration of a potential connection between WEA and social media,
• Exploration of improved targeting and content. We developed a prototype of a complete enhanced WEA service and then conducted two week-long trials with 225 subjects across three controlled experiments, exploring the usefulness of various features that could be added to WEA.
• Analysis of the feasibility and conceptual costs of extending the current WEA framework to support new features shown to be valuable.

Research Results

Thirteen in-depth interviews with alert originators in local, county, state, and national roles across a range of alert domains and regions of the country yielded these insights:

• Most interviewees were unable to craft meaningful messages to the general population within the constraint of 90 characters.
• A majority of the interviewees stated that WEA needs increased geographic precision to deliver alert messages effectively.
• The WEA service needs to interface with social media to be relevant.
• The nature of WEA is bifurcated: some AOs perceived a WEA message as a “bell ringer” technology while others believed that wireless alerts should directly embed or reference additional information.
• Interviewees agreed that not enough has been done to educate the general population about what WEA messages are, why they are important, and how the public should respond.

These insights drove us to focus on means to improve both targeting and content efficacy of alerts. Targeting is currently based on the assumption that presence in a targeted area indicates interest, and absence indicates disinterest—which may well be false. We call this misconception the location proxy fallacy.

In terms of content efficacy, WEA remains tied to the presumed constraints of 2G cellular networks (limited bandwidth) and an assumption that the mode of delivery (simple SMS-like messaging with simple pop-up per-message alerts) meets user expectations. The reality is that today’s LTE networks have far more capabilities, and user expectations have evolved over the last twenty years from text to rich media. On top of this, the AO’s task in a complex, large scale disaster will be to send multiple, interrelated alerts reflecting an evolving situation. We refer to the presumed need to keep WEA tied to antiquated network limits as the short message fallacy.

Repeated alerting when the recipient is, in fact, not interested or poor content can lead the recipient to disable WEA on his/her phone. The

recipient will then miss future alerts that would have been relevant. We call this the opt-out problem. To address these concerns, we hypothesized and evaluated a variety of new mechanisms. Three of these emerged as showing the greatest potential to improve the WEA service:

• Geo-targeting: Fine-grained, precise geo-targeting based on filtering on the phone rather than in the network and aided by a compressed representation of the target area markedly improved alert relevance to the recipients.¹¹
• Location History: Location-history-based filtering combines geotargeting with interest-based targeting (using content, rather than location, as the determining factor). The use of location history to make alert delivery decisions significantly improved relevance to recipients compared to unfiltered alerts.
• Situation Digest: We developed and evaluated smartphone software capable of digesting streams of structured alert messages and presenting these as consolidated views of the latest information. We measured recipients’ situational awareness using their time-dependent, aggregate understanding of the type, recommended action and immediacy of an underlying emergency scenario. Overall, there was a marked improvement in situational awareness compared to the single stream normal WEA view. The feasibility study demonstrated a way to graft this on top of existing WEA.

We believe that recipient-context-specific alert personalization (overturning the location proxy concept) and situational digests (overturning the short message fallacy) may together mitigate opt-out and improve actionability. The interested reader is referred to the full report.¹²

Identified Gaps in Today’s WEA

Our study resulted in five major conclusions about WEA:

• Deep integration of location-based context materially improves WEA’s value. We found strong AO support for this, and our experimentation and feasibility studies demonstrated efficacy and practicality.

___________________

¹¹ A. Jauhri, M. Griss, and H. Erdogmus, 2015, “Small Polygon Compression for Integer Coordinates,” paper presented at the Third Conference on Weather Warnings and Communication, June 12, Raleigh, NC, https://ams.confex.com/ams/43BC3WxWarn/webprogram/3WXCOMM.html.

¹² M. Griss, H. Erdogmus, and B. Iannucci, 2015, Opportunities, Options and Enhancements for the Wireless Emergency Alerting Service, Washington, DC: Department of Homeland Security.

Our research shows support from AOs and from the experiments for a fundamental change in WEAs focus from issuing alerts to creating awareness,¹³ pointing to abilities to digest complex situational information and to support the needs of complex and evolving situations.

• WEA must be well positioned as a peer in the social networking pantheon. WEA serves a critical role as an alarm bell. But the inherent limitations of WEA further suggest that it cannot and will not be the only source to which the public will turn.
• Education, testing and measurement are necessary for WEA’s success. Our interviews and surveys revealed the need for education of both AOs themselves and the general public. This feedback correlates with results of previous studies.^14,15 We were surprised to learn how few AOs in our study had actually used WEA. AOs further highlighted the need for regular systematic testing of WEA with the public—not unlike Civil Defense, the EBS, and the EAS.
• Rich media integration into WEA is a question of how, not if. Our research, like that of others, provides support for the integration of rich media into WEA. We believe this arises from the fundamentals of widespread use of smartphones and mature, HTML-based information authoring tools.

Future Research

In our final report, we identify future work including larger scale evaluations, further studies into message compression, the use of URLs and/or images in alerts, and other ideas. Additional concepts include:

• Exploiting the capabilities of smartphones: Re-cast on-phone WEA handling software as an app that can be securely updated and evolved without upgrading the phone per se. This affords continual improvements as user expectations, network capabilities, and current alerting research evolve. Legacy WEA can be maintained in parallel for some years to accommodate the long tail of recipients with 2G phones.

___________________

¹³ B. Iannucci, J. Falcao, H. Erdogmus, M. Griss, and S. Kumar, 2016, “From Alerting to Awareness,” paper presented at the 2016 IEEE International Conference on Technologies for Homeland Security, May 12, Waltham, MA, https://ecfsapi.fcc.gov/file/60002085022.pdf.

¹⁴ Carnegie Mellon University, 2014, Study of Integration Considerations for Wireless Emergency Alerts, CMU/SEI-2013-SR-016, Pittsburgh, Pa., http://repository.cmu.edu/cgi/viewcontent.cgi?article=1781&context=sei.

¹⁵ M. Wood, H. Bean, B. Liu, and M. Boyd, 2015, Comprehensive Testing of Imminent Threat Public Messages for Mobile Devices: Final Report, National Consortium for the Study of Terrorism and Responses to Terrorism.

• Extending communications for resilience and local access: Exploit capabilities of smartphone RF subsystems to allow wireless alerting even when the recipient’s carrier network is down. Police and other authorities should have the ability to wirelessly broadcast alerts over their own, hardened systems in a way that smartphones will recognize and display as part of a survivable communications strategy.¹⁶
• Enriching alert creation: Enable AOs to author rich (HTML-based) content, augment CAP to carry it, and expand WEA to WECAP.¹⁷ Rendering should use the browser mechanisms built into phones. Take advantage of LTE broadcast and remove the short text limit. Digitally sign all alerts.
• Closing the alerting loop: Today, WEA is open loop in the sense that alerts go out and AOs only see the results in terms of the collective actions of the served population. Examine the possibility of closing the loop: provide for recipient responses in future alerting apps (e.g., a button saying “this alert was not relevant for me”) to enable deeper studies of alert targeting. The responses need not come back during the emergency but can be trickled back over days following an alert so as to not create inappropriate network load.

GEOTARGETING

Wireless Emergency Alerts in Arbitrary Sized Target Areas: Mobile Location Aware Emergency Notification

Emre Gunduzhan, Applied Physics Lab, Johns Hopkins University; Bharat Doshi, US Army CERDEC (formerly JHU/APL); Lotfi Benmohamed, NIST (formerly JHU/APL); Jay Chang, Applied Physics Lab, Johns Hopkins University; Osama Farrag, Applied Physics Lab, Johns Hopkins University

The Wireless Emergency Alerts (WEA) service provides the ability to send geographically targeted text alerts to the public. However, the current WEA geotargeting mechanism is limited by the relatively coarse granularity of cellular network sites and numerous benefits of more accurate geotargeting for public alerts and warnings have been identified in earlier studies. The Department of Homeland Security (DHS) Science and

___________________

¹⁶ B. Iannucci, J. Cali, R. Caney, and S. Kennedy, 2013, “A Survivable Social Network,” paper presented at the 2013 IEEE International Conference on Technologies for Homeland Security, Waltham, MA, http://repository.cmu.edu/cgi/viewcontent.cgi?article=1174&context=silicon_valley.

¹⁷ B. Iannucci, J. Falcao, H. Erdogmus, M. Griss, and S. Kumar, 2016, “From Alerting to Awareness,” paper presented at the 2016 IEEE International Conference on Technologies for Homeland Security, May 12, Waltham, MA, https://ecfsapi.fcc.gov/file/60002085022.pdf.

Technology Directorate (S&T) has engaged the Johns Hopkins University Applied Physics Laboratory (JHU/APL) to investigate methods that can improve the accuracy of the WEA geotargeting mechanism.

JHU/APL proposed a new WEA geotargeting mechanism, called Arbitrary-Size Location-Aware Targeting (ASLAT), and conducted several analyses to characterize the performance of the new mechanism and to assess feasibility of its deployment. JHU/APL compared the performance of the new mechanism with existing WEA geotargeting, developed functional requirements for the new mechanism and identified the required changes to existing WEA and applicable geolocation standards.

ASLAT utilizes the location awareness of mobile devices to improve geotargeting accuracy. In ASLAT, WEA alerts are broadcast to an area wider than the target area, but are only displayed to the user if the mobile device is inside the target area. This approach eliminates the false positives and the false negatives that occur due to the mismatch between the shape of the target area and the shape of the set of cellular network sites selected to broadcast the alert. In addition to enhancing the geotargeting accuracy, ASLAT would enable people to receive alerts when they are in the vicinity of a target area and have interest in a particular location inside the target area.

Performance analysis of ASLAT shows that it can improve the geotargeting accuracy of WEA significantly without consuming excessive mobile device power or radio resources. ASLAT introduces some delay in delivering alerts because mobile devices need to learn their location before processing a received alert. However, the maximum delay introduced by ASLAT can be controlled by a configurable parameter. The mobile device can use the default WEA behavior if the ASLAT delay reaches this maximum value. We used a maximum ASLAT delay of one minute in the analyses. Highly delay-sensitive alerts such as earthquake warnings would bypass ASLAT automatically and be processed using the default WEA behavior. These alerts would be displayed to the user immediately as in current WEA, without comparing the location of the mobile device with the target area.

ASLAT depends on a variety of geolocation technologies to determine the location of a mobile device. We investigated different geolocation technologies to see what technologies would be suitable for use with ASLAT and concluded that mobile-device-based technologies should be used. These geolocation technologies include the Global Positioning System (GPS), mobile-device-based Time Of Arrivals (TOA) and Time Difference Of Arrivals (TDOA) techniques and Wi-Fi proximity. These are all suitable for ASLAT because they provide adequate location precision, they do not introduce additional load on the cellular network and they maintain user privacy.

ASLAT would require some changes to existing standards. Specifically, WEA standards that specify cellular network functionality and mobile device behavior would require amendments to support ASLAT. Some modifications to GPS and TOA/TDOA standards would further enhance ASLAT performance. Since these standards have been developed primarily for navigation and E911 services, extensions to support emergency alerting and the impact of these extensions on existing services need further study. Finally, modifications to Wi-Fi standards and implementation of new indoor location capabilities are needed to enhance geotargeting accuracy indoors.

The proposed geotargeting mechanism and the related results could affect important technical, programmatic and policy decisions regarding the evolution of the WEA system. The DHS S&T WEA Program Management Office should work with other stakeholders, including the Federal Communications Commission, Federal Emergency Management Agency, cellular service providers, the Alert Originator community and state and local first responders, to determine detailed requirements on geotargeting accuracy and to further analyze various alternatives to meet these requirements.

Geo-Targeting Performance (GTP) of Wireless Emergency Alerts (WEA)

Dan Gonzales, RAND Corporation

The objectives of this study were to evaluate the public benefit and performance trade-offs of geo-targeted WEA messages using alternative WEA antenna selection methods and to identify the optimal WEA radio frequency geo-targeted areas for imminent threat scenarios. This briefing addresses these questions for two imminent threat scenarios: tornado warnings and earthquake early warning. The briefing concludes with the following recommendations.

Employ WEA Antenna Selection Method 2 In Urban and Mixed Areas

We found WEA Method 1 provides lower over alerting rates (OARs) than Method 2 in tornado warnings. However, if Alert Failure Rate (AFR) is considered to be the primary metric used to assess WEA GTP, Method 2 provides superior GTP in urban and mixed areas. Because of RF spillover effects it is not clear which WEA antenna selection method is better in rural areas.

Upgrade Sirens and the WEA Service to Improve Geo-Targeting of Siren Tornado Warnings

Previous studies found that siren-based warnings were ignored by residents under threat because of alert complacency or fatigue. Complacency occurred because tornado warning sirens were sounded on a countywide basis in many areas. WEA can be used to geo-target siren tornado warnings. A WEA compatible receivers should be installed on sirens.

Explore the Implications of the FACETs and Threats in Motion (TIM) Initiatives for WEA

Forecasting a Continuum of Environmental Threats (FACETs) is an NWS initiative to improve the NWS forecast and warnings for high-impact weather events. FACETS messages include nested, color-coded polygons to indicate the level of threat present in each area. The current version of the WEA service cannot support FACETS.

Another NWS initiative is Threats In Motion (TIM). TIM will improve tornado warning geo-targeting by updating the position of the warning polygon more rapidly. Today tornado warning polygons may remain fixed for hours. TIM warning polygons are updated every minute. TIM presents a number of challenges for the WEA service. TIM would require the transmission of more WEA messages. The capacity of the IPAWS aggregator will likely have to be increased to support this increased message load. Testing will be needed to ensure WEA can handle TIM-based tornado warnings.

The Federal Communications Commission (FCC) and DHS are considering upgrades to WEA. Such upgrades should consider the implications of FACETS and TIM based tornado warnings. The FCC and DHS should consider changes to WEA that will enable FACETS tornado warnings to one day be transmitted as WEA messages.

WEA Testing Is Needed to Determine Whether WEA Preserves Tornado Warning Lead Time and Can Support Earthquake Early Warning (EEW)

The average lead time or warning time provided by NWS tornado warnings is about 13 minutes. While there appears to be no formal WEA message latency requirement, previous industry studies indicate WEA message latency may be as high as 12 minutes. If WEA tornado warnings are delayed by this much then almost all of the lead time provided by NWS tornado warnings would be consumed by time delays within the WEA service infrastructure.

An EEW sensor network is under development in California. However, a means has yet to be identified for transmitting EEW messages to the public using cell phones. An EEW system could provide 40-60 seconds of warning prior to the start of the worst shaking, which could save many lives in a large earthquake. To provide such warning the EEW message would have to be disseminated in 10 seconds or less to be effective.

A recent industry study determined that the current version of WEA could not support EEW because of WEA message time delays, without presenting specific evidence of this assertion. The timeliness of the WEA service has never been evaluated in an end-to-end test. Such testing is needed to determine how effective WEA tornado warnings are and whether WEA EEW is feasible. Cell broadcast-based warning systems in other countries may have much lower message latency than the U.S. WEA service. If it is found that WEA service message latency is high, it should be technically feasible to reduce these time delays. WEA testing can determine if this is necessary.

DHS or NWS Should Conduct an Education Campaign to Inform the Public that WEA Geo-Targeting Is More Accurate than Sirens

Previous studies indicate that a large percentage of the population ignores siren tornado warnings because of over alerting. Many people may not be aware that WEA tornado warnings can be geo-targeted more precisely than sirens, so they may also ignore WEA tornado warnings. To prevent this an education campaign is required to inform the public of the superior geo-targeting performance of the WEA service.

Develop Tools to Help Alert Originators Estimate WEA Local Area Coverage

The coverage provided by wireless cellular networks can vary significantly from one region to another. This is especially true in rural areas where cell towers are sparsely distributed over the terrain. In contrast in urban areas cellular coverage is generally good and cell sizes are small, which leads to high WEA geo-targeting accuracy. Alert originators in rural areas may therefore have greater uncertainty as to how far a WEA message will propagate and where to draw a warning polygon in an imminent threat scenario. New tools for alert originators that provide WEA coverage estimates would be valuable in such environments.

Exploring the Effect of the Diffusion of Geo-Targeted Emergency Alerts: The Application of Agent-Based Modeling to Understanding the Spread of Messages from the Wireless Emergency Alerts (WEA) System

Andrew M. Parker, Brian A. Jackson, Angel Martinez, Ricardo Sanchez, Shoshana R. Shelton, & Jan Osburg, Homeland Security and Defense Center, RAND

The complexity of individual behaviors in emergency alerting situations, and the way those behaviors affect the ability of alerts to serve as a protective measure, has long been a topic of interest. With the evolution of new relationships between citizens and technology—which have affected the utility of legacy alerting modes and the arrival of new options like WEA—new facets have been added to that complexity. The ability to geo-target alerts to mobile devices provides new capability, but it also raises questions about how to use geo-targeting effectively and how the interaction between targeted messages and human communication behavior will affect diffusion of messages beyond the area where they are delivered initially. This project took on the question of how important diffusion behavior was for understanding the value of geo-targeting WEA messages. The study used agent-based models (ABMs) to examine the diffusion of alerts within an area populated by individual recipients who move from place to place in the course of their daily activities.

Building a Model of Emergency Alerting and Diffusion

An ABM is a computer simulation where a population is represented by individual “agents” (in our model, representing people in at-risk areas) who interact and make individual choices about their behavior based on a set of rules. In our model, the agents could move geographically, since understanding the effect of individuals moving in and out of areas where geo-targeted alerts were sent was the central aim of our work. The model was represented a chain of events, including (1) receipt and understanding of an alert message, whether from WEA or another alerting channel, (2) the decision by the agent to share or not share that alert with others, the central mechanism for alert diffusion in the model, and (3) the decision by the agent whether or not to take whatever protective action was recommended. Agents resided and moved in a realistic geographic landscape, including a network of roads.

We considered four alerting scenarios: (a) flash flood, a frequent focus of past WEA messages with a small area, short timeline, and evacuation goal, (b) tornado, also a frequent WEA focus but covering a larger area with a shelter-in-place goal, (c) hazmat plume, a small-sized event

with the potential for multiple hazard areas with differentiated protective actions (evacuation or shelter), and (d) major flood, a rarer but larger and slower event, with the potential for multi-stage evacuation strategies.

By distilling the elements of alerting to a set of simplified parameters, the model allowed us to look at how changing some of those parameters could affect outcomes—dialing up and down the parameters that shaped how people communicated and forwarded messages to get insights into how that behavior could become a multiplier for alerting even as it fought efforts to precisely geo-target alerts.

Key Conclusions from the Model

Our initial framing of the study considered forwarding as a potential threat to the value of geo-targeting. This perspective considers forwarding as producing a practical limit on geo-targeting precision. The simulations certainly showed this—as would be expected—but the relationship was more nuanced. Very tight geo-targeting will always be compromised some by forwarding, but if very few agents are in the alert zone (i.e., are in or near the emergency zone) that compromise will be small, and hence should not hurt the value of investing in geo-targeting. Diffusion is likely of greatest concern for intermediate-sized events, where the population to be alerted is reasonably large, yet the geographic area is small enough to suggest value in precise geo-targeting. As the number of people being alerted in our simulations went up, the number of potential forwarders also went up, and the ability to target precisely went down—though, depending on the scenario (e.g., how quickly it evolved), that could matter to differing extents.

That said, instead of considering forwarding as a threat to geotargeting, it might be more valuable to think of forwarding as a compliance enhancer. Significant forwarding could increase the total effect of alerting a great deal. In that sense, forwarding converts the set of individuals’ social networks into a new mass alerting channel of its own. In this way, forwarding or communication among individuals about the alert, here via such electronic means as social media, is simply one more type of the communication that has always occurred during milling before citizens make the decision to comply with the alert. The price of this effect is significant increases in out- of-emergency-zone alerting and potentially unnecessary action in response.

The model also taught us that precise geo-targeting can have different meanings when we consider how agents are dynamic, rather than thinking about geo-targeted alerts being aimed at a set of agents that is sitting still “waiting for the alert to arrive.” We may want to get the message not

only to people who are in the emergency zone but also to those contemplating entering the emergency zone.

Our results demonstrate quite clearly that forwarding threatens the value of trying to deliver different messages to different geographic areas in an effort to either provide messages relevant to individual risk areas or to guide population behavior in ways designed to enable more-effective response (i.e., time-phased evacuation). The dynamics of forwarding and compliance mean that success in these sorts of differential alerting efforts would require more-nuanced communication strategies (e.g., telling later staged-evacuation zones to “shelter for now”).

Limitations

Perhaps the most important caution to be placed on these results derives from the very nature of such a modeling exercise, which by design is an abstraction from reality. All models are abstractions, and aspects of reality may not have been completely captured (e.g., the current model does not capture non-verbatim forwarding of alerts). As an abstraction from reality, it is important to remain clear that the model itself relies on the estimates of the parameters that serve as its basis, as well as the scenario specifications.

Conclusions and Future Directions

Complexities of human behavior—including both message forwarding and movement—mean that the use of geo-targeting is not as simple as just restricting the transmission of an alert to the smallest area at risk from an emergency event. The ability to transmit messages to smaller areas—which is indeed a major technical jump in emergency alerting capability—requires similar innovation in policy and practice to ensure that emergency managers are outfitted with the best understanding and tools to make the best choices regarding the use of geo-targeted alerts during the critical and time-limited decisions made in the warning phase of natural, technological, or other emergency incidents.

This project demonstrated the value of agent-based simulation models for capturing key complexities—such a social networks and geographic movement—in ways that wouldn’t have been possible otherwise. These or similar models could be used to run experiments examining policy questions, such as the strategic use of forwarding as an alerting channel, conditions under which over-alerting pays dividends, and the impact of new WEA capabilities or new technologies.

Using RF Coverage to Improve Geotargeting Granularity and Accuracy for Delivery of WEA

Dara Ung, Comtech Telecommunications Corporation

Comtech TCS carried out research and development on constructing geo-targeting algorithm that utilizes Radio Frequency (RF) cell site propagation footprints. Research focused on using RF coverage area footprints to improve geo-targeting granularity and accuracy for delivery of Wireless Emergency Alert (WEA) messages. The project consists of field testing activities, conclusions, and the analysis of the results that used the enhanced geo-targeting algorithm previously developed for this research.

The WEA standard (J-STD-101) defines two methods that can be used to select cell towers to deliver WEA messages for a given targeted geographical area. The first method calls for the ability to determine the cell towers at the county level of granularity. This level of granularity is a minimum requirement for all mobile carriers that offer the WEA service. The second method is optional and allows the targeted area to be defined by polygons instead of fixed county boundaries and determines if the targeted cell tower physical position (latitude/longitude) is found inside the target area polygon. Both of these methods have been found to be highly inaccurate as the alert target areas become smaller and therefore cannot be used to issue alerts that require target area size to be within a few square miles. This inaccuracy introduces situations known as “over-alert,” when an alert reaches population that is not intended for, or “under-alert,” when the alert does not reach the people in harm’s way.

This research included the modification of WEA software using enhanced geo-targeting algorithms that take into account more than just the physical location of cell towers. The algorithm was tested both in the laboratory and in the live production environment. The outcomes of the research include obtaining the live test results that are keys to validate the lab simulation. The results will also confirm the successful development of tools and software needed to collect the data in the live environment without impacting (due to sending test alerts) to the public.

The test results obtained from the field clearly demonstrated the strength and weakness of both the existing and the new enhanced methods. The results show that the enhanced algorithm using cell RF propagation footprints is convincingly superior to any existing method used today. When implemented, the enhanced method developed in this research will provide new benefits to WEA users in several ways, including:

• The ability to target much smaller alert areas down to a square mile regardless of the physical location of the cell towers;

• The ability to use location based Required Monthly Test (RMT), allowing WEA alerts to be tested at chosen live site without impacting the general public;
• Geo-targeting at the cell sector granularity;
• Enhanced reachability to the people in harm’s way;
• The ability to enable other alert categories to be defined because of allowable small alert target area size; and
• A solution that requires no change to the current WEA network.

The key lessons learned in this project consist of understanding the effects of a live environment and how different real-world factors can affect the expected results. These lessons learned are very important because they allow WEA vendors to improve the techniques that will ultimately enhance their solution in the future.

Due to the limitation of the cell broadcast technology, no geo-targeting method can provide 100 percent accuracy. Based on the results obtained, however, the algorithm that uses cell tower RF propagation footprint clearly offers better accuracy than the methods used to date. Although this method will not solve the over-alerting problem within the cell sector level, it will improve the reachability to people in harm’s way very effectively. Since the alert target area size can now be defined as small as a square mile, over-alerting can significantly be reduced. Therefore, this method will be suitable for such alerts as a campus emergency, a chemical spill or a road block due to a major accident. These instances would not be possible using the various methods available today.

The cell RF propagation footprint algorithm could be provided as the best-effort solution for cell broadcast technology currently available. The attractiveness of this method is that it does not require any change in the standards and specifications for it to be deployed today. The existing WEA regulatory mandatory requirement for geo-targeting is limited to county-level only. It is therefore recommended that the regulatory requirement be changed to obligate the service providers to offer WEA service with geo-targeting at cell sector level accuracy.

Given the limitations discussed in this document, further improvement can still be made, perhaps in cooperation with a mobile device application developers or manufacturers. The Dynamic Plus method allows a very small area to be targeted and can identify exactly the list of cell sectors affected by this area. Cell sectors, however, can be quite large and can extend for several miles in rural areas. Over-alerting can therefore extend for miles. To avoid this problem, intelligence in the mobile device is needed. A GPS capable mobile device knows its own location. Thus, if the target area LAT/LONs or circle can be conveyed to the mobile device over the cell broadcast message, the mobile device would be able to

determine if it is located inside the target area and subsequently notify the user. Otherwise no alert will be triggered on the device. Such combined technology would provide the best possible result and is recommended as a subject of future experiment.

TECHNOLOGIES

Accessible Common Alerting Protocol Radio Data System Demonstration: Gulf Coast States

Rich Rarey, NPR Labs

The goal of this project was to create and demonstrate end-to-end accessible radio emergency alerting using Common Alerting Protocol (CAP) messages from the FEMA’s Integrated Public Alert and Warning System (IPAWS) aggregator. The resulting alert messages were specifically formatted for and targeted to deaf and hard of hearing individuals living in the U.S. Gulf Coast. It was understood that deaf and hard-of-hearing residents living in the weather-vulnerable Gulf States region would benefit directly from receipt of emergency messages that are currently provided in an audio format to hearing individuals.

NPR proposed using existing satellite delivery and broadcast technologies for secure and timely transmission of text alerts in an innovative way: Create a delivery system for emergency information relying on broadcast radio technology that is accessible and available regardless of power outages, Internet disruptions or limitations of cellular service.

Tools created during the project were designed to be used by Public Radio Satellite System (PRSS)-connected FM radio stations and to have broad application for adoption by other commercial and non-commercial FM broadcasters using Radio Data Systems (RDS). RDS is a long-established data subchannel technology, typically used by an FM station to broadcast multiple data streams containing traffic data, artist/song/album data, and similar information.

NPR Labs used the PRSS distribution path to reach 26 public radio stations scattered across Alabama, Florida, Louisiana, Mississippi and Texas. The stations locally broadcast the alerts to deaf and hard-of-hearing participants who received the alerts on specially designed ‘visual’ FM radio receiver. The disseminated emergency messages from FEMA’s test platform, using the Emergency Alert System (EAS) architecture, demonstrated how attributes of the CAP message, embedded within an RDS signal, can greatly expand the distribution of EAS alerts for the nation’s 36 million citizens who have hearing loss.

To build the demonstration, significant hardware and software development was required, as well as system integration to provide a way to access, process, and transmit text emergency messages. Vendors were identified and recruited to help design and manufacture a specialized message encoder for the stations, and mass-produce an easy-to-use, specialized ‘visual’ FM radio receiver for the deaf and hard of hearing users. Additionally, software was developed that automated the relay of the IPAWS alerts from reception at NPR, through the PRSS satellite system, through the stations, to the users, thence displayed to the deaf and hard of hearing participants.

Two rounds of consumer field tests were conducted during the summer of 2014. Sample messages were sent from the PRSS Network Operations Center to local public radio stations for broadcast to field-test participants. Consumers read the messages, some up to 4,000 characters in length, on an Android tablet running an application developed for the project that linked the tablet to the specialized FM-radio receiver. The FM receiver automatically tuned to the strongest signal of a participating station and was designed to be operated by battery and/or wall power. It had a connector to activate a bed shaker to awaken the user. During an alert, bright lights on the FM receiver and the bed shaker notified the user that a message was being received and the text of the alert, on the Android tablet screen, advised the reader of the emergency and actions to be taken. Participants were surveyed daily for their impressions and experiences. Results from the first set of tests were used to improve and upgrade software for the second round of tests.

The project employed best practices, and identified areas in technology development and design, engineering management, station installation, recruitment and management of field-test participants that could greatly improve future product development and testing methodologies. The specialized FM radio receiver developed for this project, dubbed the NIPPER ONE, was a 2014 International Consumer Electronics Show Innovations Design and Engineering Award Honoree. The project work was featured by the White House at its Innovation for Disaster Response and Recovery Day in July 2014. The novel design of the RDS Emergency Alert signaling was incorporated into the National Radio Systems Committee standard NRSC-G300-A Radio Data Systems (RDS) Usage (April 2014) and the subsequent update NRSC-G300-B (September 2014).

Recommendations and next steps are in two categories: Refining technological understanding and delivering non-English text alerts. Most critical task in the first category is developing a terrain-based prediction model for RDS coverage (including FM transmission injection level variables) to more accurately predict where and how well users will receive alerts. Also required are enhanced error correction of incoming messages;

improving the look, feel of the consumer hardware while reducing cost; and reducing the cost of station based equipment.

While this project made English language text alerts available, follow-on work could focus on investigating and developing methods to create emergency alerts for non-English speakers. Similar technology could be utilized to deliver alerts in other languages, such as Spanish, Vietnamese, and Chinese.

SEI Wireless Emergency Alerts (WEA) Research 2013 Through 2016

Carol Woody, Software Engineering Institute

Research Activities

The Software Engineering Institute (SEI), a federally funded research center sponsored by the Department of Defense, addressed a research need from Denis Gusty, Department of Homeland Security Science & Technology Division, to assist alert originators (AOs) in their efforts to implement Wireless Emergency Alerts (WEA). SEI developed an integration strategy to aid AOs in adopting and utilizing (WEA). In addition, best practices for addressing requirements, vendor product selection and acquisition, technology choices such as cloud technology, cybersecurity risks, testing, and implementation were assembled and published. Twenty-eight emergency organizations were contacted and fourteen vendors were interviewed. Workshops were conducted to identify concerns and develop feasible approaches. In-depth analyses focused on issues of trust and cybersecurity risk were conducted. The papers were published in the following areas:

• Integration. Commercial Mobile Alert Service (CMAS) Alerting Pipeline Taxonomy and Study of Integration Strategy Considerations for Wireless Emergency Alerts
• Best Practices. Best Practices in Wireless Emergency Alerts; Wireless Emergency Alerts: New York City Demonstration; Maximizing Trust in the Wireless Emergency Alerts (WEA) Service; and Wireless Emergency Alerts: Trust Model Technical Report
• Security. WEA Cybersecurity Risk Management Strategy (for AOs); WEA Security Pocket Guide (delivered as a section of best practices); Mapping WEA Security Requirements and Guidance to Cybersecurity Risk Mitigation Recommendations (delivered separately to DHS - some requirements are restricted); and INCOSE Insight essay, Evaluation of Security Risk for WEA Alert Originators Using Mission Threads (published July 2013, Volume 16, Issue 2)

In a follow on project, SEI assessed Commercial Mobile Service Providers (CMSPs) cybersecurity risks that affect the WEA service and developed the Wireless Emergency Alerts CMSP Cybersecurity Guidelines which were published.

Research Results for Alert Originators

Research completed by SEI provides the following advice for alert originators:

• Requirements. Spend the time to identify the key requirements, and specify them meaningfully including what the system must do (functional requirements) and how well the system must operate (quality attribute requirements).
• Cloud trends. Know what quality of service you need, and ask how the vendor will achieve it; know how to look beyond the jargon and hype.
• Cybersecurity. Learn about the security risks associated with modern alerting technologies and establish a culture of good security; Implement a cybersecurity risk management strategy.
• Product selection: Confirm WEA capabilities before purchasing the product; develop a customized prioritization method that documents the progression from operational expectations to prioritized features; those that lead tradeoff discussions should acquire sufficient knowledge of tradeoff definitions and consequences to lead these design discussions; address the role of a “lead integrator” if you will use multiple vendor products.
• Testing. Attend FEMA IPAWS webinars and outreach sessions; conduct periodic tests of the individual system and software to include interface testing between the alerting software and IPAWS-OPEN using available testing platforms; periodically conduct end-to-end testing using available testing platforms.
• Operations. AOs need a method for identifying the operational impacts of WEA; they need to ad-dress and manage operational challenges prior to an emergency incident; Good practices can assist in sending rapid, clear, and timely messages; large-scale exercises and training are important to exercise cross-agency and cross-system scenarios; there are cross-organization coordination challenges in issuing WEA messages; there are challenges in synchronizing WEA information with other media channels.
• Alternatives to buying a WEA solution. There are options for obtaining a WEA solution, and each has advantages and disadvantages; there are special considerations for developing your own WEA solution; there are important considerations related to authentication and message validation; there are challenges with error-handling message propagation.

Challenges for Trust and Cybersecurity

Alert originators do not have a security mindset at the leadership and management level; security is seen as “someone else’s job.” CMSPs have taken steps to address cyber security concerns and the guidelines provide a means for them to review their current security position, identifying gaps, and chart a course for improvement.

WEA, like all other cyber-enabled systems, is subject to technology weaknesses and cyber threats that may prevent its use or damage the credibility of the service that it provides. Attackers may attempt to delay, destroy, or modify alerts, or even to insert false alerts. These actions may pose a significant risk to the public.

Trust is a key factor in the effectiveness of the WEA service. AOs must trust WEA to deliver alerts to the public in an accurate and timely manner. Alert recipients must also trust the WEA service before they will act on the alerts that they receive. All participants in the WEA Pipeline must work together to ensure trust is maintained.