Introduction
Measuring and analyzing public opinion comes with tremendous challenges, as evidenced by recent struggles to predict election outcomes and to anticipate mass mobilizations. The National Academies of Sciences, Engineering, and Medicine publication Measurement and Analysis of Public Opinion: An Analytic Framework (hereafter referred to as the Analytic Framework) presents in-depth information from experts on how to collect and glean insights from public opinion data, particularly in conditions where contextual issues call for applying caveats to those data (NASEM, 2022). The Analytic Framework is designed specifically to help intelligence community (IC) analysts apply insights from the social and behavioral sciences on state-of-the-art approaches to analyze public attitudes in non-Western populations.
Sponsored by the IC, the National Academies’ Board on Behavioral, Cognitive, and Sensory Sciences hosted a 2-day hybrid workshop on March 8–9, 2022, to present the Analytic Framework and to demonstrate its application across a series of hypothetical scenarios that might arise for an intelligence analyst tasked with summarizing public attitudes to inform a policy decision (see Box I-1 for the workshop’s Statement of Task and Appendix A for the workshop agenda). Workshop participants explored cutting-edge methods for using large-scale data as well as cultural and ethical considerations for the collection and use of public opinion data.
OPENING REMARKS
Charles Lau (co-lead expert contributor for the Analytic Framework and director of the International Survey Research Program at RTI International) described the work of intelligence analysts as particularly challenging: they are expected to evaluate disparate sources of information that are rarely designed to address the question at hand and that could be of varying levels of quality, and to provide recommendations to policy makers in extremely brief time frames. He invited workshop participants to consider how to apply the Analytic Framework to real-world situations as a means to best assist intelligence analysts in their day-to-day tasks.
The Analytic Framework
Lau explained that the Analytic Framework is not, for the most part, a prescriptive document; rather, it offers approaches to inform the work of intelligence analysts who are evaluating public opinion research in international contexts. It serves as a guide both for the collection of new data and the assessment of existing data, and is meant to be (1) rigorous, (2) helpful, and (3) suitable for a wide range of audiences. First, leading academic and applied research was produced and compiled for the Analytic Framework by an interdisciplinary panel of authors and expert contributors. Second, because intelligence analysts encounter different challenges each day, the Analytic Framework offers both a broad set of tools that apply for various intelligence analysts in different regions and contexts as well as focused checklists to structure tasks. Third, to address the varied levels of experience with public opinion research throughout the IC, the Analytic
Framework is presented in three layers—the foundational layer includes four commissioned academic papers, the synthesis layer provides a high-level summary of these academic papers, and a graphic layer illustrates key messages about relevant phases of data collection and analysis.
Workshop Goals
Lau presented four goals for the 2-day workshop: (1) provide an overview of the Analytic Framework’s components—the foundational papers, the synthesis, and the graphic depiction of the phases involved in collecting and analyzing public opinion data; (2) examine key messages in the Analytic Framework and discuss what it does and does not do; (3) explore cultural and ethical considerations in the collection and use of public opinion data; and (4) engage in several exercises applying the Analytic Framework to theoretical scenarios and address intelligence analysts’ questions about its application.
ORGANIZATION OF THIS PROCEEDINGS
Chapter 1 describes the role of public opinion in intelligence analysis, with particular emphasis on the challenges that analysts face amidst a continually evolving landscape. Key ethical and cultural considerations for the use of public opinion data are discussed, and a more detailed overview of the goals for, structure of, and responses to the Analytic Framework is provided. Chapter 2 presents four hypothetical scenarios and the ensuing conversations about how an intelligence analyst could use the Analytic Framework to approach them. Chapter 3 summarizes insights from the Analytic Framework and the workshop, and highlights areas for future discussion.
This proceedings document has been prepared by the workshop rapporteur as a factual summary of what occurred at the workshop. The workshop planning committee’s role was limited to organizing and convening the workshop (see Appendix B for biographical sketches of the planning committee members, expert contributors, authors, and other workshop participants). The views contained in the proceedings are those of individual workshop participants and do not necessarily represent the views of all workshop participants, the planning committee, or the National Academies.
This page intentionally left blank.
