drivers from the work in subsequent layers to help the intended audience apply knowledge to the situation of interest. Layer 2 is an authored summary of the four Layer 3 papers that includes a callout to key points from each paper. Finally, Layer 3 comprises four authored papers that review literature from various disciplines containing overview information as well as how the topic/situation of interest can enable better analysis across different situational constraints.

Additionally, each of the three layers is designed with a particular audience in mind. Layer 1 has two primary audiences: (1) an analyst with an extremely limited amount of time to understand a topic; and (2) senior policy makers who have little time to make a decision. Layer 2 is designed

for an analyst with a relatively short period of time to quickly understand a broad topic and apply that knowledge to a national security situation. Lastly, Layer 3 is designated for an analyst with a relatively longer period of time to either get up to speed on a topic they may have studied previously, or learn the basics if the topic is new to them.

CONTEXT

Analytic framework projects are relatively new deliverables for the National Academies and represent the institution’s commitment to innovation and engaging with stakeholders including the government on issues of national importance. The first analytic framework project began in 2017, then referred to as a “pilot” project. That project began with the goals of the Intelligence Community (IC), which was to empower intelligence analysts with insights from social and behavioral sciences to their assessments of real-world situations.

The analytic framework pilot project was groundbreaking for the National Academies in many ways. The project involved consistent sponsor engagement. Individuals from the IC provided input on both content and style of presentation in order to ensure the pilot analytic frameworks would best align with their work. Each of the analytic frameworks was a unique three-layer document with the sole purpose of practical “off-the-shelf” utility for the end user. In the pilot, a variety of audiences and uses were envisioned, including as educational tools for junior analysts, to guide analysts’ thinking as they make intelligence assessments, and to brief policy makers, among other purposes. The pilot also served to bridge the gap between social sciences and intelligence and between academics and intelligence analysts.

This analytic framework on measuring and analyzing public opinion largely followed the process developed for the pilot frameworks. The primary audience is intelligence analysts who are tasked with examining public opinion in countries or regions outside the United States. Myriad questions related to national security may call for examining public opinion in other countries, such as examining public sentiment toward current leadership or the extent of support for a regime change, understanding attitudes toward the United States, or exploring views about global events (for more information on the work of intelligence analysts, see National Academies of Sciences, Engineering, and Medicine, 2019).

APPROACH TO THE CHARGE AND PROCESS

The National Academies assembled a group of expert contributors with expertise in political science, measurement, data science, sociol-

ogy, mobilization, psychology, and communication. This group worked closely with representatives from the IC to identify topics and authors for the commissioned papers included in Layer 3 of this volume. At the first meeting of the project, six possible topics were identified for commissioned papers:

1. Guideposts and guardrails for the current state of public opinion research, including grounding for connecting to other data sources
2. Design alternatives—looking at alternative sources of public opinion data, such as social media
3. Getting to true attitudes (assessing whether existing/found data reflect true attitudes vs. designing collection efforts to yield data that reliably reveals true attitude)
4. Exporting methods
5. Synthesizing and making inferences from different sources, and
6. Attitudes (to intent) to behavior/taking action

Subgroups of expert contributors then refined each of topic and ultimately commissioned five papers. Authors for each paper were identified and had the opportunity to discuss the scope of each paper with the expert contributors and IC representatives. The following final list of paper topics included:

• Drawing Inferences from Public Opinion Surveys: Insights for Intelligence Reports
• Ascertaining True Attitudes in Survey Research
• Alternatives to Probability-Based Surveys Representative of the General Population for Measuring Attitudes
• Integrating Data Across Sources, and
• A Social Network Perspective on “Tipping Points” for Behavioral Change and Mass Action

Between the second and third meeting, the author of the paper on “Tipping Points” had to withdraw from the project, and the topic was ultimately omitted from the framework to ensure that the project would remain on schedule.

At the third meeting, the content of the papers included in Layer 3 was discussed in depth, and the fourth meeting focused on the content of the synthesis in Layer 2 and the graphic in Layer 1.

This analytic framework is one of three deliverables, the second deliverable is a workshop on March 8–9, 2022, and the third is a proceedings of the workshop. The workshop will provide intelligence analysts the opportunity to discuss this framework with individuals with expertise in the

design of data collection and analysis of public opinion data, and offer extended guidance on the application of the analytic framework through several example scenarios.

GUIDE TO THIS FRAMEWORK

As discussed above, the framework that follows contains three layers of information. The framework proceeds from the most distilled level of detail—a graphic presentation of the key messages in this analytic framework—to an intermediate layer (the synthesis), to the foundational white papers that were commissioned to address the sponsor’s key questions about the collection and analysis of public opinion data in depth.

REFERENCE

This page intentionally left blank.