Applications of an Analytic Framework on Using Public Opinion Data for Solving Intelligence Problems: Proceedings of a Workshop (2022)
Chapter: Appendix A: Workshop Agenda
Appendix A
Workshop Agenda
MEASUREMENT AND ANALYSIS OF PUBLIC OPINION:
AN ANALYTIC FRAMEWORK
WORKSHOP ON THE ANALYTIC FRAMEWORK
AND ITS APPLICATIONS
MARCH 8–9, 2022
The National Academy of Sciences
Lecture Room and via WebEx
2101 Constitution Avenue NW
Washington, DC 20418
AGENDA
Measuring and analyzing public opinion comes with tremendous challenges, as evidenced by recent struggles to predict election outcomes and to anticipate mass mobilizations. The recent National Academies publication Measurement and Analysis of Public Opinion: An Analytic Framework presents in-depth information from experts on how to collect public opinion data and glean insights from such data, particularly in conditions where contextual issues call for applying caveats to public opinion data.
This workshop will briefly present the Analytic Framework and demonstrate its application across a series of hypothetical scenarios that might be faced by an intelligence analyst tasked with summarizing public attitudes to inform a policy decision.
Workshop Objectives
- Provide an overview of the Analytic Framework’s components: the foundational papers, the synthesis, and the graphic depiction of the phases involved in collecting and analyzing public opinion data
- Examine key messages in the Framework and discuss what the Framework does and does not do
- Explore cultural and ethical considerations in the collection and use of public opinion data
- Engage in several exercises applying the Analytic Framework to theoretical scenarios and address intelligence analysts’ questions about its application
Tuesday, March 8, 2022 (Please note all times are Eastern Standard Time)
| 10:00 am EST | WELCOME | |
|
Samantha Chao
Associate Executive Director Extension, National Research Council Programs National Academies of Sciences, Engineering, and Medicine | ||
| 10:10 am | THE ROLE OF PUBLIC OPINION IN INTELLIGENCE ANALYSIS | |
|
Regina Faranda
Director, Office of Opinion Research U.S. Department of State | ||
| 10:20 am | WORKSHOP GOALS | |
Charles Lau, Co-Lead Expert Contributor for the Analytic Framework Director, International Survey Research Program RTI International | ||
| 10:35 am | CHALLENGES ANALYSTS FACE | |
| A representative from the Intelligence Community will describe some of the challenges intelligence analysts face to foreground the workshop in their experiences. | ||
| 10:50 am | OVERVIEW OF THE ANALYTIC FRAMEWORK AND KEY MESSAGES FOR INTELLIGENCE ANALYSTS | |
| Representative from the Intelligence Community | ||
| Charles Lau | ||
Elizabeth Zechmeister, Co-Lead Expert Contributor for the Analytic Framework Cornelius Vanderbilt Professor of Political Science and Director, LAPOP (Latin American Public Opinion Project) Vanderbilt University | ||
| 11:35 am | BREAK FOR LUNCH | |
| 1:05 pm | CULTURAL AND ETHICAL CONSIDERATIONS IN THE COLLECTION AND USE OF PUBLIC OPINION | |
Ethical Considerations Zachariah Mampilly, Expert Contributor for the Analytic Framework Marxe Endowed Chair of International Affairs, Marxe School of Public and International Affairs City University of New York | ||
Kanisha Bond, Author, Ascertaining True Attitudes in Survey Research Assistant Professor of Political Science Binghamton University (SUNY) | ||
Cultural Considerations Michele Gelfand, Expert Contributor for the Analytic Framework John H. Scully Professor in Cross-Cultural Management and Professor of Organizational Behavior, Stanford Graduate School of Business Professor of Psychology (by courtesy), School of Humanities and Sciences Stanford University | ||
| 2:20 pm | APPLYING THE ANALYTIC FRAMEWORK TO HYPOTHETICAL SCENARIOS | |
| Charles Lau | ||
| In each of the next four sessions, three of the Expert Contributors for the Analytic Framework that guided its development will describe how they would approach a hypothetical scenario from their academic perspective (30 minutes) | ||
| Following this discussion, a representative from the Intelligence Community will offer initial reactions. (10 minutes | ||
| The remainder of each session is reserved for Q&A/dialog with Intelligence Community audience members. We invite reactions, comments on where more detail is needed, and questions for the Expert Contributors and Foundational Paper authors. | ||
| 2:25 pm | SCENARIO 1: Evaluating Political Support in a MultiEthnic Country | |
| IC Analyst | Rachel is an early career IC analyst with limited background in survey research. | |
| Country | Multi-ethnic, low-income country | |
| IC analyst task | Rachel is tasked with summarizing the public’s attitudes about the prime minister within 5 business days, with a focus on variation by ethnic groups. | |
| Data sources available |
Rachel has access to:
| |
party and published on the party’s website, with minimal documentation. Individual-level (micro) data are available for (1) and (2) but not (3). | |||
William “Chip” Eveland, Expert Contributor for the Analytic Framework Professor of Communication, Professor of Political Science (by courtesy) The Ohio State University | |||
Scott E. Page, Expert Contributor for the Analytic Framework John Seely Brown Distinguished University Professor of Complexity, Social Science, and Management University of Michigan | |||
| Elizabeth Zechmeister | |||
| Representative from the Intelligence Community | |||
| 3:25 pm | WRAP UP DAY ONE | ||
| Charles Lau | |||
| 3:35 pm | ADJOURN DAY ONE | ||
Wednesday, March 9, 2022
| 10:00 am | WELCOME AND OVERVIEW OF DAY TWO | |
| Charles Lau | ||
| 10:10 am | SCENARIO 2: REACTING TO A MILITARY COUP | |
| IC Analyst | Tanya is a mid-level IC analyst with 5 years of experience working surveys | |
| Country | Middle-income country. This country is an important security partner of the US in a strategic region. The country | |
| recently experienced a military coup and there are significant security challenges in the northern part of the country. The government has declared a state of emergency, so face-to-face surveys are not feasible. | |||
| IC analyst task | Tanya is tasked with developing a plan for how the agency can understand public opinion within 1 month. | ||
| Data sources available | Tanya has access to no existing data sources. | ||
D. Sunshine Hillygus, Expert Contributor for the Analytic Framework Professor of Political Science, Professor of Public Policy (by courtesy) Duke University | |||
Courtney Kennedy, Expert Contributor for the Analytic Framework Director, Survey Research Pew Research Center | |||
Lisa Mueller, Expert Contributor for the Analytic Framework Associate Professor of Political Science Macalester College | |||
| Representative from the Intelligence Community | |||
| 11:10 am | SCENARIO 3: ANTICIPATING POLITICAL INSTABILITY | ||
| IC Analyst | Luis is an early career IC analyst with no background in survey research | ||
| Region | Politically unstable region (~15 countries) with significant security challenges | ||
| IC analyst task | Luis is tasked with describing public opinion in the region, with a focus on recommending countries where popular discontent with the regime may emerge in the next year. | |
| Data sources available | 20 surveys (5 mobile phone, 5 social media nonprobability web surveys, 10 face-to-face) from the past 5 years. The surveys vary in quality: some follow highest quality standards (micro-data available), some are low-cost/lower-quality surveys (micro-data available), some are poor quality (no micro-data available). | |
James N. Druckman, Expert Contributor for the Analytic Framework Payson S. Wild Professor of Political Science Northwestern University | ||
| Charles Lau | ||
| Representative from the Intelligence Community | ||
| 12:10 pm | SCENARIO FOUR SELECTION | |
| National Academies Staff | ||
| 12:30 pm | BREAK FOR LUNCH | |
| 2:00 pm | SCENARIO 4: AUDIENCE CHOICE | |
| Expert Contributors for the Analytic Framework | ||
| Representative from the Intelligence Community | ||
| 3:00 pm | WHAT WE HAVE LEARNED AND FUTURE DIRECTIONS | |
| Expert Contributors will reflect on the following: | ||
| ||
| 3:45 pm | WRAP-UP/CLOSING REMARKS | |
| Elizabeth Zechmeister Charles Lau Representative from the Intelligence Community | ||
| 4:15 pm | ADJOURN WORKSHOP | |
