Appendix C

Westat Key Informant Interviews Findings Report

Support to the NASEM Review of CDC COVID-19 Vaccine Safety Research and Communications

May 30, 2025

Authors:

Jennifer Edwards, Neni Osuoha, Jennifer Maizel, Rachel Neenan, Nina Page, Paul Reischmann, Beth Slotman

This study was conducted by Westat with the National Academy of Sciences. The findings and conclusions in this report are those of the authors and should not be construed to represent any official NASEM determination or policy.

CONTENTS

A. Background

B. Study Design

Sample

Recruitment and Data Collection

Interviews

Analysis

C. Findings: Cross-Cutting Themes

Infrastructure, Resources, and Processes

Coordination and Collaboration

Communications

D. Findings: Distinct Themes by Group

CDC VSD Sites

CDC-Funded Researchers: CISA

Healthcare Professionals

Vaccine Safety Experts

State and Local Public Health Officials

E. Conclusions

Study Limitations

Overarching Findings

Appendix A. Interview Guides

Appendix B. Recruitment Letters

Appendix C. Informed Consent

A. BACKGROUND

The National Academies of Sciences, Engineering and Medicine (NASEM) Health and Medicine Division tasked Westat with supporting a consensus study evaluating the Centers for Disease Control and Prevention (CDC) Immunization Safety Office (ISO) systems, methods, and processes for monitoring COVID-19 vaccine safety during the U.S. COVID-19 public health emergency (PHE). This study aimed to identify recommendations for sustaining, maintaining, and strengthening CDC ISO’s monitoring systems and communications moving forward.

Westat was responsible for conducting confidential key informant interviews with a wide array of individuals who have direct experience and knowledge of CDC ISO efforts. These interviews solicited perspectives on ISO’s processes, research, and communications about COVID-19 vaccine safety during the COVID-19 public health emergency (PHE), interagency and cross- sector collaboration, successes and challenges during the PHE, and recommendations for process improvements. This report summarizes key themes found across these interviews.

B. STUDY DESIGN

Sample

The NASEM Committee appointed to conduct the overarching study, supported by NASEM staff, identified several categories of key informants relevant to this qualitative study: CDC ISO staff, staff from other CDC offices, staff from other federal agencies, CDC-funded researchers from Vaccine Safety Datalink (VSD) sites and the Clinical Immunization Safety Assessment (CISA) program, external vaccine safety experts (VSEs), healthcare professionals (HCPs), and public health officials (PHOs). NASEM staff compiled lists of potential participants from these groups who in effect served as the population of interest for this study. Sources of information for developing these lists included the ISO Directory, Advisory Committee on Immunization Practices (ACIP) current and former members and ex-officio representatives, publications and public information, and Committee recommendations.

Key Informant Interview Groups

The Westat team developed a sample design with targets for each identified key informant group; however, recruitment proved to be challenging, precluding reaching these targets for most groups.

Table 1 provides a summary of the key informant groups invited to participate and the number of interviews across each group that were completed. In late January 2025, the Department of Health and Human Services instructed staff to pause external public health communications.

Out of an abundance of caution, ISO directed a pause on proceeding with interviews of CDC staff. As a result, this group had to be excluded. Some interviews were with multiple participants; for example, VSD site interviews typically involved two participants. Additionally, one interview with VSEs, originally planned as a focus group, had only two participants. In this instance, we made an effort to provide each respondent with an opportunity to answer each interview question separately.

In the case of non-ISO CDC staff and other federal agency staff, we received responses from one former CDC employee, one National Institutes of Health (NIH) employee, and one Food and Drug Administration (FDA) employee. These three interviews were grouped together as “other federal staff.” However, given that each participant came from distinct offices and agencies with different roles, the findings from this group essentially present three distinct perspectives. We discuss how these perspectives were integrated into our thematic analysis in the following sections.

NOTE: Several participants had multiple identities and professional experiences influencing their perspectives (e.g., physician and PH official).

Recruitment and Data Collection

Introductory Emails

To encourage participation, ISO and NASEM sent out initial emails to the participant pool to inform them of the study and advise them of Westat’s pending invitation to participate. Once this occurred, Westat sent individual emails to key informants inviting their participation; these emails included a link to a brief Qualtrics survey and information regarding interview scheduling, which are described below. Some of the emails were unique in that two individuals that worked together in the CDC-funded research group were sent a joint email to schedule a meeting together.

Westat’s outreach emails were sent on three different occasions, approximately one week apart to increase engagement. We also followed up individually with potential participants who noted interest in the survey but did not schedule an interview.

Qualtrics Survey

Westat created a recruitment survey administered via Qualtrics, a FedRAMP-compliant and secure online platform. The survey was sent to all potential participants as a tool to signal their interest in participating in the study, confirm their role or type of key informant group during the PHE, and collect their preferred method of contact and form of participation (individual interview or focus group). Additionally, the survey asked questions (some of which were optional) pertaining to a participant’s general professional information, such as their current role/area(s) of expertise, their office/agency/institution, and their familiarity with different vaccine safety systems.

In total, 69 survey responses were received. In addition, a participant who did not fill out the survey sent an email to signal interest.

Interview Scheduling

To schedule interviews, we sent out links via Calendly, an online meeting scheduling app, that provided participants the opportunity to select a time that worked for them. Of the 69 individuals who completed the Qualtrics survey, seven noted at this time that they no longer wanted to participate, and 11 were in the CDC ISO group. Of the remaining individuals who

expressed an interest in completing interviews, three canceled their interviews, one did not show up to their interview, and 17 did not respond to interview scheduling requests.

Interviews

All interviews were held via Zoom. Individual interviews lasted approximately 60 minutes and our one small-group interview lasted approximately 90 minutes. Interviews were semi-structured; Westat developed a set of core interview questions for each key informant group based on Committee questions and research aims (see Appendix A), and additional probes or follow-up questions were asked as needed and based on informants’ unique responses. The interview questions spanned an array of topics related to vaccine safety research processes, agency/institutional collaborations, resources, infrastructures, leadership, communications, and others. Some topics were cross-cutting and asked of multiple key informant groups, while other topics were only applicable to one group.

Cross-Cutting Topics

• Relationship with/perceptions of CDC ISO
• Sufficiency of vaccine safety resources and systems
• CDC vaccine safety communications, public perception, and influence
• General experience during the PHE, suggestions for improvement at ISO

Group-Specific Topics

• Researchers: Vaccine safety research priorities and independence
• Other federal agency staff: Resource allocation, coordination in federal vaccine safety efforts
• Healthcare professionals (HCPs): Use of CDC vaccine safety systems
• External vaccine safety experts (VSEs): Use of CDC data and collaborations
• Public health officials (PHOs): Vaccine safety information access/adaptation

Topics Not Able to Be Addressed

The Committee expressed interest in and developed questions on staffing and reporting structures within ISO during the PHE, ISO organizational issues, intra-CDC coordination, and the impact of ISO’s placement within CDC. These questions were unique to CDC staff and ISO. Because we were unable to interview this key informant group, these topics could not be addressed. Additionally, questions around resource sufficiency were generally answered by other key informant groups at a macro level, and they provided limited insight on post-PHE changes.

Analysis

We interpreted participant’s responses using an iterative, multistep process of coding, analysis, and synthesis of findings. Team members were divided by key informant groups for each step.

Codebook Development

As we neared the end of data collection, the team collaborated on developing a codebook for both structural and thematic analysis based on insights collected from the interviews. The codebook was designed to be broadly applicable and capture information at a high level across key informant groups. It included structural codes based on interview topics and thematic codes that captured distinct issues specific to key informant groups. Not all codes were relevant for all key informant groups, though some codes were comprehensive, capturing the perspectives of multiple key informant groups.

NVivo Analysis

We uploaded interview transcripts¹ to NVivo, a qualitative data analysis program, to conduct structural coding based on interview topics and thematic coding for specific concepts that were discussed. Transcripts were organized in NVivo using case classifications, which categorized transcripts by key informant groups.

Team members reviewed interview transcripts to code potential findings according to the codebook and were encouraged to use annotations to note any specific details for further analysis during the synthesis phase. For a consistent and reliable application of the codebook, all team

___________________

¹ Interviews were transcribed by Rev.com unless the participant requested not to be recorded. In those handful of cases, a second researcher participated in the interview to note-taking purposes.

members first coded the same transcript and discussed which codes could use further specificity or a refined definition. Once completed, the remaining transcripts were divided among the team for individual coding. Team members met on a weekly basis to ask questions and discuss preliminary findings during the coding process.

Synthesis of Findings

Once all coding was completed, the team used NVivo to run queries of the codes and annotations of interview transcripts to generate coding results (i.e., all the information coded to a specific topic and key informant group). Team members first reviewed the coding results for each key informant group, identifying reoccurring themes and shared perspectives within groups as well as outlier perspectives. Then the team met to determine whether themes emerging for a specific key informant group overlapped with themes that came up in other key informant groups. Themes that overlapped with more than one key informant group were considered cross-cutting, and themes that were distinct to one key informant group were considered group-specific.

C. FINDINGS: CROSS-CUTTING THEMES

Cross-cutting themes were categorized into three general thematic areas: (1) Infrastructure, Resources, and Processes; (2) Coordination and Collaboration; and (3) Communication.

Infrastructure, Resources, and Processes

All participants were asked to describe the sufficiency of ISO’s and the existing vaccine safety systems’ infrastructure, resources, and processes during the COVID-19 PHE. As mentioned above, the study team’s inability to interview CDC and ISO staff members meant that some facets of ISO’s infrastructure, including reporting structures, internal organization and processes, and the influence of ISO’s placement within CDC, could not be investigated. Participants across all groups shared strong positive reactions to the increase in resources and processes put in place in response to the PHE, as well as ISO leaders’ qualifications and strong response to the pressures of the PHE. However, researchers and VSEs shared that even this bolstered infrastructure was still insufficient in several ways

during the PHE, and they shared several suggestions for developing an infrastructure better prepared for future crises.

Successes

Increased resources and funding during the PHE

Multiple groups reported increased resources and funding allocated toward vaccine safety during the PHE, and these increases facilitated strong research and an effective PHE response. All federal agency staff felt they had the resources needed to ramp up vaccine safety efforts during the PHE (e.g., due to additional funding, they had the ability to hire more staff). A few CISA and VSD researchers also felt that the influx of funding and staffing facilitated an effective emergency response.

Effective structures and processes during the PHE

Multiple groups described successes tied to the structures and processes of federal vaccine safety systems. All federal agency staff agreed that established processes at CDC, the FDA, and other agencies worked well during the PHE, and that what changed the most was the frequency, intensity, and/or speed of work. These participants noted the value and importance of already having the systems, relationships, and communication mechanisms in place to enable what they saw as a successful response to the PHE. All CISA researchers noted that the existing infrastructure and relationships within CISA and between CISA and ISO enabled effective clinical and research responses. A few CISA researchers specifically mentioned the effectiveness of the co–principal investigator structure that prepared each CISA institution for the eventual retirement of its “founding principal investigator” (PI) by training their successor on the job through co-PI status. All VSD researchers thought the infrastructure and contract structure of VSD facilitated rapid and responsive research during the PHE.

Two federal agency staff also noted that new vaccine safety study protocols and tools developed during the PHE (e.g., V-safe) have had a lasting impact, as they are being adapted for other needs and can help with future PHE preparedness.

Effective leadership during the PHE

Several groups indicated that CDC ISO “did the best they could” with the vaccine safety resources they had available during the PHE. Most VSD researchers noted CDC ISO leadership (and specifically, the CDC

ISO director and VSD lead) was effective and collaborative during the PHE. Most CISA researchers also spoke highly of ISO leadership during the PHE, for example, noting their effectiveness at navigating bureaucracy, fostering a collaborative spirit, and mobilizing resources. One CISA researcher praised ISO’s pandemic-era CISA Team Leader for her outstanding capacity to expedite project launches despite bureaucratic hurdles.

Challenges

Additional resources and infrastructure were needed during the PHE

Despite the robust existing vaccine safety infrastructure and the increased resources and funding during the PHE, most groups stated that these were still insufficient, and more resources and funding were needed.

All CISA researchers and most VSEs noted that CISA’s funding, prioritization, and awareness- building resources were insufficient. VSEs noted that CISA particularly lacked the infrastructure for adverse event follow-up. Most VSEs also thought that VSD was limited by its size and funding, and half the VSEs felt V-safe had insufficient infrastructure to support adverse event follow-up.

Although they viewed VSD’s database as large, a few VSEs noted it did not have enough data to stratify adverse events by subpopulations. Additionally, one VSE described challenges gaining access to VSD’s data, which needed to be provided by the VSD team.

Recommendations

More proactive systems supported and sustained by a robust infrastructure

All external VSEs and some CISA researchers called for a more proactive and robust vaccine safety infrastructure, including more funding, resources, and adverse event follow-up efforts. All VSEs advocated for increased and sustained funding and resources put toward vaccine safety systems and research. A few CISA researchers suggested that the infrastructure developed during the PHE needs to be used more proactively, rather than reactively, to prepare for potential future pandemics; notably, by maintaining communication channels between researchers and frontline providers that were developed during the PHE and leveraging the large groups of researchers and experts brought together for clinical consultations during the PHE to anticipate and study future vaccine safety questions.

Coordination and Collaboration

Interview protocols for VSD and CISA researchers and other VSEs included questions exploring how successful or challenging it was to work with CDC during the COVID-19 PHE. Because of this, findings on coordination and collaboration primarily reflect the perspectives of these research-focused groups and cannot be generalized to other key informant groups or entities. Overall, interviewees shared mostly positive experiences collaborating with CDC on vaccine safety research. However, some researchers encountered challenges and delays in CDC approval and publishing processes, which also posed potential risks to their research independence. Notably, VSD and CISA researchers shared conflicting perspectives on whether their level of input into research priorities was sufficient for effective collaboration. Based on interviewees’ feedback, recommendations to improve coordination and collaboration include reducing barriers to CDC’s approval and publishing of research, and enhanced coordination between CDC and other federal agencies.

Successes

Effective partnerships between CDC and other researchers

Interviewees across several groups described positive experiences working with CDC staff. Nearly all VSD researchers and federal agency staff characterized their relationship with CDC as a collaborative and respectful partnership. Some VSEs echoed this sentiment, noting that ISO staff were responsive to their questions and keen to share information. Most CISA researchers felt that the ISO team demonstrated strong leadership and communications skills during the PHE, developed positive and productive relationships with contracted researchers, and were effective in establishing the infrastructure and administrative support needed to sustain CISA research and clinical consideration activities.

Challenges

Delays and barriers in research publishing and approval processes

Most VSD and CISA researchers, and a few VSEs, described challenges with CDC’s research clearance and publishing processes. They noted that the review process could be slow and cumbersome, sometimes delaying the distribution of time-sensitive research findings. A few VSD researchers thought there were too many separate rounds of review, or that the clearance process included reviewers who did not have appropriate expertise

to effectively and efficiently review manuscripts. One VSE described an experience in which concerns regarding potential Freedom of Information Act (FOIA) requests significantly delayed their research being published.

While most interviewees’ concerns pertained to clearance and review processes, some researchers also described barriers related to award and approval processes. Most CISA researchers agreed that the yearly timeline for proposal awards is too slow and too rigid, preventing researchers from studying quick-turnaround events such as seasonal vaccine administrations.

Potential risks to research independence during the publishing process

A few VSD researchers described scenarios during the PHE in which CDC reviewers suggested changes that the VSD researchers did not agree with during the clearance process. In two of these cases, researchers perceived that the issue stemmed from reviewers lacking adequate expertise to understand and interpret the study design and findings. One of these researchers noted they were able to provide input to ensure CDC communications about the study accurately reflected its findings, while another researcher explained that they were unsure how to get their publication cleared without having to accept the edits that they disagreed with.

Another VSD researcher shared an instance in which CDC reviewers outside of ISO wanted to highlight and overstate a potential vaccine benefit identified in their study. The researchers who conducted the study believed the finding of this potential vaccine benefit was untrue and likely resulting from bias, and they strongly disagreed with any mention of potential vaccine benefits when the study was designed to assess vaccine safety risks. The researchers involved had to repeatedly emphasize and explain this distinction to CDC staff throughout the clearance process to ensure communications about their study stayed accurate. One VSE also noted challenges and delays tied to cooperative agreements requiring CDC staff to coauthor publications with them, since this requires their research to be vetted by government clearance processes and reduces scientific independence. They noted this contrasts with NIH, which does not require NIH staff to coauthor publications.

Mixed Perspectives

Level of involvement in setting research priorities

VSD and CISA researchers held contrasting views regarding their involvement throughout the research process, particularly in setting research

priorities. All VSD researchers noted they were able to provide input on research priorities and propose vaccine safety studies. They described the research priority setting as a collaborative and iterative process with CDC rather than a hierarchy in which CDC would have given rigid task orders to researchers. A few CISA researchers shared this perspective and noted that they were involved to some extent in priority setting.

However, most CISA researchers wished that they could have been included earlier in the priority- setting process and desired greater flexibility to collaborate with CDC. VSEs were split between these two perspectives; a few agreed that they were sufficiently involved in the process, and a few wished they could have been more involved.

Recommendations

Improve collaboration and coordination of vaccine safety information among federal agencies and more broadly

While CDC’s collaboration with researchers was largely characterized as effective during the PHE, interviewees from various groups called for increased coordination between CDC and other entities. All federal agency staff and some VSD researchers and external VSEs agreed there is a need for CDC to better coordinate and streamline vaccine safety research and communications with other federal agencies, including the FDA, NIH, the Department of Defense (DoD), Department of Veterans Affairs, and Indian Health Service. A few interviewees mentioned that federal agencies appeared siloed and have limited communication despite conducting work related to that of other agencies. One VSE described how federal agencies collaborated more during the COVID-19 PHE and other PHEs; however, these enhanced collaborations were temporary, and they emphasized a need to sustain this level of collaboration over time.

Additionally, one PHO emphasized the need for CDC and federal agencies such as the Administration for Strategic Preparedness and Response to share information and updates with states immediately and specifically recommended increasing the frequency of multistate conference calls during PHEs.

Streamline processes to speed up study development and publication clearance

Most CISA researchers, some VSD researchers, and a few VSEs agreed that barriers and delays in study development and publication clearance processes need to be addressed. VSD researchers shared possible

approaches for speeding up or streamlining the process, such as having multiple rounds of review happen simultaneously, only including reviewers with appropriate expertise, and better defining or limiting the scope of CDC’s review process. One VSE wished CDC’s cooperative agreements for publications could be eliminated. CISA researchers suggested that ISO begin consulting researchers earlier in the development of its public health priorities in order to better align study teams with reviewers; that ISO offer flexibility around proposal timing in any way possible; and that a “careful [but] not overly conservative” response to political pressures, including susceptibility to FOIA requests, would reduce operational barriers to publication.

Communications

Interview protocols for all key informant groups investigated participants’ perceptions of CDC and ISO’s public-facing communication strategies, public vaccine safety awareness, and public perceptions of ISO. Broadly, participants across groups praised the accuracy, credibility, and timeliness of ISO’s public communications, with PHOs and HCPs yielding additional insights regarding their own efforts to make ISO’s vaccine safety information more accessible. However, PHOs and HCPs also described a multitude of barriers posed by widespread misinformation and limited public awareness of ISO’s vaccine safety systems. CDC-funded researchers and HCPs suggested several tactical and structural changes to improve the accessibility of and public trust in CDC’s vaccine safety communications.

Successes

Transparency, timeliness, and reliability of vaccine safety information

Participants across multiple key informant groups largely perceived CDC’s vaccine safety information as transparent, timely, and reliable. A few PHOs, VSEs, and HCPs noted the value of ACIP in relation to the quality of CDC’s vaccine safety information, with a few VSEs and HCPs specifically describing ACIP as being transparent and responsive. All PHOs commended the speed at which CDC produced and disseminated vaccine safety information, with a few noting that CDC was successful despite the challenging circumstances of the PHE. One CISA researcher also noted that high-impact research findings were disseminated very rapidly.

Most PHOs also touched on the reliability of CDC’s vaccine safety information. They described CDC as a trusted resource throughout the pandemic and noted that they encouraged the public to trust CDC’s guidance as well.

Usage of accessible and tailored communication strategies

PHOs and HCPs both reported using accessible and tailored communication strategies to ensure underserved groups received vaccine safety information. For example, all PHOs explained that vaccine safety communications were translated into different languages and adapted for different reading levels. A few PHOs and HCPs also described using trusted messengers or community representatives to disseminate information to underserved populations.

Accurate representation of CDC-funded research in CDC’s public-facing communications

All CISA and most VSD researchers agreed that their research was accurately represented and described in CDC’s public-facing communications. One VSD researcher shared an example in which CDC staff consulted the researchers before presenting study findings to ensure they were not misinterpreting or misstating study results. Federal agency staff also reported no significant disagreements with CDC on the interpretation or reporting of vaccine safety signals.

Challenges

Limited knowledge of ACIP and vaccine safety systems

Most VSEs and HCPs agreed that the public has limited awareness or understanding of ACIP and ISO’s vaccine safety systems, and this posed substantial barriers to effective vaccine safety communication. Most of the HCPs who were interviewed were all affiliated with ACIP in some capacity and noted that their colleagues, administrators, and patients generally had little or no awareness of ACIP and vaccine safety systems; therefore, they served as their workplaces’ primary source of information during the PHE. One HCP described how the substantial lack of awareness in their workplace inhibited patient and clinician use of the vaccine safety systems. A PHO corroborated that they frequently needed to include an explanation about the Vaccine Adverse Event Reporting System (VAERS) in communications materials.

Misinformation and responding to public concerns

All HCPs and most PHOs cited misinformation as a pervasive barrier to effective vaccine safety communication, and they reported that CDC did not thoroughly respond to or refute instances of misinformation. HCPs

unanimously stated that CDC’s spokespeople and communications staff did not comprehensively address public concerns arising from misinformation, noting that CDC communications staff failed to address legitimate questions or concerns from the public possibly because the questions were regarded as unscientific or silly. Furthermore, all HCPs agreed that CDC’s ineffective response to misinformation hampered their own vaccine safety communication efforts. Almost all PHOs reported that misinformation posed major communication challenges; however, several said they developed processes to address it during the PHE. These processes are described in more detail in Section D along with other successes reported by PHOs.

Recommendations

Improve accessibility and clarity of public-facing vaccine safety communications

While many interviewees commended the transparency, timeliness, and reliability of CDC ISO’s vaccine safety information, some still felt there was room for improvement. All CISA and most VSD researchers suggested making communications more accessible to the public, for example, by translating research findings into plain language. Several PHOs agreed and specifically suggested that CDC communication materials be made accessible at the beginning of or early in a PHE to ensure quick dissemination to the public, rather than requiring additional time and steps to adapt language later.

All VSEs and several VSD and CISA researchers emphasized the importance of CDC effectively balancing the goals of informing the public of vaccine safety issues while still promoting public trust in vaccines. A few CISA researchers and a few VSEs felt that CDC should more clearly communicate the level of evidence or support for different vaccine safety signals. For example, they thought it would be useful for CDC to differentiate between safety signals identified in unverified VAERS reports and safety signals identified in rigorous studies, since some safety signals with little supporting evidence could be taken out of context by the public and used to draw inaccurate or incomplete conclusions.

Strengthen vaccine safety communications capacity

Interviewees from multiple groups provided suggestions for bolstering the capacity of CDC’s vaccine safety communications. All HCPs advocated

for more resources and funding toward communications. Most CISA researchers desired to contribute to public-facing communications about their research findings to ensure accurate presentation of data; they recommended that CDC specifically allocate funding with the purpose of involving researchers in communications development. Most HCPs strongly recommended having credentialed health professionals with neutral political stances serve as CDC’s spokespeople for vaccine safety communications to increase public trust.

D. FINDINGS: DISTINCT THEMES BY GROUP

CDC VSD Sites

Vaccine Safety Datalink (VSD) is a collaborative project between CDC ISO and healthcare organizations across the United States. VSD monitors the safety of vaccines and conducts studies about rare and serious adverse events following immunization. The group of VSD researchers interviewed were leaders from seven of the 13 sites who each had extensive experience planning and executing vaccine safety studies. The group included multiple VSD site PIs or co-PIs, many of whom were physicians, epidemiologists, or biostatisticians. These researchers have published numerous studies and made recommendations to ACIP. All interviewees were a part of VSD during and after the COVID-19 PHE.

Successes

VSD researchers agreed that the infrastructure of VSD was highly effective in supporting their work during the COVID-19 PHE.

VSD infrastructure facilitated rapid and responsive vaccine safety research

As noted in theme “Effective structures and processes during the PHE” in Section C above, VSD researchers reported that the infrastructure and contract structure of VSD enabled them to rapidly respond to vaccine safety research needs during the COVID-19 PHE. While this perspective was not exclusive to VSD researchers, it is worth noting that this perspective was unanimous among interviewed VSD researchers, several of whom shared additional details on the topic. One researcher mentioned specifically that the umbrella contract structure of VSD eliminated some of the bureaucratic hurdles that can hinder collaboration common in other types of contracts. Another researcher noted that their team was able to plan for COVID-19 vaccine safety research well in advance; this enabled them to begin studies as soon as vaccines were authorized. They added

that key research elements, such as necessary data use agreements, project management processes, and communications streams, were all already in place.

Challenges

VSD researchers described confusion regarding communication responsibilities as being a challenge during the PHE.

Unclear who was responsible for communicating vaccine safety findings

Some VSD researchers felt they did not receive clarity regarding who was responsible for sharing findings from VSD studies with the public. Specifically, a few researchers described feeling unsure as to who was ultimately responsible for communicating vaccine safety findings—CDC ISO, CDC staff from other departments, or the VSD researchers themselves. One or two interviewees noted that VSD researchers do not typically have expertise in research communications and would require additional support if they were to play a larger role in sharing study findings.

Recommendations

VSD researchers provided several suggestions for improving vaccine safety communications.

Better support and coordination for vaccine safety communications

Although multiple key informant groups called for improved vaccine safety communications, VSD researchers shared several distinct recommendations for these communications. A few VSD researchers felt it would be valuable to adapt their study findings for communication formats besides academic journals, such as news articles and social media posts. One researcher also felt it would be valuable to distribute educational flyers or videos about their studies with participating health systems that contribute data to VSD. Several researchers who provided these recommendations noted that they would need additional support and more communications expertise to adapt study findings for communications that target various audiences. Lastly, a few researchers recommended that CDC regularly update the VSD website to ensure information about VSD and its research processes remain current and are easy to find.

CDC-Funded Researchers: CISA

The group of CISA researchers interviewed was made up of five leaders from five of eight medical research centers involved in CISA nationwide, with decades of collective experience planning and executing CISA research and clinical consultation tasks. This group included multiple CISA site PIs, medical professionals with extensive backgrounds in several specializations, and authors and leads of numerous vaccine safety studies and clinical trials. All interviewees had experience working on CISA efforts before, during, and after the PHE.

Successes

CISA researchers spoke at length about the effectiveness of ISO’s leadership, infrastructure, and resource allocation during the PHE. These successes are discussed in “Successes” in Section C above.

Challenges

CISA researchers had unique concerns about their limited ability to contribute to public health communications, the level of public awareness of CISA’s activities and publications, and the recent staff turnover at ISO.

Division between researchers and public-health communications may limit impact of research

Most CISA researchers described how the division between research and public-health communications functions within ISO limited the potential impact of their research and expertise on public awareness and vaccine safety. Several pointed out that CISA researchers, with their unique combination of clinical experience and specialized vaccine safety research knowledge, could provide valuable consultation on communications materials and talking points for doctors and PHOs, but the current structure of CISA restricts them from contributing to many public- health communications efforts.

Concerns about readiness after turnover at ISO

Most CISA researchers expressed concerns about the long-term readiness and capacity of ISO’s staff, particularly after a period of significant turnover following the PHE. One CISA researcher also pointed out that ISO’s interdepartment rotation program reduces each department’s

efficiency by temporarily depriving teams of the “institutional awareness” of experienced staff.

Although they generally spoke highly of the communication, leadership, and expertise of the ISO leaders in place during the PHE, these researchers spoke at length about the unique skill set needed for ISO to function well. They mentioned strong leadership, communication skills, flexibility, and the ability to navigate bureaucracy as vital characteristics for ISO staff. To these points, researchers described a great loss of institutional knowledge at ISO due to turnover that occurred between the PHE and the present day, which has reportedly impacted ISO’s ability to navigate the overlapping bureaucratic structures of CDC, state public health systems, and academia.

Rigidity of ISO processes

Most CISA researchers noted there were challenges with rigidity and inefficiency in ISO’s organizational structure, highlighting the restrictive timelines of ISO research grant applications and a restrictive adherence to CDC and ACIP public health priorities during study selection. A few researchers specifically expressed concerns about the current ISO management and communications teams being restricted by the priorities of CDC leadership, positing that this hampers ISO’s capacity to communicate effectively rather than “toeing the line” and worrying about “saying things the right way.” Researchers described how these tendencies can limit research teams’ ability to work flexibly and independently, though CISA researchers did not share VSD researchers’ concerns about direct ISO influence on research products and spoke highly of ISO’s accurate portrayals of their research in communication products.

A few CISA researchers also mentioned they had greater clinical consultation capacity than was utilized during the PHE, and that ISO restricted the number of cases they were referred to avoid overwhelming CISA staff. Notably, this description stands in contrast to the perceptions shared by most VSEs, who perceived that CISA lacked the infrastructure to keep up with incoming adverse event reports during the PHE.

Awareness of CISA as a body is low, hampering impact of research results

Although they spoke highly of the short-term funding allocated during the pandemic (discussed in Section C), several CISA researchers expressed concern about the resources allocated to driving awareness and scaling of CISA research on an ongoing basis. Regarding ISO funding and

programmatic support, they described the treatment of CISA as a “little sibling” of VAERS and VSD, calling for more egalitarian support for CISA and better integration between the three systems. They described how limited public awareness of CISA reduces the number of HCPs and members of the public who seek out its research products and limits its potential to improve vaccine confidence through research dissemination.

Recommendations

CISA researchers included some basic suggestions for improvement in their descriptions of the challenges described above. They also offered recommendations for better integrating research teams into ISO’s research priority-setting process.

Desire for researchers to be more involved in study or grant priority setting

Almost all CISA researchers described how, in earlier iterations of the CISA program years before the PHE, ISO’s process for designating research priorities was more researcher-driven. However, in more recent years, they mentioned that priorities were mostly discussed internally within CDC, with the process being a “black box” from the researcher’s perspective. Several researchers mentioned that they did approve of CDC’s research priorities and that they were consulted to some extent in recent years, but others asked to be more included. They proposed a start-of-year meeting between ISO and its funded researchers to gauge priorities among the research community, and in one case called passionately for academics to be more involved in the development of project best practices and funding priorities, referring to the strict imposition of a “top-down CDC way” in recent years. As mentioned in Section C, VSD researchers described a much more positive account of their own involvement in priority setting.

Healthcare Professionals

Interviewed HCPs (n=4) worked in a variety of settings, including clinical, academic, and nonprofit institutions. It is important to note that all interviewed HCPs were affiliated with ACIP, and not all were practicing clinicians during the COVID-19 PHE; as such, they sometimes referred to the experiences or perspectives of colleagues who were practicing and had direct interactions with patients during that time. Interviews with HCPs focused on understanding their perceptions of and experiences with the

four vaccine safety systems, as well as professional- and public-facing vaccine safety communications.

Successes

HCPs described successes in using targeted approaches to communicate vaccine safety information to underserved patient populations.

Trusted community representatives helped with communicating vaccine safety information to patients from underserved groups

Half the HCPs noted that using trusted community messengers was an effective approach for communicating vaccine safety information to patients from underserved groups. For example, they described having female and Black clinicians share information with pregnant and Black patients, respectively. Half the HCPs also mentioned that effectively communicating with underserved patients was especially important since they had less access to information and more often expressed vaccine-related concerns.

Challenges

As mentioned previously in the discussion of the “Communications” theme in Section C, most challenges reported by HCPs were tied to vaccine safety communications.

Clinicians, health administrators, and patients had limited awareness of ACIP and vaccine safety systems

Most HCPs reported that their colleagues, institution’s leaders, and patients had limited or no awareness of ACIP or the vaccine safety systems during the COVID-19 PHE. Some HCPs indicated that if they had not mentioned ACIP or the vaccine safety systems at their workplaces, their colleagues, leaders, and patients would likely not have been informed about these topics. However, they noted that some patients may have seen related information on social media.

Lack of refutations to misinformation hindered the effectiveness of communications

All HCPs expressed that CDC did not effectively refute misinformation about vaccine safety, nor did they thoroughly address public concerns regarding vaccines during the COVID-19 PHE. They described CDC’s communication approaches as rigid and using technical jargon, which

ultimately failed to address questions from the public that were perceived as silly or unscientific.

Recommendations

To overcome the challenges described above, HCPs recommended that CDC enhance the capacity and credibility of their communication efforts.

Increase funding for vaccine safety systems education and communications

All HCPs called for significant increases in the amount of funding allocated to CDC’s vaccine safety communications, as well as more public-facing vaccine safety education initiatives and enhancements to CDC’s communication infrastructure and processes.

Utilize a credentialed, unbiased professional as vaccine safety spokesperson

Most HCPs recommended that CDC appoint a credentialed health professional who is unbiased and not politically driven to be a vaccine safety spokesperson to communicate information externally. HCPs indicated that designating a spokesperson who has these qualities is crucial for building public trust in vaccines and improving the coordination of vaccine safety information.

Vaccine Safety Experts

VSEs (n=6) were primarily external researchers focused on vaccine safety. Interviews with VSEs aimed to understand their experiences using the four vaccine safety systems, perceptions regarding data transparency, and experiences with publishing vaccine safety research.

Successes

Most successes reported by VSEs pertained to ACIP’s transparency during public-facing meetings and specific components or attributes of the four vaccine safety systems.

Data transparency was valuable in ACIP meetings

Half the VSEs described the value of ACIP openly sharing vaccine safety information during their public-facing meetings throughout the COVID-19

PHE. A few VSEs mentioned they felt it was ACIP’s “duty” or “role” to be transparent with the public. One VSE mentioned that ACIP was also responsive to safety concerns brought forward by clinicians during their meetings.

Vaccine safety systems’ benefits: CISA’s case consults, VAERS’ rapid surveillance methods, V-safe’s text capabilities, VSD’s scientific rigor

Most VSEs reported that CISA’s case consultations were valuable. Specifically, they noted that CISA’s clinician consultants efficiently provided individual-level vaccine safety recommendations, and they also liked CISA’s structure, which is composed of clinicians proactively working together to address adverse vaccine events. Most VSEs mentioned that VSD’s level of scientific rigor, active surveillance methods, and wide scope and reach were highly valuable for research. Half the VSEs reported that VAERS’ rapid surveillance methods were efficient for quick signal detection and data collection and analysis. Additionally, half the VSEs liked V-safe’s text message capabilities; one VSE described how this modality of communication conveyed to the public that researchers were interested in addressing and responding to their adverse events.

Challenges

Challenges reported by VSEs pertained to pitfalls or limitations of the four vaccine safety systems, as well as barriers associated with prioritizing and publishing vaccine safety research.

Vaccine safety systems’ limitations: VAERS data can be misused, CISA’s case consults were underutilized, V-safe has limited infrastructure, VSD has limited data

All VSEs reported that VAERS data were misused or misinterpreted and had limited scientific rigor due to its passive surveillance methods. They discussed specific limitations of VAERS, including that its data was under-reported, biased, and incomplete due to self-reporting, and that it lacked clear case definitions. Most VSEs perceived that CISA is significantly underfunded, underused, and has insufficient infrastructure to follow up on all reported adverse events. Half the VSEs also mentioned that V-safe lacked sufficient infrastructure to support adverse event follow-up, similar to CISA. They noted that the initial intention of V-safe’s text messaging service was to respond to all individuals who reported adverse events; however, they felt that V-safe’s staff quickly became overwhelmed by the number of messages and were unable to respond to them. Finally, most

VSEs noted limitations of VSD’s structure; these included observations that VSD’s database did not always have sufficient data to stratify adverse events by subpopulations (e.g., different age groups) and VSD’s data was more difficult to access than VAERS.

Cooperative agreements and prioritization of leaders’ research interests

Some VSEs described challenges associated with prioritizing and publishing or sharing vaccine safety research. Two VSEs reported each of the following challenges: delays in publishing research due to CDC cooperative agreements that required a CDC employee to coauthor manuscripts; CDC ISO’s reluctance to pursue or publish research that does not align with leaders’ interests or may be deemed as controversial; and insufficient collaboration between CDC ISO, other federal agencies, and external vaccine researchers and institutions.

Recommendations

Overarching recommendations from VSEs pertained to increasing the structural capacity, funding, and resources put toward vaccine safety systems, both during and outside of PHE contexts.

More funding for vaccine safety system infrastructure, particularly supporting follow-up to reported adverse events

All VSEs called for more sustained funding and resources put toward vaccine safety systems. They indicated that the challenges encountered during the COVID-19 PHE would repeat in the future if more funding and resources are not allocated to these systems. Most VSEs specifically recommended that additional support be put toward reported adverse event follow-up efforts to address the challenges described above.

Sustained interagency and external collaborations

Half the VSEs called for sustained collaborations among federal agencies as well as with external institutions (e.g., academic institutions and international organizations). They noted that CDC ISO is only one “player” in the vaccine safety space, and that CDC ISO would benefit from stronger partnerships with the FDA, NIH, DoD, and the Veterans Health Administration.

Furthermore, some VSEs recommended that CDC ISO more proactively work with and listen to vaccine safety researchers employed at academic institutions and other organizations (e.g., the WHO).

State and Local Public Health Officials

State and local PHOs (n=5) shared successes, challenges, and recommendations regarding vaccine safety information sources and described communication strategies for disseminating vaccine safety information to the public. Due to the small sample size, we did not differentiate findings by state and local perspectives.

Successes

PHOs found value in collaborative partnerships and continued to disseminate accurate and reliable sources of information to the public despite misinformation challenges.

Positive, collaborative relationship between state and local health departments and other partners

Sustaining partnerships was integral for mobilizing and disseminating a consistent message within states regarding COVID-19 and vaccine safety. Most PHOs described having positive, collaborative relationships within their respective state and local health departments and with hospital systems, healthcare providers, elected officials, law enforcement and emergency services, and community partners. For example, one PHO described their health department as a “regional hub” for supporting various entities to effectively guide the public through the PHE.

Development of communication strategies to address the public’s vaccine safety concerns and mitigate the spread of misinformation

Most PHOs recognized there were sources of misinformation that could influence vaccine safety communication efforts. Some PHOs discussed the importance of developing processes for acknowledging and addressing public concerns, even if they were unfounded or “silly,” and they described a few strategies they used to get ahead of the spread of misinformation. One PHO’s health department reported success using a vaccine safety hotline to address the public’s questions and concerns and hiding any posts or comments that were spreading misinformation on the health department’s social media platforms. Another PHO encouraged frontline staff at vaccine clinics to leverage one-on-one conversations with patients as an opportunity to connect and encourage vaccine trust.

Challenges

PHOs experienced challenges navigating misinformation and constant changes to public health and vaccine safety guidance.

Misinformation was a persistent public-health and vaccine-safety communication issue

Many PHOs described misinformation as a primary contributing factor to vaccine hesitancy and distrust. Some PHOs described how conflicting public health guidance provided early in the COVID-19 PHE fueled public confusion and concern; for example, rapidly changing guidance regarding masks was viewed as unclear. One PHO attributed vaccine distrust and confusion to publicizing the vaccine manufacturing process without providing adequate context for the years of research and testing that contributed to the vaccine’s development. Nearly all PHOs mentioned that unverified VAERS reports often fueled public misinformation.

CDC’s public health and vaccine safety information was unclear and delayed at times

All PHOs understood that COVID-19 and vaccine safety information rapidly changed as research evolved during the PHE; however, many PHOs struggled to identify guidance updates. Some PHOs described CDC’s website as a “tangled web” that can be challenging to navigate even for the most experienced individuals. Another PHO expressed frustration with learning information at the same time as the public and feeling unprepared to address the public’s questions without adequate notice of updated guidance.

Recommendations

PHOs suggested improvements on how to provide clear and complete public health and vaccine safety information and minimize the potential for misinformation to erode public trust in the process.

Acknowledge guidance will change as new data emerges

Although all PHOs found it challenging to keep up with rapidly changing guidance, they appreciated CDC’s transparency in informing the public as new data emerged. A few PHOs encouraged CDC to consider acknowledging unknown circumstances at the onset of PHEs to minimize confusion or skepticism when guidance changes.

Clearly identify changes to vaccine safety guidance

Most PHOs recommended that when changing prior or existing guidance, CDC should clearly highlight the changes to make it easier for people to identify new or updated information. They also recommended that communications be dated and time-stamped and include a summary of what

changes were made and why so that PHOs can efficiently identify and then disseminate updated guidance to the public.

Develop a provider-facing and public-facing system for VAERS reports

Some PHOs recommended CDC develop a provider-facing and public-facing system for VAERS reports. PHOs appreciated having VAERS reports as a reference when talking to patients who experienced an adverse event, but PHOs encouraged CDC to consider a public-facing system that contextualizes the adverse events to help the public better understand vaccine risk and the role of reporting systems as a vaccine safety tool.

E. CONCLUSIONS

This qualitative study assessed key informants’ views regarding CDC ISO’s systems, methods, and processes pertaining to COVID-19 vaccine safety during the U.S. COVID-19 PHE. Key informants, who included federal agency staff, CDC-funded researchers, HCPs, outside VSEs, and state and local PHOs, shared perspectives reflecting an array of professional disciplines integral to vaccine safety and communications during the PHE.

Study Limitations

Although this study yielded important insights regarding CDC ISO and vaccine safety efforts, it had a low sample size, with n≤7 participants from each key informant group. The perspectives expressed by participants in this study are not necessarily generalizable to other researchers, HCPs, VSEs, or PHOs. Westat’s pool of participants was derived essentially from a convenience sample, as opposed to a randomly drawn sample from the full population of each key informant group. While this approach was appropriate for the Committee’s aims and the available time and resources for conducting the study, there are unavoidable selection and sampling biases inherent in this approach. For example, the population of HCPs during the PHE is much greater than Westat or NASEM could identify, sample, and conduct outreach to for this study. However, for VSD sites, we were able to start with a complete population and reach a much larger proportion of sites.

Numerous study participants were connected with ACIP in some capacity and/or shared perspectives from roles other than those tied to their key informant group (or those that occurred outside the COVID-19

PHE). Thus, some participants’ responses were not necessarily representative of non-ACIP-affiliated professionals and/or could have been affected by recall bias. Some participants referenced their colleagues’ or patients’ perspectives in response to certain questions for which they had limited knowledge.

Most importantly, Westat was unable to interview individuals who were employed at CDC during the COVID-19 PHE, per direction from CDC ISO leadership. As such, perspectives from this critical key informant group were missing from our analysis. Additionally, only a very limited pool of other federal staff working with ISO were able to be identified.

Other limitations of this study were tied to interview durations, participants’ unwillingness or inability to answer certain questions, and the potential for additional biases in the small-group interviews. Regarding interview durations, Westat was not always able to ask all questions on the interview guides if participants provided extensive details or wanted to focus on specific topics. Additionally, some questions were deemed not applicable to certain participants due to their lack of personal experience or perspectives regarding those topics. Participants may have also limited their responses to certain questions due to concerns regarding political sensitivities associated with the topic of vaccine safety. In the handful of small-group interviews consisting of two participants, participants’ responses could have been influenced by peer responses and social desirability biases.

Finally, this study primarily focused on processes during the PHE. As a result, the findings may not reflect current CDC policies and procedures.

Overarching Findings

Despite the limitations discussed above, the results of this study can be used to provide insight into the strengths and limitations of the systems and processes in place during the PHE and the array of challenges faced by these key informant groups, as well as to identify potential improvements for further examination and areas for additional research and evaluation.

Successes and Challenges

Overall, key informants reported that CDC ISO’s COVID-19 vaccine safety efforts were strong; however, they noted substantial challenges and presented recommendations to enhance CDC ISO’s efforts moving forward. With regards to infrastructure, resources, and processes, key informants from most groups commended the existing structures of the four vaccine safety systems (CISA, VAERS, VSD, and V-safe), enhanced processes

during the PHE, and increased resources and funding during the PHE. They emphasized that these fundamental operational components, combined with CDC ISO’s effective leadership, enabled rapid COVID-19 vaccine safety research and emergency response efforts. However, they conveyed that vaccine safety structures, resources, and processes were still insufficient, and CDC ISO’s COVID-19 response would have been stronger with additional funding and staff, and a more robust infrastructure.

Key informants’ perspectives regarding CDC ISO’s coordination and collaboration with other agencies and researchers during the COVID-19 PHE were largely positive. CDC-funded researchers and VSEs shared contrasting views regarding their level of involvement and ability to provide input into CDC ISO’s vaccine safety research, especially priority setting. Some CDC- funded researchers and VSEs expressed concerns with CDC’s research clearance and publishing procedures, which created delays and reduced their scientific independence.

Regarding communications, key informants from multiple groups perceived CDC’s and ACIP’s vaccine safety information as transparent, timely, and trustworthy. CDC-funded researchers largely felt that their findings were accurately represented in CDC’s public-facing communications. HCPs and PHOs described successes in adapting CDC’s vaccine safety communications for various audiences. However, key informants across multiple groups discussed pervasive challenges with COVID-19 and vaccine-related misinformation, and they indicated that CDC did not thoroughly address public concerns or debunk misinformation. They also noted that the public, other HCPs, and health administrators have limited or no understanding of ACIP and the four vaccine safety systems, which posed substantial barriers to effective communication during the PHE.

Recommendations

As discussed throughout this report, key informants presented recommendations to address vaccine safety process and communication challenges encountered during the COVID-19 PHE. Additionally, their recommendations aimed to strengthen future PHE preparedness and response efforts.

Key informants noted that it is of the utmost importance to continue building CDC ISO’s vaccine safety systems and infrastructure. They recommended increasing and sustaining the amount of funding and resources allocated toward the four vaccine safety systems over the long term. Key informants also called for a more proactive approach to monitoring and addressing potential vaccine safety issues. Additionally, they recommended that CDC ISO increase and sustain their collaborations with other federal agencies and external researchers/entities outside of PHE contexts to ensure proactiveness and preparedness. Several

CDC-funded researchers and VSEs described a need to alleviate barriers with CDC’s research clearance and publication processes to reduce delays and promote scientific independence.

To strengthen public awareness of and trust in vaccine safety efforts, key informants recommended that more funding should be allocated to CDC’s vaccine safety communications, and that CDC’s vaccine safety information should be shared externally by an unbiased spokesperson who is credentialed in the health sciences. Lastly, key informants recommended that CDC create more accessible vaccine safety information materials and more clearly communicate the level of evidence for different safety signals to strengthen the public’s trust in vaccines.

Areas Warranting Further Examination

The perspectives and experiences of these key informants suggest additional examination of issues and processes may be beneficial to CDC ISO’s long-term improvement efforts. Regarding the infrastructure, systems, and resources for vaccine safety research and communication, a full understanding of the scope and impact of any resource increases occurring during the PHE is needed and warrants further study. Additionally, further study is necessary to develop more detailed recommendations regarding the appropriate amount of resources, staff, and funding that should be allocated to specific vaccine safety systems, processes, and communications.

Information on program specific resource changes and the inclusion of agency staff perspectives would both benefit that examination.

Interviews with VSD and CISA researchers also suggest that structural and process differences in these two systems may be affecting their views of the effectiveness of collaboration. As these systems have different objectives, their processes are necessarily different; however, the identification of system-specific improvements requires more in-depth study. Similarly, there may be value in deeper examination of methods for ensuring engagement and collaboration with external VSEs and researchers. The small pool of external VSEs included in this study offered insights on several vaccine safety systems that other key informants largely did not raise.

Recommendations for process improvements across all four vaccine safety systems requires additional input and further study.

Moreover, further examination of publication clearance challenges and timeliness would be beneficial to more clearly identify and evaluate process improvements. Case studies examining the factors influencing the publication process for a sample of papers could help to facilitate the development of specific recommendations.

Lastly, a more targeted study of communication specialists within CDC, other agencies, and at the state and local level (including Tribal Health Service Areas) would offer valuable insights regarding vaccine safety communication challenges and improvements. This study largely included researchers and HCPs. To further investigate effective methods for tailoring vaccine safety communications for various audiences (researchers, HCPs, and the public) and to ensure transparency and understanding of research on safety signals, insights from communication specialists in this area are needed.

APPENDIX A. INTERVIEW GUIDES

Introductory Script for All Interviews

Thanks very much for agreeing to speak with us. My name is (if applicable: and I’m here with.) We’re from Westat, a health and social policy research organization. In early 2024, the Centers for Disease Control and Prevention Immunization Safety Office (CDC ISO) asked the National Academies of Sciences, Engineering, and Medicine (NASEM) to evaluate CDC ISO’s systems, methods, and processes for monitoring COVID-19 vaccine safety during the COVID-19 public health emergency. We have been funded by the NASEM to conduct confidential interviews and focus groups as part of that consensus study.

We’re conducting this interview with you today as part of this evaluation. The goal of this discussion is to better understand key stakeholders’ perspectives regarding CDC ISO’s vaccine safety processes, priorities, communications, and collaborations during the COVID-19 emergency. We’re speaking with individuals from numerous key informant groups, including CDC staff, ISO contractors, staff from other federal, state, and local health agencies, healthcare professionals, and other vaccine researchers. Ultimately, insights shared through these interviews and focus groups will be used to develop recommendations for sustaining, maintaining, and strengthening CDC ISO’s vaccine safety monitoring systems and communications.

We have a series of questions to guide our discussion. You may not know the answer to every question, and there are no right or wrong answers. We are interested in your individual perspective. Your participation in this [interview/focus group] is voluntary. If there are any questions that you don’t feel knowledgeable about or don’t feel comfortable answering, please let us know, and we will move on.

[Only if focus group/multi-person interview]: We will also be using a tool called a Mural Board that allows you to anonymously comment and respond to the discussion.

You should have received a copy of the consent form via email prior to this meeting. It explains much of the information I just reviewed, including why we are conducting this [interview/focus group].

[Only if focus group]: As we noted there, because this is a focus group, you all are of course aware of each other’s participation and will have knowledge of each of your perspectives. We ask all of you to respect the privacy of each other and not disclose other participants’ feedback.

Information gathered during our [interview/focus group] will be used in combination with other interviews and focus groups in a summary report which will be available on NASEM’s website. None of the information you share with us today will be attributed to you personally in any way. We are interviewing many people for this study, and we will paraphrase, rather than direct quote, participants. Agency and organization names will be mentioned, but again, no individuals will be directly identified. We’ll be taking notes during our discussion, and if [you/everyone] agrees, we will record this discussion as a backup to those notes.

Please note: the recording, transcript, and notes from our discussion will be stored on Westat’s secure server and will only be accessible to the Westat staff working on this study. Any sensitive material, including personally identifiable information (PII), will be redacted from transcripts and notes. Recordings will be deleted upon review and redaction of PII from transcripts and notes. All other data, such as transcripts and analytic files, will be destroyed after the overall study is completed and published.

Since this is a videoconference, we strongly encourage you to participate in a private setting away from others. Also, while we are employing all appropriate security measures during this virtual interview, it is important that you know that no system such as Zoom is 100 percent secure.

1. Do you have any questions?
2. Do you agree to proceed with our interview or not?
3. Do you agree to have this conversation recorded or not?

[Note to researchers: If the key informant does not give permission to record, proceed with the interview. In this case, it is essential that the non-interviewing researcher(s) on the team take extensive notes, capturing as much detail as possible.]

Federal Government Staff Guide

Interviewee’s Role

I’d like to start by learning about your work and area of expertise and focus. This is to help us interpret your responses and analyze them. We will not directly quote you or refer to your position directly in our summary report.

1. What is your area of expertise/position?
2. What was your role or focus during the COVID-19 emergency?

Resource Allocation

We’d like to learn about your experiences during the COVID-19 emergency starting in March 2020 through May 2023. Let’s start by talking about vaccine safety resource allocation.

3. [P] Thinking back to the time of the COVID-19 emergency, how did resourcing for vaccine safety evaluation, assessment and research change?
4. Was additional funding made available to your agency? What was the additional funding used for? What was impact (positive/negative)?
5. [P] Were additional staff hired/brought in to support vaccine safety evaluation, assessment and research? What were their roles?
6. [P] What was the impact of these changes?
7. [P] Have programmatic resourcing increases (new efforts, new staff, funding streams) targeted to Covid vaccine safety been retained post COVID-19 emergency? If so, in what areas/what ways?

Vaccine Safety Evaluation, Assessment and Research Priorities

8. How do you/ does your organization set priorities for vaccine safety evaluation, assessment and research? How does that align with CDC’s priorities? a. How effective is that process? Did it change during the COVID-19 emergency, and if so, how?
9. [P] How do you/ does your organization engage with CDC ISO in setting priorities for vaccine safety evaluation, assessment and research?
   1. [P] Is this engagement sufficient? How can vaccine safety evaluation, assessment and research collaborations with CDC ISO be improved?

10. How do the evaluation, assessment and research priorities differ between you/your organization (or office) and CDC ISO?
    1. How did that impact (benefit or hinder) your mission or overall vaccine safety research?

Coordination in Federal Vaccine Safety Efforts

Our next several questions pertain to coordination efforts between numerous organizations, systems, and processes that contribute to vaccine safety.

11. [P] From your perspective, how robust and effectively coordinated are vaccine safety efforts within the federal government?
12. Are coordination mechanisms adequate to prevent unnecessary duplication of effort or to ensure optimal vaccine safety and community?
    1. If there are coordination gaps, what are they and what are their impacts?
    2. How could these coordination gaps be addressed?
13. [P] How could federal vaccine safety efforts be improved overall?

Public Communication

Lastly, we’d like to discuss efforts pertaining to vaccine safety communication.

14. What was your agency’s process for determining vaccine communications content/messages?
15. [P] How was this process coordinated within your agency and with CDC’s ISO and what other agencies or stakeholders were involved?
16. [P] Did vaccine safety communications differ in terms of content or prioritization between your agency and CDC ISO?
17. [P] During the COVID-19 emergency, what formal guidance or criteria, if any, was provided for communications about potential vaccine safety issues?
    1. Were there competing organizational perspectives on the right approach for communicating potential safety issues?
    2. If so, how were these adjudicated?
    3. Can you think of a time when a safety issue message was enhanced or softened as it went through the adjudication process?
18. [P] If your office or agency put out vaccine safety information, how did that information or guidance change over time during and after the COVID-19 emergency?

Those are all the questions I have for you today. Is there anything else you’d like to share regarding your experiences during the COVID-19 emergency?

Thank you so much for taking the time to share your experiences and feedback. This concludes our discussion. Have a great rest of your day!

ISO Funded Researchers - VSD

Interviewee’s Role

I’d like to start by learning about your role within VSD and your area of expertise or focus. This is to help us interpret your responses and analyze them. We will not be directly citing or referring to your position in our summary report.

1. What is your role or area of expertise (what are your current responsibilities)?
2. Was your role during the COVID-19 emergency different from what you had been doing prior to the COVID-19 emergency?

Vaccine Safety Evaluation, Assessment and Research Priorities and Independence

First, we’d like to learn about how vaccine safety research and evaluation priorities are determined within the VSD project.

3. What is the process for setting vaccine safety research and evaluation priorities within VSD and within your site?
   1. [P] What works well in this process and what is challenging?
   2. How could this be improved?
4. [P] Did the process for setting priorities change during the COVID-19 emergency? If so, how?
5. Did you feel that you had input into CDC ISO’s overall process of setting research priorities around vaccine safety assessment and communications? Why or why not?

Relationship with CDC

Now, we’d like to learn about your experiences during the COVID-19 emergency starting in March 2020 through May 2023. Let’s start by talking about your relationship with CDC.

6. [P] In what ways did your relationship with CDC change during the transition from non- emergency or routine vaccination safety assessment, evaluation and research to the COVID-19 public health emergency response?
   1. [P] What worked well? What were the challenges?
   2. Were there ways in which your relationship with CDC could have been improved?
7. [P] Please describe any changes made during the COVID-19 emergency you feel had an impact on your work. These could be evaluation, assessment or research-related changes or administrative or procedural changes.
8. Were there challenges due to these changes? How did they help or improve your work?
9. What other improvements or changes were needed?
10. Are there changes that you think should or should not have remained in place when transitioning back to non-emergency operations?
11. How does your experience working with ISO compare with your experience working with other research funding sources, including industry (note to interviewer – this often is tied more to contract type and should potentially be asked about)?
    1. In what ways is it similar? How is it different?
    2. In terms of independence? Ease of collaboration? Publishing? Speed and ease of contracting, other mechanisms?
12. [P] How effective was the leadership provided by ISO during the pandemic?
13. Were there any challenges you or your organization encountered with the CDC contracting process? How is it now?
    1. How can the contracting process be improved, if at all?
14. [P] Have you ever experienced a situation in which there was disagreement between you or your team and ISO on the seriousness of a vaccine safety or how to respond to a vaccine safety signal (timeliness of reporting, etc.)?
    1. What happened as a result of the disagreement or how was the disagreement resolved?
    2. Did this situation occur during the COVID-19 emergency? Does being in an emergency response situation change the process for addressing situations like this?
    3. What changes or improvements could be made to resolve these situations?

Communication and Public Perception

Next, we’d like to discuss how CDC communicated the results of vaccine safety evaluation, assessment and research to the public.

15. [P] How effectively do you think CDC communicates research findings to healthcare professionals and the public, especially individuals from marginalized groups?
    1. Did you feel that your work was sufficiently shared/distributed?
16. [P] During the COVID-19 emergency (or at any other time), did you experience a time in which you were surprised by or disagreed with how a CDC communication described VSD research? Can you describe this situation?
17. [P] What changes could be made to improve CDC’s process for communicating findings from VSD research?

General Experience During Public Health Emergency and Suggestions for Improvement

The last few questions are about your general experiences during the COVID-19 emergency and how you feel systems, processes, research, and communication could be improved.

18. [P] What would you recommend to improve CDC’s safety monitoring efforts going forward (aside from additional funding)?
19. [P] How can CDC vaccine safety researchers improve public communications pertaining to vaccine safety research?
20. What was your biggest accomplishment during the COVID-19 emergency and what made it successful?
21. [If needed] What was your biggest disappointment during the COVID-19 emergency?

Those are all of the questions I have for you today. Is there anything else you’d like to share regarding your experiences during the COVID-19 emergency?

Thank you so much for taking the time to share your experiences and feedback.

ISO Funded Researchers - CISA

Interviewee’s Role

I’d like to start by learning about your role and area of expertise and focus. This is to help us interpret your responses and analyze them. We will not be directly citing or referring to your position in our summary report.

1. What is your role/area of expertise (what are your current responsibilities)?
2. Was your role during the COVID-19 emergency the same as prior to the COVID-19 emergency or did you assume a new role or position during the COVID-19 emergency? What office were you in?

Vaccine Safety Evaluation, Assessment and Research Priorities and Independence

First, we’d like to learn about how vaccine safety research and evaluation priorities are determined within [CISA].

3. How do you set your vaccine safety research and evaluation priorities or how are they set within CISA?
   1. [P] What works well in this process and what is challenging? How could this be improved?
4. [P] Did the process for setting priorities change during the COVID-19 emergency? If so, how?
5. Did you feel that your work/results (or input on priorities) were appropriately incorporated in CDC ISO’s overall process of vaccine safety assessment and communications? Why or why not?
6. Did you feel that your work was sufficiently shared/distributed?

Relationship with CDC

Now, we’d like to learn about your experiences during the COVID-19 emergency starting in March 2020 through May 2023. Let’s start by talking about your relationship with CDC.

7. [P] In what ways did your relationship with CDC change during the transition from non- emergency/routine vaccination safety assessment, evaluation and research to the COVID-19 public health emergency response period of time (3/2020 to 5/2023)?

7. How did your safety assessment, evaluation, research and work change?
   1. [P] What worked well? What were the challenges?
   2. How could your relationship with CDC have been improved?
8. [P] Please describe any evaluation, assessment or research-related or administrative or procedural changes made during the COVID-19 emergency you feel had an impact on your work.
   1. Were there challenges due to these changes? How did they help or improve your work?
   2. What other improvements or changes were needed?
   3. Are there changes that you think should or should not have remained in place when transitioning back to non-emergency operations?
9. How does your experience working with ISO compare with your experience working with other research funding sources, including industry (note to interviewer – this often is tied more to contract type and should potentially be asked about)?
   1. In what ways is it similar? How is it different?
   2. In terms of independence? Ease of collaboration? Publishing? Speed and ease of contracting, other mechanisms?
10. [P] Describe the quality or effectiveness (clarity) of leadership provided by CDC during the pandemic? Were there any challenges you encountered with the CDC contracting process? How is it now?
    1. How can the contracting process be improved, if at all?
11. [P] Have you ever experienced a situation in which there was disagreement between you/your team and CDC on the seriousness and response to a vaccine safety signal (timeliness of reporting, etc.)?
    1. What happened as a result of the disagreement or how was the disagreement resolved?
    2. Did this situation occur during the COVID-19 emergency? Does being in an emergency response situation change the process for addressing situations like this?
    3. What changes or improvements could be made to resolve these situations?

Communication and Public Perception

Next, we’d like to discuss how CDC communicated the results of vaccine safety evaluation, assessment and research to the public.

12. [P] How effectively do you think CDC communicates research

12. findings to healthcare professionals and the public, especially individuals from marginalized groups?
13. [P] During the COVID-19 emergency (or at any time alternatively), did you experience a time in which you were surprised by or disagreed with how a CDC communication described CISA research? Can you describe this situation?
14. [P] What changes could be made to improve CDC’s process for communicating findings from CISA research?

General Experience During Public Health Emergency and Suggestions for Improvement

The last few questions pertain to your general experiences during the COVID-19 emergency and how you feel systems, processes, research, and communication could be improved.

15. What was your biggest accomplishment during the COVID-19 emergency and what made it successful?
16. [If needed] What was your biggest disappointment during the COVID-19 emergency?
17. [P] What would you recommend to improve CDC’s safety monitoring efforts going forward (aside from additional funding)?
18. [P] How can CDC vaccine safety researchers improve public communications pertaining to vaccine safety research?

Those are all of the questions I have for you today. Is there anything else you’d like to share regarding your experiences during the COVID-19 emergency?

Thank you so much for taking the time to share your experiences and feedback.

Healthcare Professionals

Interviewee’s Role

I’d like to start with brief introductions.

1. What is your area of expertise/profession? [Note: If they mentioned this in the survey, consider rephrasing to “You mentioned your area of focus is X.. Is there anything else you’d like to share about your profession or areas of expertise?”
2. What was your role or focus during the COVID-19 emergency (from March 2020 through May 2023)?

Vaccine Safety Systems

Next, we’d like to learn about your experiences as a healthcare professional during the COVID-19 emergency, starting in March 2020 through May 2023. Let’s start by talking about your familiarity and experience with the CDC ISO’s vaccine safety systems. [May require providing some brief info on CDC ISO role vs FDA.]

3. [P] [If they did not indicate this in the survey] Which of the following vaccine safety reporting systems are you aware of: V-safe, VAERS, VSD, or CISA?
4. [P] [If aware of V-safe] How were patients encouraged to use V-safe to report vaccine side effects by either you or your colleagues in your practice during the COVID-19 emergency?
   1. [If encouraged]: How frequently were patients encouraged to use V-safe?
   2. [If encouraged]: How did patients respond to this encouragement?
   3. [If encouraged]: What did patients like or dislike about V-safe?
5. [P] [If aware of VAERS] Did you or your patients ever make a report to VAERS during the COVID-19 emergency?
   1. [If yes]: What was that process like?
   2. Were there barriers or challenges for you to report to VAERS?
   3. From your perspective, what could be done to improve this reporting process?

Communication, Public Perception, and Influence

The next several questions focus on your experience with vaccine safety communications during the COVID-19 emergency.

6. [P] How and where did you and your colleagues get COVID-19 vaccine safety information during the COVID-19 emergency?
7. If you had questions regarding vaccine safety during the COVID-19 emergency, what resources did you use to get answers and did you know who to contact for answers?
8. [P] How confident are you or your colleagues in the trustworthiness of COVID-19 vaccine safety information provided by CDC?
   1. [If confident]: What factors have contributed to high confidence in COVID-19 vaccine safety information provided by CDC?

2. [If not confident]: What factors have contributed to low confidence in COVID-19 vaccine safety information provided by CDC?
3. [P] What could CDC do to increase confidence in COVID-19 vaccine safety information?
9. [P] From your perspective, how clear and timely were CDC’s communications regarding COVID-19 vaccine safety during the COVID-19 emergency?
10. [P] From your perspective, how effectively does the CDC address public concerns about COVID-19 vaccine safety?
    1. How does your perception of CDC’s effectiveness at addressing public concerns differ between during a public health emergency to other times?
11. What ways have patients shared they receive COVID-19 vaccine safety information?
    1. What ways have patients from marginalized groups shared they receive COVID-19 vaccine safety information?
    2. What improvements would you recommend for communicating COVID-19 vaccine safety information to patients, the public, and marginalized groups?

General Experience During Public Health Emergency and Suggestions for Improvement

Lastly, we’d like to learn about your overall experience with CDC’s vaccine safety research and communications, and suggestions for improvement for the future.

12. [P] What is your overall perception of CDC’s efforts pertaining to vaccine safety?
    1. What could the CDC do differently to enhance these efforts?
    2. What kinds of process improvements to vaccine safety systems, research, and communications would you recommend to the CDC?
13. [P] What do you think is the most important way to improve CDC’s vaccine safety systems, research, and communications?
14. Is there anything else you’d like to share regarding your perspectives on vaccine safety as a healthcare professional?

Those are all the questions we have for you today. Thank you so much for taking the time to share your experiences and feedback. This concludes our discussion. Have a great rest of your day!

Outside Vaccine Safety Experts

Interviewee Role’s

I’d like to start with brief introductions.

1. What is your area of expertise/profession?
2. What was your role or focus during the COVID-19 emergency (from March 2020 through May 2023)?

Perception of CDC ISO Vaccine Safety Research, Expertise, and Communications

Next, we’d like to learn about your perspectives regarding the CDC ISO’s vaccine safety research focus areas, as well as the CDC ISO’s capabilities and expertise. [Research team – there may be limited understanding of CDC efforts vs. other agencies and we need to probe on responses to distinguish.]

3. How do you view and balance different sources of vaccine safety research information?
4. How do you view the CDC ISO’s vaccine safety research and communications?
   1. What are the strengths of or gaps in CDC ISO’s vaccine safety research and communications?
   2. How has your perception of CDC ISO’s vaccine safety research and communications changed since during the COVID-19 emergency to now?
5. How do you view the CDC ISO’s vaccine safety capabilities and expertise?
   1. What are the strengths of or gaps in CDC ISO’s vaccine safety capabilities and expertise?
   2. How has your perception of CDC ISO’s vaccine safety capabilities and expertise changed since during the COVID-19 emergency to now?
6. From your perspective, does CDC ISO ask the right vaccine safety questions? Why or why not?
7. From your perspective, does CDC ISO have the necessary resources to research vaccine safety in a timely manner? Why or why not?

8. From your perspective, does the CDC ISO communicate their vaccine safety research results in a timely and appropriate manner? Why or why not?
9. What is your perspective on the issue of balancing informing the public about potential vaccine safety issues and providing the public with clear vaccine guidance?
   1. How well did the CDC ISO strike this balance during the COVID-19 emergency? How could the CDC ISO better address this balance?
10. How do you view CDC ISO’s vaccine safety information in comparison to other sources?

Usage of CDC Data and Collaborations

Our next several questions focus on your experiences with the CDC ISO’s vaccine safety data, as well as your collaborations and communications with the CDC ISO.

11. How have you used VAERS or VSD public data in your own vaccine safety research?
    1. How easy or difficult was it to use these data sources?
    2. What made these data sources easy or difficult to use?
    3. How could these data sources be improved?
12. How useful were the following resources to you during the COVID-19 emergency compared to now? [probe on each resource]
    1. V-safe
    2. CISA
    3. VAERS
    4. VSD
13. Outside the context of a public health emergency, how do you collaborate and communicate with CDC’s ISO?
    1. How frequently do you try to use CDC ISO vaccine safety data?
    2. [If not mentioned]: How frequently do you collaborate with the CDC ISO?
    3. [If not mentioned]: What types of collaboration and communication do you have with CDC’s ISO?
    4. What worked well in your collaboration with the CDC ISO?
    5. What could be improved in your collaboration with the CDC ISO?

General Experience During Public Health Emergency and Suggestions for Improvement

Lastly, we’d like to learn about your overall experience with CDC ISO’s vaccine safety processes and communications, and suggestions for improvement for the future.

14. How can CDC ISO’s vaccine safety systems, processes, research, and communications be improved going forward?
    1. How can the efficiency and effectiveness of vaccine safety data sharing be improved?
15. How can CDC ISO’s vaccine safety processes and communications be improved to strengthen the public’s confidence in vaccines?
    1. How can CDC’s vaccine safety processes and communications be improved to strengthen marginalized groups’ confidence in vaccines?
16. Is there anything else you’d like to share regarding your experiences regarding vaccine safety as a vaccine researcher during the COVID-19 emergency?

Those are all the questions we have for you today. Thank you so much for taking the time to share your experiences and feedback. This concludes our discussion. Have a great rest of your day!

Public Health Officials

Interviewee’s Role

I’d like to start by learning about your role and experience. This is to help us interpret and analyze your responses. We will not directly cite or refer to your position in our summary report.

1. What is your position, and what are your current responsibilities?
2. What was your role during the COVID-19 emergency (from March 2020 through May 2023)?

Communication, Public Perception, and Influence

Next, we’d like to learn about your experiences as a [state, local, or tribal] public health official with vaccine safety communications during the COVID-19 emergency, starting in March 2020 through May 2023.

3. [P] How and where did you get your COVID-19 vaccine safety messaging and talking points?
   1. [If not mentioned]: What resources from the CDC and other U.S. federal agencies did you use?
   2. [P] [If not mentioned]: Did you use the vaccine safety information contained on CDC’s Interim Clinical recommendation for COVID-19 vaccines webpage or CDC HAN alerts?
      1. [If no]: Why did you not use these resources?
   3. Other than U.S. federal agencies, what sources did you use to find and communicate COVID-19 vaccine safety information?
      1. [If not mentioned]: Did you use messaging from vaccine manufacturers, other countries, or the World Health Organization? If so, which of these did you use?
      2. [If yes]: If you used messaging from sources other than CDC and U.S. federal agencies, why did you use them?
   4. When your health department received questions about COVID-19 vaccine safety from healthcare providers or other groups, what resources did the health department use to answer these questions?
4. [P] Did providers raise questions or concerns regarding information they were leveraging from CDC on vaccine safety?
5. [P] What feedback can you share about COVID-19 vaccine safety information or resources that you used from the CDC/Advisory Committee on Immunization Practices (ACIP)?
   1. What was your experience navigating these resources to find needed information and answer questions, especially considering when vaccine safety information recommendations evolved over time?
6. [P] What challenges did your health department encounter with regards to disinformation about COVID-19 vaccine safety?
   1. How useful were CDC’s resources for addressing or clarifying issues?
7. [P] What types of additional vaccine safety communications support from CDC would have been helpful for your health department?
   1. [If not mentioned]: Which of the following resources would have been helpful: template presentations, handouts, job aids, FAQ documents, office hours, or others?

General Experience During Public Health Emergency and Suggestions for Improvement

Lastly, we’d like to learn about your overall experience with and perspective regarding vaccine safety information and communications.

7. What is your overall perception of CDC ISO’s vaccine safety information?
8. [P] How, if at all, has your overall perception of CDC ISO’s vaccine safety information changed from during the COVID-19 public health emergency to now?
9. [P] What factors do you consider when determining what vaccine safety information to share with the public?
   1. Which factors do you think are the most important in determining vaccine safety information to share with the public?
10. Is there anything else you’d like to share regarding your experiences with vaccine safety information during the COVID-19 emergency?

Those are all the questions we have for you today. Thank you so much for taking the time to share your experiences and feedback. This concludes our discussion. Have a great rest of your day.

APPENDIX B. RECRUITMENT LETTERS

Section A. Westat Recruitment Letters

Health Professionals, Vaccine Safety Experts, State and Local PH officials

To: X

Cc: Jennifer Edwards, Westat Project Director

Subject: [Response Requested]: Participation in NASEM/Westat interview on COVID-19 PHE experience

Dear X,

The Centers for Disease Control and Prevention (CDC) Immunization Safety Office (ISO) is sponsoring a study by the National Academies of Sciences, Engineering, and Medicine (NASEM) to evaluate COVID-19 vaccine safety research and communications during the public health emergency. Please see additional information about the study here:

Review of CDC COVID-19 Vaccine Safety Research and Communications | National Academies.

As a part of this study, NASEM and a NASEM appointed Committee would like to solicit input and feedback from government staff, researchers, healthcare professionals, and state and local health department officials about their experiences during the public health emergency (from December 2020 – May 2023) and perspectives on policies and procedures that worked or need improvement, collaboration and coordination, and communications about vaccine safety.

NASEM staff, in consult with the Committee, identified you as an expert in one or more of these topic areas. We are writing to see if you would be willing to participate in either a 60-minute one- on-one interview or a 90-minute focus group in January or February of 2025. To identify your interest or potential willingness to participate, we are providing a link below to a very brief survey (Please feel free to complete this from home as well if you prefer; please check your agency/organization IT policies before responding to ensure your privacy on those systems). In this survey, you can provide your preferred contact method, indicate your willingness to participate in either an interview of focus group, and answer a few questions to help us identify or confirm your areas of expertise. We may not be able to interview all interested participants.

This study is being carried out under the oversight of the NASEM IRB. Your participation in the study will be confidential – only Westat will have access to participant information and will not disclose the names, positions, or other identifying information of participants; all information to include your name, contact information, response to the survey, and subsequent interview material will be deleted. We will provide additional information regarding participant privacy and confidentiality during scheduling should you be selected to participate.

Link:
Password:
Participants can also access the survey via this QR code:

Please reach out to the Project Director, Jennifer Edwards, with any additional questions you may have.

Jennifer Edwards, PhD
Principal Investigator and Project Director, Westat

Sincerely, Neni Osuoha

Neni Osuoha, MPH, PMP (she, her, hers) Senior Research Associate | Westat Social Policy and Economic Research

1600 Research Blvd. | Rockville, MD 20850

CDC Staff, Funded Researchers, and other Govt Staff

To: X

Cc: Jennifer Edwards, Westat Project Director

Subject: Participation in NASEM/Westat interview on COVID-19 PHE experience Dear X,

The Centers for Disease Control and Prevents (CDC) Immunization Safety Office (ISO) is sponsoring a study by the National Academies of Sciences, Engineering, and Medicine (NASEM) to evaluate COVID-19 vaccine safety research and communications during the public health emergency. You should have received notification of this study in an email from Dr. Meyer on January 10th. See additional information about the study here: Review of CDC COVID-19 Vaccine Safety Research and Communications | National Academies.

As a part of this study, NASEM and a NASEM-appointed committee would like to solicit input and feedback from government staff, researchers, healthcare professionals, and state and local health department officials about their experiences during the public health emergency (from December 2020 – May 2023) and perspectives on policies and procedures that worked or need improvement, collaboration and coordination, and communications about vaccine safety.

NASEM staff, using publicly available information, identified you as an expert in one or more of these topic areas. We are writing to see if you would be willing to participate in a 60-minute one- on-one interview in January and February. To identify your interest or potential willingness to participate, we are providing a link below to a very brief survey. In this survey, you can provide your preferred contact method, indicate your willingness to participate, and answer a few questions to help us identify or confirm your areas of expertise. We may not be able to interview all willing participants.

This study is being carried out with the oversight of the NASEM IRB. Your participation in the study will not be disclosed to CDC staff, NASEM staff, or the Committee and all information to include your name, contact information, response to the survey, and subsequent interview material will be deleted. To further ensure your privacy, please refer to agency IT policies before using your official email to respond or choosing your method of response as communications on federal systems may be subject to FOIA requests. We provide additional information regarding participant privacy and confidentiality on the survey site.

Link:
Password:
Participants can also access the survey via this QR code:

Please reach out to the Project Director, Jennifer Edwards, with any additional questions you may have.

Jennifer Edwards, PhD

Principal Investigator and Project Director, Westat

Sincerely, Neni Osuoha

Neni Osuoha, MPH, PMP (she, her, hers) Senior Research Associate | Westat Social Policy and Economic Research

1600 Research Blvd. | Rockville, MD 20850

APPENDIX C. INFORMED CONSENT

Individual Interview Informed Consent Form

Purpose of Study

In early 2024, the Centers for Disease Control and Prevention (CDC) Immunization Safety Office (ISO) asked the National Academies of Sciences, Engineering, and Medicine (NASEM) to conduct a consensus study to evaluate the systems, methods, and processes for monitoring COVID-19 vaccine safety during the U.S. COVID-19 public health emergency, and provide recommendations for sustaining, maintaining, and strengthening CDC ISO current monitoring systems and communications moving forward.

Westat was contracted by NASEM’s Health and Medicine Division (HMD) to conduct this study for the CDC. Our purpose is to conduct confidential interviews with key informants regarding the ISO’s research on

COVID-19 vaccine safety and communications about COVID-19 vaccine safety during the public health emergency (PHE). The findings from these interviews will be analyzed and summarized for inclusion in a publicly available report produced by NASEM.

Participant Selection

You were identified by NASEM staff and NASEM appointed committee members as knowledgeable about the CDC ISO’s systems, methods, and processes for monitoring COVID-19 vaccine safety. You completed a participant screening form and were selected to participate in a 60-minute interview for this study. Although NASEM identified potential study participants, Westat will not disclose participants names to NASEM or CDC.

Study Procedure

Depending on your role, we may ask you questions about three general topic areas regarding CDC vaccine safety research and communications during the COVID-19 emergency: (1) policies and procedures; (2) interagency collaboration and coordination; and (3) communication and public perception.

To confirm your knowledge of these topics, the brief recruitment survey asks you to confirm your professional affiliation (where you work) now at and at the time of the COVID-19 public health emergency as well as your familiarity with or use of ISO information. It also asks for your preferred contact method. If selected for an interview, we will reach out to schedule that interview. Once completed, survey responses including your contact information and name are destroyed. If not selected, your survey response and contact information will also be removed/destroyed.

Following completion of your interview, conducted via Zoom and recorded (if you agree), we will transcribe the interview and redact any potentially identifiable information. At that time, the recording of your interview will also be destroyed as will the unredacted transcript. The redacted transcript will be retained on Westat secure systems until the report is published and then will be destroyed.

Benefits

There will be no direct benefit to you for your participation in the study. An indirect benefit of your participation will be the knowledge that your input was invaluable and contributed to strengthening future CDC vaccine monitoring systems and communication efforts.

Risks and Discomfort & Privacy and Confidentiality

The potential risks to participation in this study may vary based on the role and affiliation of the participant, with government employees facing additional professional risks potentially due to the high level of public interest in the topic and the potential for FOIA requests in regard to the study. While our procedures minimize this risk and there is an exemption category to a FOIA request that should apply here (category 6), we cannot completely eliminate risk. We recognize conversations about vaccine safety and the COVID-19 emergency can be politically sensitive and reflecting on that time period can cause emotional discomfort. The information you provide will only be used for the research purposes of this study and it will be handled in a private and confidential manner. What we discuss will not be shared with your colleagues at any level or the public. Please be advised that we may include paraphrases of comments made in the report, but these will not be attributed to your name or position title.

We would like to record the interview for notetaking purposes only with your permission. You may decide not to be recorded, pause the recording, or stop the recording altogether at any time. All study materials, including the meeting recording, transcript, and notes will be stored on Westat’s secure server and will only be accessible to the Westat staff working on this study. Any sensitive material, including personally identifiable information (PII), will be redacted from interview transcripts and notes. Participant names and contact information, including Qualtrics survey responses, will be destroyed upon interview completion. Interview recordings will be deleted upon review and redaction of PII from transcripts and notes. All other data, such as transcripts and analytic files, will be destroyed after the overall study is completed and published.

Study Costs and Compensation

There are no costs for your participation in this study apart from the time you spend with us for this interview. There is no monetary compensation for your participation in this study.

Voluntary Participation

Your participation in the interview as well as responding to individual interview questions is voluntary. You may decide not to participate, and it will not have any impact on your position. You may take a break, skip questions, speak off the record, or stop participating at any time.

More Information

If you have any questions or would like more information about this study, please contact Jennifer Edwards, the study Project Director, at 1-301-212-3216. If you have questions about your rights and welfare as a research participant, please call the NASEM Institutional Review Board office at 202.334.1616. Please leave a message with your full name, the name of the research study that you are calling about (Key Informant Interviews for the National Academies’ Health and Medicine Division), and a phone number beginning with the area code. Someone will return your call as soon as possible.

Certification of Informed Consent

We will ask you to verbally consent to participate in the study and confirm you have reviewed this agreement during the interview.

Focus Group Informed Consent Form

Purpose of Study

In early 2024, the Centers for Disease Control and Prevention (CDC) Immunization Safety Office (ISO) asked the National Academies of Sciences, Engineering, and Medicine (NASEM) to conduct a consensus study to evaluate the systems, methods, and processes for monitoring COVID-19 vaccine safety during the U.S. COVID-19 public health emergency, and provide recommendations for sustaining, maintaining, and strengthening CDC ISO current monitoring systems and communications moving forward.

Westat was contracted by NASEM’s Health and Medicine Division (HMD) to conduct this study for the CDC. Our purpose is to conduct confidential interviews with key informants regarding the ISO’s research on COVID-19 vaccine safety and communications about COVID-19 vaccine safety during the public health emergency (PHE). The findings from these interviews will be analyzed and summarized for inclusion in a report produced by NASEM.

Participant Selection

You were identified by NASEM staff and NASEM appointed committee members as knowledgeable about the CDC ISO’s systems, methods, and processes for monitoring COVID-19 vaccine safety. You completed a participant screening form and were selected to participate in a 90-min-ute focus group for this study. Although NASEM identified potential study

participants, Westat will not disclose participants’ names to NASEM or CDC or any other entity.

Study Procedure

Depending on your role, we may ask you questions about three general topic areas regarding CDC vaccine safety research and communications during the COVID-19 emergency: (1) policies and procedures; (2) interagency collaboration and coordination; and (3) communication and public perception.

To confirm your knowledge of these topics, the brief recruitment survey asks you to confirm your professional affiliation (where you work) now at and at the time of the COVID-19 public health emergency as well as your familiarity with or use of ISO information. It also asks for your preferred contact method. If selected for a focus group, we will reach out to schedule that. Once completed, survey responses including your contact information and name are destroyed. If not selected, your survey response and contact information will also be removed/destroyed.

Following completion of your focus group, conducted via Zoom and recorded (if all agree), we will transcribe the session and redact any potentially identifiable information. At that time, the recording of your session will also be destroyed as will the unredacted transcript. The redacted transcript will be retained on Westat secure systems until the report is published and then will be destroyed.

Benefits

There will be no direct benefit to you for your participation in the study. An indirect benefit of your participation will be the knowledge that your input was invaluable and contributed to strengthening future CDC vaccine monitoring systems and communication efforts.

Risks and Discomfort & Privacy and Confidentiality

The potential risks to participation in this study may vary based on the role and affiliation of the participant. We recognize conversations about vaccine safety and the COVID-19 emergency can be politically sensitive and reflecting on that time period can cause emotional discomfort. We encourage you to only answer questions you feel comfortable responding to. There is no obligation to respond to each question. To facilitate information sharing, the focus group will also use a tool/platform that enables anonymous responses. The information you provide will only be used for the research purposes of this study and it will be handled in a private and confidential manner. Please be advised that we may include paraphrases

of your statements in the report, but these will not be attributed to your name or position title.

Because this is a focus group the other participants will of course be aware of your feedback and you will be aware of theirs. We will remind all participants to respect the privacy of each other and not disclose individuals’ feedback. However, disclosure from other participants is a risk.

We would like to record the session with your permission. All study materials, including the meeting recording, transcript, and notes will be stored on Westat’s secure server and will only be accessible to the Westat staff working on this study. Any sensitive material, including personally identifiable information (PII), will be redacted from interview transcripts and notes. Participant names and contact information, including Qualtrics survey responses, will be destroyed upon interview completion. Interview recordings will be deleted upon review and redaction of PII from transcripts and notes. All other data, such as transcripts and analytic files, will be destroyed after the overall study is completed and published.

Study Costs and Compensation

There are no costs for your participation in this study apart from the time you spend with us for this interview. There is no monetary compensation for your participation in this study.

Voluntary Participation

Your participation in the interview as well as responding to individual interview questions is voluntary. You may decide not to participate, and it will not have any impact on your position. You may take a break, skip questions, speak off the record, or stop participating at any time.

More Information

If you have any questions or would like more information about this study, please contact Jennifer Edwards, the study Project Director. If you have questions about your rights and welfare as a research participant, please call the NASEM Institutional Review Board office at 202.334.1616. Please leave a message with your full name, the name of the research study that you are calling about (Key Informant Interviews for the National Academies’ Health and Medicine Division), and a phone number beginning with the area code. Someone will return your call as soon as possible.

Certification of Informed Consent

We will ask you to verbally consent to participate in the study and confirm you have reviewed this document at the start of the focus group.