Summary¹

The Centers for Disease Control and Prevention’s (CDC’s) Immunization Safety Office (ISO) is responsible for studying vaccine risks once vaccines are administered to the public. ISO was already a federal focal point for vaccine risk evaluation when the COVID pandemic emerged and the Secretary of Health and Human Services (HHS) declared a nationwide public health emergency (PHE). By the end of 2020, vaccines were available to the general public due to Operation Warp Speed and a Food and Drug Administration (FDA) regulatory pathway called Emergency Use Authorization.

ISO is within the Division of Healthcare Quality Promotion of the National Center for Emerging and Zoonotic Infectious Diseases and administratively and organizationally separate from offices in the National Center for Immunization and Respiratory Diseases that support vaccine use. During the PHE, ISO underwent temporary restructuring as part of CDC’s comprehensive response. Its vaccine specialists were integrated into the broader COVID-19 Vaccine Task Force, enabling rapid scale-up of monitoring and evaluation capabilities by leveraging both established systems and novel platforms for timely assessments of vaccine risks.

Vaccines have long been essential in public health, recognized as one of the most significant achievements of the 20th century. Unlike treatments to manage infectious diseases, vaccines prepare the immune system to prevent or mitigate infections and are often administered well before potential disease exposure. Given their preventive nature, their importance is often

___________________

¹ References are not included in this summary; citations are in the report chapters.

overlooked and may create unique ethical considerations, especially since vaccination decisions are sometimes mandated in certain settings, such as schools or health facilities. Despite rigorous scientific scrutiny through clinical trials and ongoing postmarket safety monitoring, the COVID vaccine became suspect to some because of public confusion, concern, and mistrust about recommended public health measures employed during the PHE, including vaccines.

STATEMENT OF TASK

CDC commissioned the National Academies of Sciences, Engineering, and Medicine to undertake a comprehensive evaluation of ISO. An ad hoc committee was tasked with conducting a thorough assessment of ISO’s statistical and epidemiological methods in vaccine risk monitoring and evaluation, including detailed scrutiny of processes designed to detect, evaluate, and report potential problems associated with COVID vaccines, which encompassed systems like the Vaccine Adverse Event Reporting System (VAERS) and Vaccine Safety Datalink (VSD).

Critically, the committee was also charged with evaluating CDC’s external communication strategies targeted toward diverse audiences, including health professionals, public health authorities, and the general public. The evaluation involved reviewing communication practices and analyzing content.

Additionally, the committee’s mandate included providing actionable recommendations designed to sustain and enhance ISO’s vaccine risk monitoring and communication systems. These recommendations aim to ensure the robustness, scientific rigor, transparency, and responsiveness of ISO’s activities, positioning it to more effectively manage future vaccine risk² monitoring and public communication during ongoing and emergent public health initiatives.

COMMITTEE APPROACH

The committee conducted its review of ISO’s vaccine risk monitoring, evaluation, and communication practices using diverse and complementary approaches. This included a detailed analysis of ISO’s systems and publications for effectiveness in identifying, assessing, and communicating about vaccine risks.

The committee held multiple public meetings to gather information from ISO, health professionals, and researchers that explored ISO functioning

___________________

² The committee uses the word “risk” to indicate serious, untoward health consequences and avoids the word “safety” because many infer from it a complete lack of risks or a balance of risks and benefits.

and strategies to study and communicate with historically marginalized populations that might be skeptical of vaccination. A targeted public comment sessions addressed criteria for studies of vaccine risks, strategies for improving research, communication practices, and methods to bolster public confidence.

The committee also commissioned confidential key informant interviews with experts in vaccine safety, epidemiology, public health communication, and policy. These yielded qualitative insights and actionable recommendations. In addition, the committee developed case studies analyzing ISO’s handling of specific vaccine risk issues, which informed its final recommendations by illustrating both strengths and gaps in current practices.

Building on this extensive methodological review, public engagement, expert consultation, and case analysis, the committee turned its focus to evaluating the specific systems and practices ISO employed during the COVID pandemic. It did so by examining ISO public scientific reporting, such as presentations at federal advisory committee meetings and scientific publications, and CDC webpages. The following findings highlight how ISO’s key monitoring and evaluation platforms functioned in practice, the strengths and limitations of its risk assessments, and the communication challenges that emerged in a rapidly evolving public health landscape. Figure S-1 shows the committee’s understanding of ISO’s workflow.

Monitoring and Evaluation Systems

ISO leveraged a suite of systems to monitor vaccine risks, including VAERS, VSD, Clinical Immunization Safety Assessment network (CISA), V-safe, and the COVID-19 Vaccine Pregnancy Registry. These platforms formed a deliberately overlapping architecture—early signal detection, rapid active analysis, clinical assessments, near-real-time self-reporting, and longitudinal follow-up for sensitive populations—so that no single evidence stream bore the entire burden of risk assessment and layered and comprehensive monitoring was possible. Table S-1 presents a quick snapshot of the five CDC-affiliated systems that formed the backbone of COVID vaccine safety monitoring.

VAERS, a joint CDC-FDA passive reporting system, provided early signals, though it was limited by underreporting and lack of control groups. For example, it produced the first U.S. alerts of myocarditis after mRNA vaccination in May 2021, prompting deeper analysis in VSD.

VSD offered rapid assessments and in-depth targeted analysis using electronic health records. Weekly rapid-cycle analyses (RCAs) across roughly 12 million patients quantified the myocarditis incidence at about

datasets, methods, and populations to COVID vaccine safety monitoring. These systems helped triangulate evidence, extended surveillance beyond CDC-covered populations, and enabled early signal identification across diverse geographies and contexts.

Timeliness and completeness of signal evaluation were further strengthened by the Vaccine Safety Technical (VaST) Work Group, which met weekly during the first 18 months of the vaccine rollout and monthly thereafter. Its rapid review—often within days of the first doses administered—enabled ACIP and CDC to issue interim clinical guidance on anaphylaxis, myocarditis, TTS, pregnancy safety, and other adverse events of special interest. This prespecified outcome list gave the program an analytically focused backbone, but it remained flexible: unexpected conditions, such as menstrual changes or tinnitus, were added as public concern arose. Where data were sparse, VaST and ISO triaged resources—a quick descriptive VAERS analysis for early context while labor-intensive VSD chart reviews ran in parallel. These efforts were facilitated by temporary data-use agreements that expanded access to vaccination data and enabled timely linkage to health outcomes. However, without a permanent national reporting infrastructure, fragmentation across state systems and inconsistent standards continued to hinder comprehensive, long-term surveillance.

Timeliness brings unavoidable opportunity costs. Every new investigation diverted epidemiologists, statisticians, and clinician reviewers from other monitoring tasks, yet the criteria for elevating, deferring, or ceasing work on a specific signal were not fully explained publicly. Greater transparency around those prioritization decisions and the resource implication across safety systems would bolster confidence in ISO’s capacity to target effort where it yields greater public health return.

Despite the robustness and thoroughness of ISO’s activities, challenges remained, particularly regarding clear communication of findings to health professionals and the public. Translating RCA statistical output into plain language and actionable guidance proved difficult: weekly myocarditis risk estimates, for example, were posted in ACIP slide decks long before they appeared in peer-reviewed journals, leaving frontline clinicians to interpret evolving numbers without clear context or caveats. Communication issues occasionally undermined the extensive scientific efforts undertaken to assess vaccine risks comprehensively. Additionally, ISO’s scientific publications and communications from CDC consistently included strong messages about vaccine benefits, fueling perceptions of bias.

The COVID pandemic underscored ISO’s critical role in vaccine risk monitoring and evaluation, demonstrating the importance of integrating multiple complementary monitoring systems to rapidly and accurately assess vaccine risks. This integrated approach not only facilitated immediate public health responses but also established a strong foundation for

future vaccine risk monitoring. Sustaining this level of performance will require durable governance structures and improved coordination across federal agencies. While ISO partnered effectively with organizations like the National Institutes of Health, CMS, and FDA, the lack of centralized coordination with other state and local agencies limited alignment of analytic priorities and data integration. Enhancing transparency, clearly distinguishing ISO’s role from efforts to encourage vaccination, and continuously refining methodologies will be essential to maintain public trust and effectively address emerging vaccine risk challenges.

Communication Challenges and Public Trust

CDC has been a pivotal institution in addressing public health crises, with vaccine effectiveness and risk communication becoming particularly crucial during the unprecedented PHE; it developed and implemented comprehensive strategies to communicate clearly and effectively about the benefits and risks associated with the vaccines. However, the complexity and evolving nature of the pandemic introduced significant challenges, affecting how health professionals, policy makers, and the general public received and acted upon information.

At the onset of the pandemic, CDC recognized the necessity for transparent and timely communication. Despite extensive communication efforts, vaccine hesitancy persisted due to concerns about the novelty of the mRNA technology, FDA’s regulatory approach, uncertainties about potential side effects, trust in authorities, and skepticism about effectiveness.

CDC’s communication strategies were multifaceted, designed to meet the distinct informational needs of various stakeholders, such as health professionals, policy makers, public health officials, and the broader community. CDC employed a variety of tools, including ACIP meetings, Clinician Outreach and Communication Activity calls, Morbidity and Mortality Weekly Reports (MMWRs), and extensive online content. These platforms provided continuous updates and detailed discussions of vaccine risk data, monitoring results, and emerging scientific findings. For instance, ACIP meetings, usually three times a year, increased dramatically to 27 public sessions during the PHE, underscoring the urgency and importance of the vaccine risk discourse.

Despite these robust mechanisms, areas of communication fell short of fully engaging and adequately informing all intended audiences. One primary concern was the overly technical language in many CDC communications. The agency often presumed a baseline understanding of scientific terminology and vaccine risk principles. This posed challenges for not only the general public but also many health professionals, who, despite their backgrounds, may not have specialized expertise in vaccine

risk communication. It limited the accessibility and effectiveness of some of CDC’s critical risk messages. State and local health departments were often better positioned than federal agencies to engage trusted community messengers and respond in real time. Strengthening the complementary roles of CDC and these local entities—and tailoring communications to local needs—can enhance message effectiveness and credibility.

Another significant issue was the inconsistent use of and lack of clear definitions for important terms, like “risk,” “adverse events,” “adverse events of special interest,” “side effects,” and “safety signals.” Without clear, uniform definitions, these terms became sources of potential misunderstanding and confusion, particularly when comparing different vaccine risks. For example, risk estimates were sometimes presented in vague terms, like “higher” or “most frequently,” without precise quantification or context, which could lead to misinterpreting actual risks.

The communications occasionally included broad, generalized statements, such as references to “the most intense safety monitoring in U.S. history,” without adequate supporting details about what precisely made it more rigorous. Such broad statements, without substantive backing, risked undermining public trust by appearing overly promotional rather than factual. Further complicating the situation, structural issues with CDC’s web presence impacted how easily individuals could locate relevant information. The content often lacked clear organization, making it challenging to discern whether it was intended for health professionals, public health officials, or the general public. This structural confusion was compounded by multiple pages with overlapping or redundant content, hindering effective navigation and engagement.

In addressing these shortcomings, CDC could significantly enhance its communications by adopting clearer, more accessible language and structured, consistent terminology. This approach would not oversimplify the information but instead ensure that scientific findings were effectively translated into actionable and understandable messages for all audiences. Detailed explanations of the methods and rationale behind risk monitoring efforts would reinforce credibility and facilitate greater public trust.

Further challenges to effective communication arose from procedural and structural constraints. During the pandemic, the federal clearance process—designed to ensure message consistency—sometimes delayed or constrained the dissemination of risk findings, particularly when research involved external collaborators. These delays, combined with the predominantly unidirectional nature of vaccine safety communications, limited opportunities for timely public engagement and contributed to perceptions of opacity. Integrating more accessible formats—such as interactive explainers, visual dashboards, and moderated forums—alongside traditional outlets, like the MMWR, would improve transparency and support bidirectional communication with both professional and public audiences.

CONCLUSIONS AND RECOMMENDATIONS

During the PHE, ISO staff demonstrated commendable agility by quickly adapting existing systems and developing new tools, such as V-safe. The rapid deployment and management of risk monitoring systems facilitated an unprecedented volume of high-quality, timely data on vaccine risks. The meticulous and transparent presentation of these findings, including at open federal advisory committee meetings, was instrumental in guiding immediate public health responses. However, ISO’s ability to respond was also constrained by structural limitations in the U.S. surveillance ecosystem, many of which fell outside its direct purview. Fragmented data systems, lack of real-time data exchange, and inconsistent technical standards impeded comprehensive analysis. Additionally, the absence of transparent criteria for prioritizing safety signals meant that emerging public concerns—such as menstrual changes—often outpaced analytic response. Addressing these gaps will require a clear framework for signal prioritization and investment in interoperable infrastructure that can support timely, cross-system risk detection.

However, despite ISO’s robust scientific processes, the office faces ongoing challenges, particularly concerning public trust and understanding. These challenges often arise from the perceived overlap between vaccine risk monitoring activities and broader vaccination efforts at CDC, which some critics argue undermines ISO’s perceived impartiality. This perception can lead to skepticism among segments of the public, complicating efforts to communicate scientifically grounded vaccine risk assessments effectively.

Conclusions

Based on its comprehensive evaluation—including analysis of monitoring and evaluation systems, stakeholder input, expert interviews, and communication products and practices—the committee concluded that ISO carried out scientifically robust, timely, and effective vaccine risk monitoring throughout the COVID pandemic. Its proactive identification and response to emerging vaccine risk concerns informed critical public health decisions. The following conclusions summarize key strengths.

Despite these accomplishments, the committee identified ongoing challenges related to public trust, transparency, and the perception of ISO’s independence. The perceived overlap between ISO’s risk monitoring activities and CDC’s broader efforts to encourage vaccine use continues to raise concerns about the impartiality and objectivity of risk assessments, negatively affecting public confidence in and use of vaccines.

Recommendations

The committee does not want its focus on the risks of COVID vaccines to detract from the overwhelming evidence of their benefits. One way to increase use of vaccines, and therefore the benefits, is to increase understanding of and confidence in federal studies of vaccine risks. Recognizing the significance of ISO’s role, the committee proposed critical recommendations aimed at enhancing its capabilities and supporting its capacity.³ These focus on five core principles crucial for a robust vaccine risk monitoring system that merits the nation’s trust: relevance, credibility, data stewardship, continuous improvement, and independence.

Relevance: Vaccine risk monitoring, evaluation, and communications activities meaningfully address the needs of health professionals, policy makers, and the public.

___________________

³ During the committee’s deliberations, major changes were announced in the structure of CDC and its parent Department of Health and Human Services. To avoid possible confusion, the committee refers to ISO when describing the activities undertaken by that office before these reorganizations but to a federal vaccine risk monitoring and evaluation office for time periods after the start of those reorganizations.

Relevance of vaccine risk monitoring and evaluation is dependent on a comprehensive strategic plan and clearly articulated mission. This plan ought to include clearly defined objectives and action steps, be publicly accessible, and incorporate meaningful input from diverse stakeholders—including health professionals, policy makers, and the public. Establishing a board of scientific counselors with deep scientific and communications expertise would enhance accountability, guide the strategic planning process, provide research advice, help ensure the independence of the vaccine risk monitoring and evaluation office, and support ongoing public engagement. Transparent communication of the scientific agenda and regular progress updates will help ensure that vaccine risk monitoring and evaluation activities remain aligned with public health priorities and needs and expectations of health professionals and the public.

Credibility: Health professionals, policy makers, and the public can rely on scientifically sound information and data analysis about vaccine risks.

Credibility hinges on communicating vaccine risk information clearly, transparently, and comprehensively. This includes maintaining a clear distinction between vaccine risk evaluation and activities designed to increase use of vaccines to avoid perceived biases and conflicts of interest. To reinforce public trust, the office ought to regularly publish detailed explanations of its monitoring and evaluation systems, research methodologies, strengths, limitations, and key findings. Making these materials—including research protocols—publicly available, along with accessible, plain-language summaries and standardized risk reporting tailored for nonexpert audiences, will enhance transparency and significantly improve public understanding. The risk monitoring and evaluation experts ought to be consulted for technical accuracy, but other offices in CDC or HHS can use the risk information in policy determinations and communications.

Data Stewardship: Vaccine risk monitoring, evaluation, and communication activities are conducted with respect for the individuals whose data are used by protecting their privacy, using their data properly to address important questions about risks, and sharing the results.

Strong data stewardship is essential, requiring the ethical and secure handling of individual data. Rigorous data protection practices are necessary to safeguard participant privacy and ensure data confidentiality. Equally important is transparent communication about how data are collected and used and why. To promote scientific integrity and public trust, the office can

work with its board of scientific counselors to explore secure, innovative methods for enabling controlled access to datasets by external researchers. This would support broader validation of findings, enhance transparency, and foster greater public confidence in vaccine risk monitoring.

Continuous Improvement and Innovation: Regular evaluation of vaccine risk monitoring, evaluation, and communication practices leads to adopting new methodologies and technologies with the capacity to address emerging questions about vaccine risks

Continuous improvement and innovation are integral to the office’s operational and strategic planning. This includes setting clear, measurable metrics for regular evaluation and systematically adopting advancements in vaccinology, epidemiology, statistical analysis, informatics, and communication practices. Regularly updating monitoring and evaluation systems and incorporating new technologies will help ensure that vaccine risk monitoring and evaluation remain responsive to evolving public health challenges. Additionally, investing in communication research will enhance the clarity, effectiveness, and impact of public messaging, strengthening audience engagement and understanding.

Independence: Vaccine risk monitoring, evaluation, and communication are free from undue internal or external influence.

Scientific independence is vital to the perceived impartiality, credibility, and effectiveness of vaccine risk data evaluation, communication, and contributions to practice. To protect its scientific integrity, the vaccine risk monitoring and evaluation office ought to remain organizationally and administratively separated from CDC units involved in vaccine promotion or policymaking. A distinct and clearly identifiable web presence for vaccine risk monitoring and evaluation and timely and unhindered publication of research findings—free from administrative interference—are essential for transparent, unbiased communication, and the delivery of scientifically rigorous information to the public. Scientific independence does not preclude consultation and collaboration with other CDC and government experts, but final decision making about monitoring and evaluation of vaccine risk information and the scientific content of risk communications needs to remain with this office.

Final Thoughts

Implementing these five recommendations would enhance the delivery of timely, trustworthy, and scientifically robust vaccine risk information and will require resources and the commitment of the CDC director. This, in turn, supports increased public confidence in vaccination programs, ultimately contributing to stronger public health outcomes and preparedness for future public health emergencies.