Rapid Expert Consultation

MAY 2022

__________________

Copyright 2022 by the National Academy of Sciences. All rights reserved.

Executive Summary

This rapid expert consultation was produced by individual members of the Standing Committee for CDC Center for Preparedness and Response (SCPR). Its aim is to review and propose modifications to an initial draft list of critical federal capabilities presented by the Centers for Disease Control and Prevention (CDC) and the U.S. Food and Drug Administration (FDA) that are needed to evaluate real-world safety, effectiveness, equitable distribution, access, and use of medical countermeasures (MCMs) during a public health emergency (PHE). This effort draws from expert input, published literature, and lessons learned from previous public health emergencies, as well as the ongoing COVID-19 pandemic.

Authors of this rapid expert consultation offer a revised framing per a concept of operations (CONOPS) approach. Under this newly proposed structure, evidence generation, communication, coordination and partnerships, and legal analyses capabilities are aligned to deliver on an overarching goal to monitor and assess the safety, effectiveness, and equitable distribution and use of MCMs during a PHE, so that policy makers can make more rapid and effective decisions, communicate more accurate information to the public, and build or sustain community relationships and trust. Capabilities are further achieved through the implementation of policies and tools, systems, and resources. Underpinning the framing are a set of core considerations, which are equity, ethics, integration/interoperability, adaptability, scalability, efficiency, reusability/generalizability, and quality. This rapid expert consultation does not include findings, conclusions, recommendations, or policy advice.

Introduction

The COVID-19 pandemic has most recently highlighted the importance of having access to real-world data (RWD) and real-world evidence (RWE) to monitor and assess MCMs use and performance so policy makers can make more effective and rapid public health decisions, protect population health, and save lives. During PHEs, the use of MCMs, such as therapeutics, vaccines, and diagnostics, can be made available to the public under a range of regulatory access mechanisms, including FDA’s Emergency Use Authorizations (EUAs).¹³

MCMs deployed under regulatory access mechanisms, like EUAs in PHEs, must generally show some evidence of safety and efficacy under pre-market, well-controlled clinical trials (i.e., randomized controlled trials for medical products), where available. However, the level of evidence required for issuing an EUA is not equivalent to that required for FDA approval. During the COVID-19 pandemic, an EUA was provided for the use of convalescent plasma to treat patients with COVID-19 that was later revised as evidence accrued demonstrating lack of benefit at lower antibody titers (FDA, 2021). EUAs are also subject to political pressure, as was the case with the EUA provided to hydro chloroquine, which was later revoked when this drug demonstrated lack of benefit and even potential harm (Zhai et al., 2020). Additionally, medical products made available through EUAs may not reach

__________________

certain populations, as observed with the administration of monoclonal antibodies that was found to favor White and non-Hispanic patients over Black, Asian, Hispanic, and other race patients in the treatment of COVID-19 (Wiltz et al., 2022). Therefore, it is necessary to monitor the safety and performance of MCMs against real-world conditions over time, against a potentially evolving threat, across different populations and contexts, and to be able to rapidly make evidence-based decisions based on the RWE available.

At the request of CDC’s Center for Preparedness and Response, the National Academies of Sciences, Engineering, and Medicine (the National Academies) were tasked with producing a rapid expert consultation that reviews an initial, draft list of critical federal capabilities¹⁴ needed to evaluate real-world safety and efficacy of MCMs during a PHE. The full Statement of Task states:

To carry out the rapid expert consultation, the National Academies convened a group of subject-matter experts primarily from the SCPR with expertise in medical and PHE preparedness and response; evaluating MCMs with RWE; regulatory policy; evaluating equity in RWE, clinical trials, and research; and data systems and informatics to review the initial draft list of capabilities.

This rapid expert consultation focuses specifically on federal capabilities needed for RWE to monitor and assess MCMs during a PHE. However, the authors recognize that federal capabilities to capture RWD and capabilities to generate and use RWE are needed across the entirety of the MCM enterprise (not just the monitoring and assessment)—from product development to “last-mile” outcomes—to inform decision making. Additionally, the use of RWE in non-emergency times and for other types of medical products beyond MCMs is invaluable for monitoring post-market safety, efficacy, and use, monitoring adverse events, and for regulatory decision making. These considerations are the focus of FDA’s Real-World Evidence Program (FDA, 2022b). Although this rapid expert consultation is limited in scope, the discussions are applicable to the broader field of RWE, and future work could explore building RWE capabilities across the entirety of the MCM enterprise.

__________________

Context for This Rapid Expert Consultation

Robust and reliable RWE is valuable for key groups involved in a PHE response to effectively monitor MCMs’ distribution, uptake, and performance. RWE is commonly defined as “evidence [based] on the benefits and harms of treatments derived from RWD, routinely collected data relating to patient health status and/or delivery of health care” (Franklin et al., 2021). That is, RWE is based on how medical products are used and perform in real-life scenarios rather than in controlled and designed experimental trials. RWE is typically characterized by three key dimensions (Simon et al., 2022):

1. Reliance on the use of RWD, such as data collected by health care systems or consumer health monitoring devices, in order to generate RWE;
2. Synthesizing findings from studies that involve monitoring and assessing the use of treatments, vaccines, and devices in real-world settings; and
3. Understanding that studies producing RWD may depart from traditional research design elements, often considered as the gold standard for evidence generation, such as individual-level randomization, blinding, etc.

The role of RWE to guide and enhance decision making regarding medical product development has been examined previously across sectors, including by the National Academies, in efforts to examine the impact of RWE on medical product development (NASEM, 2017c, 2018a,b, 2019). The capabilities needed to monitor and assess MCM use during a PHE has also been a focus in past National Academies’ work (NASEM, 2017a). Box 1 provides a summary of past work examining RWE carried out by the National Academies.

The COVID-19 pandemic brought into sharp focus the need for more robust systems to generate and manage RWD and to support the development and use of RWE from it. Unlike other types of PHEs, the ongoing pandemic exemplifies the need for networks and systems that can adapt to the requirements of a large-scale, geographically dispersed PHE response and can capture metrics on newly developed countermeasures to combat a novel pathogen. Considerations for data systems and assessing data for decision making, excerpted from prior rapid expert consultations developed during the pandemic, are summarized in Box 2 (NASEM, 2020a,b). Additionally, national clinical trial networks and infrastructure, such as those utilized for the United Kingdom’s RECOVERY trial, supported RWE generation and enabled rapid identification of effective treatments against COVID-19 (UK Research and Innovation, 2022). Recommendations made to U.S. government agencies in a consensus study following the 2014 Ebola epidemic also emphasized the need to build clinical trial research capacities, systems, and networks during the epidemic and inter-epidemic period (NASEM, 2017b).

Evaluation of Initial Draft Framework

A draft list of key federal capabilities to monitor and assess MCMs was provided to the authors of this rapid expert consultation for their review (CDC, 2022b) (see Appendix A). The list was categorized by federal capabilities in monitoring and assessment effectively used during the COVID-19 response, monitoring and assessment capabilities to be developed pre-incident, and monitoring and assessment capabilities needed during the incident.

The authors were then asked to respond to two main discussion questions for the consultation:

1. At a high level, are these the key capabilities that are needed for monitoring and assessing MCM safety, effectiveness, and use? Are there any additional ones?
2. Should all these capabilities be built and maintained at the federal level (versus in the private sector)?

The authors’ review of the proposed federal capabilities was executed through a series of three meetings held in March 2022. Comments on a specific capability or set of capabilities put forth in this document are based on expert opinion for improving the framework. Furthermore, this document does not provide specific consensus recommendations, findings, or conclusions from the National Academies.

In reflecting on the categories of capabilities presented in this initial framework, the authors recognize that the pre-incident, during incident, and post-incident groupings are not mutually exclusive categories. In other words, capabilities built pre-incident would need to be leveraged during and after an incident, and capabilities used during and after an incident should be identified, pre-positioned, and strengthened prior to an incident. Additional opportunities to improve on this initial framework include:

• Defining the term “capability” – Capabilities describing a specific ability, and achieved through a collection of federal government, or nongovernment entities’ skills, expertise, policy tools, agreements, systems, and resources, need to be explicitly defined. The authors propose a definition for capability in this consultation; however, not all proposed categories of preparedness and response work would meet a strict definition of “capability.”
• Mapping capabilities to a clear end or desired state – Grounding the capabilities to a strategic goal with specific, measurable, and achievable objectives would bring greater clarity to the rationale and purpose of the framework and each capability.
• Understanding current efforts under way or the current state – Establishing a comprehensive understanding of the current state of federal capabilities for evaluating RWE of MCMs that have been developed and were successful, that have been developed and did not work, are in the process of being developed, and those yet to be developed would assist in articulating a future state in context of the current state of federal capabilities.
• Achieving a balanced focus on each MCM type – While capabilities may apply broadly to multiple types of MCMs, sufficient detail of specific considerations tailored to each type of MCM is needed to facilitate a complete understanding through monitoring and assessment, due to differing standards for evidence generation depending on the specific MCM.
• Integrating communication, community engagement, public trust, and equity considerations as foundational elements – Capabilities to promote, rebuild, and sustain public trust, equity, community engagement, and communication could be integrated across the RWE system. Additionally, these considerations can be informed by qualitative and quantitative data, including sociodemographic data, collected through community-engaged approaches and with partners who are trusted in the community.
• Including equitable distribution, administration, and use considerations in the development and assessment of RWE – Capabilities clearly mapped to the entirety of the MCM system, from development to post-development distribution, access, administration, and use, would enable a more comprehensive assessment of critical system components. The prompt and equitable distribution and use of MCMs, in addition to safety and effectiveness, are critical components to monitoring and assessing MCMs.
• Defining roles and responsibilities – Capabilities acknowledging the roles, responsibilities, and expertise of specific federal agencies; existing or anticipated interagency bodies; industry; academic; health care; community; state, tribal, local, and territorial (STLT) partners; and other key groups would promote action by a responsible authority. It is clear from the initial, draft list of capabilities that many of these were not just CDC or FDA capabilities.
• Aligning capabilities with key legal authorities – Capabilities aligned to what the federal government and key partners are legally allowed to do during a PHE would bring clarity to opportunities and constraints that may impact MCM monitoring and assessment.

Modifications to the Initial Framework of Federal Capabilities Needed to Evaluate Real-World Safety, Effectiveness, and Equitable Distribution and Use of MCMs

A systematic approach, such as a CONOPS approach, would be more suitable to identify, define, and organize key federal capabilities in a more strategic and comprehensive framework. A CONOPS approach was discussed during the 2017 National Academies workshop Building a National Capability to Monitor and Assess Medical Countermeasure Use During a Public Health Emergency as a way to integrate a national capability to monitor and assess MCM use with existing threat response capabilities (NASEM, 2017a). A CONOPS approach provides a process by which a set of capabilities can be described to achieve envisioned outcomes, presents a summary of the process to be followed in a narrative form, with clearly defined roles for key groups involved in the PHE, and a methodology to reach stated goals and objectives (NASEM, 2017a). Additional items included in a CONOPS are presented in Box 3.

In taking a CONOPS approach, the authors offer an organizing structure of an overarching goal, capabilities to achieve that goal, and core considerations as part of the framework of federal capabilities needed to evaluate real-world safety, effectiveness, and equitable distribution and use of MCM during a PHE (see Figure 1). The elements of this organizing frame are described in detail below, in addition to further steps that would need to be undertaken in order to develop a CONOPS for assessing and monitoring MCMs.

GOAL

A statement of goals and objectives provides a description of a desirable future or end state. An example of a goal statement for the purpose of monitoring and assessing MCMs could be:

This goal statement encompasses MCM distribution and use, because equity and other monitoring and assessment parameters are dependent on knowing where the MCM is in the system, and who specifically gets and ultimately uses the MCM. In addition to this goal statement, specific, measurable, time bound, inclusive, and equitable objectives that relate to each capability would enable monitoring progress toward achieving each capability (The Management Center, 2021).

Objectives could also be separately developed (if necessary) for short-term, medium-term, and long-term priorities to ensure that capabilities are in place to be implemented in time for the next PHE.

CAPABILITIES

The capabilities offered by the authors to successfully achieve the goal are evidence generation, communication, coordination and partnerships, and legal analyses.

• Evidence Generation – The ability to obtain, analyze, and translate multiple data types, from multiple sources in real-time to inform decision making. Equally important is the ability to collect, analyze, and translate qualitative data (interviews, focus groups, community forums,

social media, and citizen science apps, etc.). Adopting a hybrid approach to evidence generation by combining native RWD (e.g., quantitative data generated during clinical care and not intended for research) and data collected specifically for purposes of supporting RWD (e.g., supplemental qualitative or survey data), enhances the value of RWE. Evidence generation comprises:

• Data Collection – The ability to collect, collate, and share valid and reliable information from multiple data sources.
• Data Analysis – The ability to synthesize and analyze information in real time to inform decision making as a threat evolves.
• Data Use and Dissemination – The ability to make decisions and implement public health action with the information at hand (e.g., actionability), and to communicate those decisions effectively (e.g., communication).
• Communication – The ability to effectively deliver, receive, and exchange culturally appropriate, locally relevant, timely, accurate, and actionable information to and from community, STLT, federal, and private-sector partners. Effective communication is critical to promoting public understanding of the PHE response and to address issues of misinformation and disinformation.
• Coordination and Partnerships – The ability to collaborate and align different skills, resource needs, and perspectives across federal, STLT, community leaders, and industry counterparts to effectively generate RWE across disparate systems, share requirements and information or data (as appropriate), and interpret and communicate results with other collaborators. This capability also includes the ability to engage in shared decision making.
• Legal Analyses – The ability to identify real-time legal issues; address legal obstacles (both real and perceived) and opportunities; analyze and address ethical, legal, and social implications; and generate legal solutions in communication, partnership coordination and collaboration, data collection, data analysis, and data use and dissemination arising from shifting legal landscapes prior to and during emergency declarations. This also includes the ability to develop or strengthen the legal and policy framework that supports the aforementioned work.

Each capability consists of the ability to implement policies and tools (e.g., policies, regulatory and legal tools, guidance, protocols), systems (e.g., processes, digital platforms and databases, programs), and resources (e.g., funding, workforce, materials). Further sub-categorization could be warranted. For example, one could further organize by type of MCM, event phase, or function required to generate RWE.

CORE CONSIDERATIONS

Adhering to a set of core considerations helps to ensure successful implementation of the capabilities described in the previous section. The following considerations are of equal importance and should be exercised in non-emergency times, as well as during a PHE.

Equity

The goal is that individuals have equitable access to MCMs with consideration given to medical need and differences in social determinants of health that exist among regions, populations, and specific socioeconomic or demographic groups. Equity includes having insight and familiarity into relevant

unique local contexts and how these may inform or modify a public health response. Some groups may need to have earlier access to and a greater intensity of resources made available to them to achieve equity, if a group is disproportionately impacted.

Ethics

Equity is a critical ethical value, but not the only value at stake when generating and using RWE on MCM. For example, maximizing benefits to end-users while minimizing adverse health impacts on populations is at the core of ensuring the safety and, effectiveness, and equitable distribution and use of MCMs. Additionally, maintaining trust, fairness, transparency, accountability, and responsible stewardship of limited resources support principles of public health ethics (NASEM, 2021). A number of additional ethical principles or issues can also come into play when collecting and using RWE to inform policy and practice—whether during a PHE or in peacetime—such as reciprocity, proportionality, privacy, personal liberty, and more. The ethical complexity of RWE generation and use argues for ongoing proactive consideration of ethical issues throughout the process.

Integration and Interoperability

Building on existing terminology, information systems, application programming interface (API), and health information exchange standards for interoperability, the system can integrate and exchange the multiple sources and types of data (e.g., genomic; consumer- and patient-level quantitative and qualitative data; surveillance- and population-level data; place-based geocoded data; program data, such as manufacture, supply chain, allocation data, claims data, death records, survey data, sociodemographic data, wastewater surveillance, and qualitative community assessments) between different federal agencies as well as between federal agencies and STLT, community, and industry counterparts. Supplementing RWD with other data that are not routinely collected by health systems also enhances the value of RWE. To utilize qualitative data (e.g., interviews, focus groups, community forums, social media and citizen science apps, etc.)—which are more difficult to acquire, but at the same time necessary to tailor communication efforts and ensure equity, particularly for locally marginalized groups—partnerships and data-sharing systems could be established and maintained with university researchers, community-based research networks (e.g., National Institutes of Health’s RADx-Underserved Populations Consortium¹⁵), community-based health systems, and other groups that routinely capture this type of data.

Adaptability

Systems need to be flexible to accommodate a variety of data types (e.g., structured quantitative data, unstructured qualitative text, sound, motion, networks, etc.) through a variety of data sources (e.g., electronic health records [EHRs], consumer apps, waste water, environmental sensors, supply chain data, etc.). Systems must also be able to accommodate for newly identified data sources and data types.

Scalability

Given the nature of RWE, obtaining meaningful signals from multi-modal data requires curation, mapping, and making the components and elements needed for RWE generation, analysis, and dissemination ready for scale. Technology needs to grow to meet demand and accommodate both

__________________

data synthesis and visualization tasks, such as user dashboards and reports, as well as high-performance computing for linking complex genomic workflows, and other high-volume, high-dimensionality data, with outcomes.

Efficiency

Operational program data are valuable but often overlooked as a RWE source. As a best practice for efficiency, operational data should be transformed into an analytic structure that is optimized for query and analysis. Efficiency is also improved by limiting the number of required data elements and leveraging existing clinical systems, infrastructure, and networks from past studies (NASEM, 2017c).

Reusability and Generalizability

To facilitate rapid response and avoid duplicative investments, frameworks and analytics can be developed prior to the next event and be used for routine health challenges. Moreover, geocoded data about disease incidence, burden of illness, vulnerable populations, mortality, morbidity, and socio-demographic factors, at minimum, could be made easily available to decision makers to facilitate the process of defining equitable allocation metrics for MCMs as well to facilitate the development of relevant communication strategies (NASEM, 2020c).

Quality

Data gathered under real-world conditions can present inherent challenges to generating RWE. Attention to methodological rigor, information reliability, validity, use of comparator or control groups, and bias is needed at all stages of evidence generation, use, and dissemination. Additional steps that can be undertaken to improve RWE quality are to conduct regular assessments of data quality and risk of bias, to triangulate evidence across multiple sources, to characterize uncertainty, and to generate evidence in a transparent manner (NICE, 2022).

Preliminary Gap Analysis of the Necessary Capabilities

Using the structured CONOPs approach outlined above, the authors attempted to streamline, edit for clarity, and categorize the initial, draft list of capabilities provided by CDC and FDA. This allowed the authors to begin to identify gaps where federal capabilities are needed. In some cases, the authors proposed ideas to fill these gaps but acknowledge that a more thorough gap analysis is necessary. Additionally, the authors determined that some of the initial, draft capabilities were relevant to product development not monitoring and assessment (e.g., develop partnerships with diagnostic developers to support rapid development, authorization of diagnostics for clinical use during PHEs, pre-identified lead for initial diagnostic reagent production, ability to scale production up or down included in federal contracts, and clarity on the use of the Stafford Act for development and validation of diagnostics) and should be classified as such. Capabilities relevant to development were excluded from the authors’ analysis below. This work was then visually organized into a table/matrix for ease of explanation (see Table 1).

TABLE 1 Summary Matrix of Proposed Policies and Tools, Systems, and Resources to Achieve the Necessary Federal Capabilities to Evaluate Medical Countermeasures During a Public Health Emergency

NOTE: CBO = community-based organization; CEAL = Community Engagement Alliance¹⁶; DMI = Data Modernization Initiative¹⁷; EHR = electronic health record; EUA = Emergency Use Authorization; MCM = medical countermeasure; MOU = memorandum of understanding; NGO = nongovernmental organization; NIH = National Institutes of Health; RADx-UP = Rapid Acceleration of Diagnostics for Underserved Populations; RWD = real-world data; RWE = real-world evidence; STLT = state, tribal, local, or territorial public health agency: USCDI = United States Core Data for Interoperability.

__________________

EVIDENCE GENERATION CAPABILITY

A majority of the capabilities initially proposed by CDC fell under the evidence generation capability, and specifically under data collection. There were gaps in capabilities related to data acquisition, analysis and translation, and gaps in terms of articulating the different types of RWD (quantitative and qualitative) necessary to informing the safety, effectiveness, and equitable distribution and use of MCMs. Box 4 includes relevant examples of policies and tools, systems, and resources to achieve evidence generation capabilities, as proposed by CDC (note these have been streamlined and edited for clarity) as well as examples proposed by the authors (in italic).

__________________

COMMUNICATION CAPABILITY

The initial draft list did not include capabilities that addressed the ability to effectively deliver information to and receive information from community, STLT, federal, industry, academic, and health care partners. Such capabilities are critical in order to communicate key findings regarding the safety, effectiveness, and distribution and use of MCMs. Examples of policies and tools, systems, and resources to achieve a communication capability proposed by the authors are included in italic in Box 5.

COORDINATION AND PARTNERSHIPS CAPABILITY

In the initial draft, CDC proposed establishing an interagency coordinating body across key partners and clearly defining roles and responsibilities for monitoring and assessing MCMs among them. A coordination body (while not technically a capability) and a clear decision-making process that involves stakeholders from groups most (likely to be) impacted are critical to support capabilities for monitoring and assessing MCM during a PHE. The authors also proposed additional examples of policies and tools, systems, and resources (in italic) to achieve a coordination and partnerships capability in Box 6.

LEGAL ANALYSIS CAPABILITY

Generating MCMs essential to national security and preparedness depends on numerous laws governing their development, production, assessment, and approval (NASEM, 2021). While the draft list omitted a capability for legal analyses, MCMs are utilized in response to extensive public health threats (e.g., emerging infectious diseases, nuclear exposures, natural disasters), and that implicates varying federal and STLT laws and policies. As described in the 2021 report Ensuring an Effective Public Health Emergency Medical Countermeasures Enterprise, there are numerous legal options for evaluating the real-world safety, effectiveness, and distribution and use of MCMs (see Box 7) (NASEM, 2021). These examples are offered by authors of this rapid expert consultation in italic.

Concluding Remarks

For policy makers to make the best decisions rapidly during PHEs to protect the health and lives of the people they serve, it is critical to have access to reliable, standardized, integrated, and interoperable RWD to monitor and assess the use and performance of MCMs deployed. In order to support the generation and interpretation of RWE, information should be shared as soon as it is available. This rapid expert consultation discussed potential federal capabilities in context of the core considerations of equity, ethics, interoperability, adaptability, scalability, efficiency, generalizability, and quality. Furthermore, it also discussed how important it is that these capabilities are established, tested, pre-positioned, and sustained prior to any large PHE. Finally, the rapid expert consultation highlighted the opportunity to develop an interagency coordinating body across key partners with clearly defined roles and responsibilities for monitoring and assessing MCMs during a PHE.

The ideas generated in this rapid expert consultation should be refined through a more careful and thoughtful analysis. The strengthening and pre-positioning of existing federal capabilities and the establishment of new federal capabilities to ensure robust data collection, analysis, translation, and communication to key stakeholders in the health care system are critical for the response to the ongoing COVID-19 pandemic and to future large-scale PHEs.

REFERENCES

APPENDIX A

Federal Capabilities Needed to Monitor/Assess Medical Countermeasures (MCMs) Used in Public Health Emergencies – An Initial Framework

SOURCE: Excerpted from the presentation “Federal Capabilities Needed to Monitor/Assess MCMs Used in Public Health Emergencies – An Initial Framework” made available by U.S. Centers for Disease Control and Prevention (CDC) to the National Academies on February 9, 2022 (https://www.nationalacademies.org/event/03-02-2022/meeting-to-discuss-rapid-expert-consultation-on-critical-federal-capabilities-needed-to-evaluate-real-world-safety-and-efficacy-of-medical-countermeasures)

Federal Capabilities in Monitoring and Assessment Effectively Used During COVID-19 Response

• U.S. Food and Drug Administration (FDA) shared resources on the COVID-19 Evidence Accelerator, which connects various real-world data streams to evaluate effectiveness of treatments (https://www.evidenceaccelerator.org)
• FDA Real-World Evidence Program (https://www.fda.gov/media/120060/download)
• Harmonization of vaccine development protocols and FDA guidance on development to ensure standardization of data collected during development (to inform monitoring/assessment after deployment)
• Early engagement of industry partners to allow for vigorous coordination
• National Institutes of Health Rapid Acceleration of Diagnostics comparative evaluation of diagnostics
• CDC Vaccine Adverse Event Reporting System (VAERS)
• CDC v-safe After Vaccination Health Checker / v-safe COVID-19 Vaccine Pregnancy Registry
• CDC Vaccine Safety Datalink (VSD)

Monitoring and Assessment Capabilities to Be Developed Pre-Incident

General

• Federal MCM development contracts should include (perhaps with a public health trigger):
  • Data use agreements (DUAs)
  • Data sharing agreements
• Definition of leads and roles and responsibilities among key stakeholders (e.g., FDA, manufacturers, Biomedical Advanced Research and Development Authority, CDC)
• Ability to scale up or down as needed should be included in federal contracts

Diagnostic development and assessment capabilities

• Pre-identified department and agency lead(s) for initial diagnostic reagent production
• Pre-positioned Material Transfer Agreements/DUAs/contract vehicles:
  • Processes for getting reference materials to private manufacturers (e.g., viral stocks, reference panels) to support scale up of diagnostics
  • Requirements for manufacturers to provide (at cost) tests for comparative U.S. government evaluation
  • Processes for receiving diagnostics at U.S. government labs for comparative evaluation
  • Agreements to consult with federal authorities regarding manufacturing reduction decisions
• Pre-developed specificity panels for diagnostics
• Pre-positioned protocols, pre-identified clinical trial networks, and related infrastructure (including information technology systems, experts, encryption protocols) to support federal comparative evaluation in real-world sensitivity and specificity over time
• Clarity on the opportunities and limitations around use of Stafford Act funding during an emergency for development and validation of diagnostics

• Charging and standing up an interagency coordinating body
• Comprised of departments and agencies with monitoring and assessment pre-event planning and exercising roles and responsibilities as well as responsibilities during and after responses
• Work to develop plans and systems to support standardization, integration and interoperability of data streams and sources resulting from pharmaceutical MCM and diagnostic use during an emergency to inform data-driven decision making
• Work with jurisdictions to identify minimum acceptable data sets and common variables that all jurisdictions will report on MCM performance (to inform safety and efficacy and sensitivity and specificity evaluations)
• Identify and/or determine the parameters for new system(s) that would be used for aggregation of this data in an emergency
• Develop MOUs across departments and agencies detailing the system(s) to be used and how the data would be integrated and shared during an emergency

Monitoring and Assessment Capabilities Needed During the Incident

Data standards and strategic guidance for data collection and reporting

• Leveraging pre-determined minimum data elements and systems agreed upon pre-incident
• Ongoing MCM performance over time as threat evolves
• Standards and guidance to ensure ongoing clinical trial data (e.g., for MCM use in additional populations, need for boosters, continued efficacy against variants) is being provided to federal data systems to inform monitoring/assessment work.
  • Data transferred should be cleaned and vetted
• Mechanisms to leverage data from use of products overseas

Additional capabilities

• Lab and clinical study capacity (appropriately resourced) to provide comparative and ongoing comparative evaluation of therapeutics/diagnostic real-world performance
• Ability to rapidly develop reference panels and specificity panels and make these widely available to support ongoing validation of diagnostic real-world performance
• Ability to collect and make available pooled patient samples (e.g., as raw materials for diagnostic panels and validation; challenge materials for ongoing MCM evaluation)

Questions Considered for Each Pharmaceutical Medical Countermeasure

• What organization(s) were involved in collecting safety and efficacy data during the COVID-19 response for this MCM? Did roles and responsibilities evolve over time?
• What type of data was needed to answer specific question (e.g., safety, effectiveness, health outcomes) under a variety of scenarios?
• How were the existing clinical networks (e.g., clinical trial networks, study and research networks) leveraged for the monitoring and assessment of this MCM? What was effective and what gaps and needs were identified? Representativeness of the clinical trial population.
• For this MCM, how was existing surveillance infrastructure (or new methods where necessary) used to collect quality safety and effectiveness data?
• How was monitoring and assessment data used in real time during the COVID-19 response to provide transparency to the public around the use of this MCM? What worked and what did not?
• What legal or administrative issues had to be addressed in using health care data? Could any of these be addressed ahead of time in advance of future emergencies?
• Was a passive surveillance system used to collect data on this MCM? What were the pros, cons, and effective use cases for the various systems used to collect safety and efficacy data (e.g., U.S. government–managed systems, sponsor-managed systems, manufacturer-provided product and safety data outside the United States)?

• What capabilities were used to manage and analyze the data in a coordinated way to inform policy making and clinical decisions at all levels for this MCM? What worked and what did not?
• During an emergency response, it is critical for MCM information to be shared quickly and accurately. How did PHE MCM information flow during the COVID-19 response for this MCM? Did all stakeholders get the information when they needed it? Why or why not?

Questions Considered for Each Diagnostics Medical Countermeasure

• What organization(s) were involved in collecting sensitivity and specificity data during the COVID19 response for this MCM? Did roles and responsibilities evolve over time?
• What type of data was needed to answer specific question (e.g., clinical sensitivity and specificity) under a variety of scenarios?
• How were the existing clinical networks (e.g., clinical trial networks, study and research networks) leveraged for the monitoring and assessment of this MCM? What was effective and what gaps and needs were identified?
• For this MCM, how was existing surveillance infrastructure (or new methods where necessary) used to collect quality sensitivity and specificity data?
• How was monitoring and assessment data used in real time during the COVID-19 response to provide transparency to the public around the use of this MCM? What worked and what did not?
• What legal or administrative issues had to be addressed in using health care data? Could any of these be addressed ahead of time in advance of future emergencies?
• Was a passive surveillance system used to collect data on this MCM? What were the pros, cons, and effective use cases for the various systems used to collect sensitivity and specificity data (e.g., U.S. government–managed systems, sponsor-managed systems, manufacturer-provided product and data outside the United States)?
• What capabilities were used to manage and analyze the data in a coordinated way to inform policy making and clinical decisions at all levels for this MCM? What worked and what did not?
• During an emergency response, it is critical for MCM information to be shared quickly and accurately. How did PHE MCM information flow during the COVID-19 response for this MCM? Did all stakeholders get the information when they needed it? Why or why not?

ACKNOWLEDGMENTS

We thank the sponsor of the Standing Committee for CDC Center for Preparedness and Response (SCPR)—the Centers for Disease Control and Prevention. Special thanks go to the members of the SCPR committee who dedicated time and thought to this project: Suzanne Bakken, Columbia School of Nursing; Emily K. Brunson, Texas State University; Andrew Cannons, Florida Department of Health; Benjamin Chan, New Hampshire Department of Health and Human Services; Francisco García, Pima County; James G. Hodge, Jr., Arizona State University; Ali Khan, University of Nebraska Medical Center; Nicolette Louissaint, Healthcare Distribution Alliance; Nicole Lurie, Coalition for Epidemic Preparedness Innovations; Jewel Mullen, Dell Medical School, The University of Texas at Austin; Jennifer Nuzzo, Johns Hopkins Bloomberg School of Public Health; and Tener Veenema (Chair), Johns Hopkins Bloomberg School of Public Health.

We extend gratitude to the staff of the National Academies, in particular Lisa Brown, Shalini Singaravelu, Kelsey Babik, Emma Fine, Matthew Masiello, and Margaret McCarthy, who contributed research, editing, and writing assistance.

We are grateful to the Report Review Committee of the National Academies and to the following individuals for their review of this rapid expert consultation: Phyllis Arthur, Biotechnology Innovation Organization; Luciana Borio, ARCH Venture Partners, L.P.; Adrian F. Hernandez, Duke University Medical Center; Boris D. Lushniak, University of Maryland School of Public Health; Daniel Masys, University of Washington; Aletha Maybank, American Medical Association; Deven McGraw, Ciitizen Corporation; and Matthew Wynia, University of Colorado.

Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse this document, nor did they see the final draft before its release. The review of this document was overseen by Linda C. Degutis, Yale University School of Public Health, and Paul A. Volberding, AIDS Research Institute. They were responsible for making certain that an independent examination of this rapid expert consultation was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authors and the National Academies.

This activity was supported by a contract between the National Academy of Sciences and the Centers for Disease Control and Prevention (200-2011-38807/75D30121F00100).