1
Introduction

The mission of the U.S. Environmental Protection Agency (EPA) is to protect human health and the environment. There are thousands of chemicals registered and used in the United States, with identified exposures in the population. Unlike pharmaceuticals, environmental and industrial chemicals are not required to undergo comprehensive testing in animal and human clinical trials prior to being approved for use. Prior National Academies of Sciences, Engineering, and Medicine (NASEM) and EPA reports found that many of these chemicals have little or no data on their potential health effects in the public domain. A key consequence of this data deficit is the limited ability to characterize the human health hazards and risks of many of the chemicals in commerce and in the environment.

For those chemicals for which data are available, the EPA relies on many types of information, including observational epidemiologic studies, experimental animal studies, and investigations into toxic mechanisms, in order to understand their potential to cause human health effects. Historically, the most influential data have come from laboratory mammalian studies, which still form the foundation of most human health assessments. Over the past decade it has become increasingly clear that relying primarily on laboratory mammalian studies limits the ability to assess the human health hazards and potential risk of the tens of thousands of chemicals in commerce to which people may be exposed. Further, there are several other issues with reliance on laboratory mammalian toxicity studies, including their efficiency in terms of costs, timeliness, and animal welfare, as well as concerns as to whether currently available laboratory mammalian-based toxicity testing study designs can adequately identify the spectrum of effects observed in humans. As a result, multiple efforts both in the United States and internationally have sought to develop, evaluate, and apply technologies and tools that simultaneously reduce reliance on animal testing and increase efficiency in terms of time and resources, while providing more comprehensive information on human health effects. In this context, the EPA (2021) has defined the term “new approach methods” (NAMs) to be “any technology, methodology, approach, or combination that can provide information on chemical hazard and risk assessment to avoid the use of animal testing.” For the purposes of the Toxic Substances Control Act (TSCA), the EPA recognizes this new term (i.e., NAMs) as encompassing any “alternative test methods and strategies to reduce, refine, or replace vertebrate animals.”

While insufficient data and studies are available to support the risk evaluations of the vast majority of chemicals in use, and the need for rapid assessment tools is clear, there are many issues that need to be addressed in transitioning away from reliance on laboratory mammalian toxicity studies. Accordingly, the EPA asked NASEM specifically to review the variability and relevance of existing laboratory mammalian toxicity tests for human health risk assessment to inform development of approaches for validation and establishing scientific confidence in NAMs. To that end, NASEM convened the Committee on Variability and Relevance of Current Laboratory Mammalian Toxicity Tests and Expectations for New Approach Methods for Use in Human Health Risk Assessment.

THE COMMITTEE AND ITS TASK

The committee convened in response to the EPA’s request included experts in mechanistic, experimental, and regulatory toxicology; pharmacology; physiologically based pharmacokinetic

modeling; computational modeling; genomics; in vitro and nonmammalian models; systematic review; human health risk assessment; clinical medicine and epidemiology; and validation of test methods and the development of scientific confidence frameworks (see Appendix A for the committee’s biographical information). As noted, the committee was asked to review information on the variability and relevance of laboratory mammalian tests and approaches to validation and establishing scientific confidence in NAMs. The committee’s verbatim statement of task is provided in Box 1-1.

THE COMMITTEE’S APPROACH TO ITS TASK

An overview of the committee’s approach is depicted in Figure 1-1. To address its task, the committee held 23 meetings. Several meetings were devoted to discussing the charge and how the committee interprets relevance (as described in the introductory language) and its relationship to

concordance (as presented in charge questions). For instance, because the charge questions used the word “concordance,” the committee adopted this term; however, in order to capture the concept of relevance, the committee included in “concordance” the distinction between “qualitative” (related to hazard identification) and “quantitative” (related to dose-response) concordance (see Chapter 2 for detailed definitions). The charge questions also use the term “laboratory mammalian toxicity studies,” which the committee used to encompass traditional toxicity testing methods as addressed in the workshops.

In addition, various information gathering sessions were conducted to solicit input from sponsor representatives and members of the public, including during two virtual workshops involving experts from academia, industry, the government, and other organizations. The published reviews and analyses of data on variability and concordance of mammalian toxicity tests that were identified during the open sessions of the committee, including the two public workshops, were compiled and evaluated. The committee also conducted a literature review as part of the effort to address its charge.

Information Gathering Sessions

The committee conducted two open sessions to gather information from sponsor representatives and two public workshops. At the open session during the first meeting, the committee discussed the statement of task with the sponsor. The committee sought additional input from the sponsor during an open session of its twelfth meeting (NASEM, 2022a).

The committee held two public workshops over the internet for information gathering purposes. The first workshop focused on laboratory mammalian toxicity studies and their use in risk assessment. Critical questions addressed included the following:

• How are traditional toxicity studies used in informing chemical safety decisions?
• What do we know about the variability and concordance of traditional mammalian toxicity studies?
• What are the needs and expectations of different stakeholders?

The second workshop deepened the discussion using case studies to illustrate strengths and weaknesses of traditional and nontraditional toxicity data and risk assessment methods. While illustrative, the case studies were not intended as the only basis of the committee’s review of variability and concordance of existing laboratory mammalian studies. Critical issues explored in each case study included

• challenges identified by communities, including environmental justice concerns;
• variability and concordance of different types of data;
• concepts of adversity both in vitro and in vivo; and
• contrasts among guideline-based studies, non-guideline-based animal studies, and nonanimal studies.

The three case studies selected were chemical mixtures, developmental neurotoxicity, and estrogenicity and were intended to illustrate different types of challenges that NAMs could potentially address.

In both virtual workshops, experts from academia, industry, the government, and other organizations explored and discussed these issues and current scientific knowledge. All presentations were prerecorded and made available via the event webpage for viewing before the workshop. Live discussions during each workshop were also recorded and available for viewing after the workshops (NASEM 2022b,c).

Literature Review

The committee reviewed existing literature that evaluated evidence on variability of laboratory mammalian studies and the concordance with human evidence. The primary literature reporting on outcomes in relevant laboratory mammalian toxicity tests and amenable to de novo review and analysis was voluminous, and a formal systematic review of this literature was not considered within scope of the committee’s effort. However, the committee considered literature consisting of reviews, wherein information from multiple relevant studies, experiments, or databases was compiled and analyzed. Systematic reviews provide a transparent, comprehensive, and consistent evaluation of available data; are designed to have less bias than other types of reviews and analyses; and have been used or recommended by the NASEM for assessment of environmental health evidence in multiple reports (NASEM, 2017b, 2019, 2022d; NRC, 2014a,b). Thus, the committee conducted an overview review to identify and evaluate systematic reviews and authoritative reviews¹ of the scientific evidence of the highest methodological quality relevant to the committee’s

___________________

¹ The committee defines authoritative reviews as reviews produced by governmental agencies and international agencies (i.e., EPA, National Toxicology Program, U.S. state agencies, foreign governmental agencies, European Union, International Agency for Research on Cancer/World Health Organization.

charge. The overview approach used by the committee has been defined and used in clinical medicine (Pollock et al., 2019), and evaluation of systematic reviews has been applied in three prior NASEM reports evaluating evidence in environmental health, including the recent NASEM study on per- and polyfluoroalkyl substances (PFAS) as well as other reports sponsored by the EPA and the Department of Defense (NASEM, 2019, 2022d,e).

The goals of the approach were to identify relevant systematic reviews and authoritative reviews, evaluate their methodological quality, and illustrate the study strengths and weaknesses as well as the populations, interventions or exposures, and outcomes covered and where significant gaps may remain. Systematic reviews and authoritative reviews of higher methodological quality formed the evidentiary basis analyzed by the committee in reaching findings and recommendations that addressed the charge questions. Overall, this approach is consistent with the goal specified in the committee’s charge for a “comprehensive, workable, objective, and transparent process.” In addition, the approach is consistent with guidance from NASEM, in multiple reports, to use systematic reviews as a transparent and robust methodology to evaluate the environmental health scientific literature in support of hazard and risk assessment conclusions (NASEM, 2019, 2022d). Further, systematic reviews have been adopted by various federal agencies, including the EPA’s Integrated Risk Information System (IRIS) program (EPA, 2022) and the National Institutes of Environmental Health Sciences (NIEHS) Division of Translational Toxicology (DTT) Integrative Health Assessments Branch (IHAB) method. The TSCA also requires systematic reviews for evaluating the weight of evidence of hazard (Bennett, 1997). Accordingly, the committee’s findings can be readily adapted as they rely on evidence of the highest methodological quality to support decision-making in environmental health by federal entities.

The approach entailed development of a prespecified method as further described in Appendix C. This method detailed the key terms and their definitions, the objectives of the review, the scoping questions and associated population, exposures, comparators, and outcomes (PECO) statements, the inclusion and exclusion criteria, the literature search strategy, the process to assess methodological quality, and the analysis plan. The overall goals identified to guide the literature review were as follows:

1. To summarize the systematic reviews and authoritative reviews that assess the concordance of adverse health effects between laboratory mammalian models and humans following exposures to environmental agents; and
2. To summarize the systematic reviews and authoritative reviews that assess and evaluate variability of laboratory mammalian toxicity studies.

In brief, a comprehensive literature search of multiple databases was conducted using relevant terms. The results were reviewed for relevance by two independent screeners, and included studies were evaluated for methodological quality using AMSTAR 2 (A MeaSsurement Tool to Assess systematic Reviews) (Shea et al., 2017) in line with assessments of environmental health information by prior NASEM committees (NASEM, 2019, 2021, 2022e). An evidence map was generated to illustrate the extent of coverage with respect to the PECO statements as well as the quality of existing systematic reviews.² Systematic reviews judged to be of “critically low” quality were not considered further by the committee, whereas those of higher quality were summarized and formed the basis of the committee’s findings and recommendations.

In addition to the comprehensive overview review approach, the committee compiled and evaluated the published reviews and analyses of data on variability and concordance of mammalian

___________________

² See https://public.tableau.com/app/profile/leslie.beauchamp/viz/NAMsEvidenceMapDashboard/EvidenceMap?publish=yes.

toxicity tests that were identified during the open sessions of the committee, including the two public workshops (see Appendix C). This literature was reviewed for relevance to the statement of task by two independent screeners using prespecified inclusion and exclusion criteria, and the included articles were evaluated for methodological quality.

Synthesis and Focus of Committee’s Response to Charge Questions

In addressing charge questions 1–3, the committee determined that a review of primary literature was beyond the scope given the available time and resources. Recognizing that systematic review is in line with best practices, the committee instead searched for and reviewed existing systematic reviews and authoritative reviews while taking into account literature presented to the committee by the EPA and through the workshops, giving greater weight to studies of higher methodological quality. The committee considered any general conclusions about variability and concordance from this literature and drew lessons learned regarding the best methodological approaches to characterize variability and concordance. Furthermore, the committee focused on applying these lessons to the question of how information on variability and concordance from mammalian toxicity studies should or should not be used to evaluate NAMs, which also overlaps with charge questions 4–5.

In addressing charge questions 4–5, the committee determined that a more overarching framework was needed involving structured, multifactor evaluations of scientific confidence, rather than considering individual factors in isolation. In addition, to ensure coherence and consistency with previous NASEM reports, the committee integrated previous findings and recommendations into its deliberations, such as those related to the role of systematic review and the risk assessment approaches needed to adequately protect public health. Moreover, the committee recognized that scientific confidence in NAMs is an issue both for development of NAM-based testing strategies as well as for use of NAMs in human health risk assessments of particular chemicals. The committee therefore sought to delineate a framework that also facilitates the handoff from data generation to assessment.

ORGANIZATION OF THIS REPORT

The committee’s report is organized into five chapters and four appendices. Chapter 2 provides historical background as well as laying out key definitions and related issues. Chapter 3 describes the results of the committee’s review of information on variability, both experimental and biological, in laboratory mammalian toxicity tests, and it addresses charge questions 1 and 2. Chapter 4 describes the results of the committee’s review of information on concordance between laboratory mammalian data and human data, and it addresses charge questions 1 and 3. In both Chapter 3 and Chapter 4, findings and recommendations also touch on how the results of the committee’s reviews frame expectations for NAMs. Chapter 5 discusses the issues involved in developing a scientific confidence framework for the use of NAMs, which also integrates findings from Chapters 3–4 and thus addresses charge questions 4 and 5.

The appendices provide additional details as to the methods and results of the committee’s work. Appendix A contains biographical information about the committee members. Other appendices provide more detailed information regarding the open sessions of the committee including Workshop 1 and Workshop 2 (see Appendix B) as well as the literature review method (Appendix C) and the committee’s summary of existing scientific confidence frameworks for NAMs (Appendix D).

REFERENCES