Dropdown items
My Academies

Personal Library

Account settings

Building Confidence in New Evidence Streams for Human Health Risk Assessment: Lessons Learned from Laboratory Mammalian Toxicity Tests (2023)

Chapter: Appendix D: Selected Examples of Scientific Confidence Frameworks for NAMs

Visit NAP.edu/10766 to get more information about this book, to buy it in print, or to download it as a free PDF.

Previous chapter
Page of 158
Search this publication

Previous Chapter: Appendix C: Evidence Review: Approach, Methods, and Results

Page 152 Cite Bookmark

Suggested Citation: "Appendix D: Selected Examples of Scientific Confidence Frameworks for NAMs." National Academies of Sciences, Engineering, and Medicine. 2023. Building Confidence in New Evidence Streams for Human Health Risk Assessment: Lessons Learned from Laboratory Mammalian Toxicity Tests. Washington, DC: The National Academies Press. doi: 10.17226/26906.

Appendix D
Selected Examples of Scientific Confidence Frameworks for NAMs

TABLE D-1. Selected Examples of Scientific Confidence Frameworks for NAMs

Source	Description	Proposed Components (if any)
NRC, 2007	The report aimed to “review the state of the science and create a far-reaching vision for the future of toxicity testing,” with the aim to create more “informed environmental regulations and dramatically reduce the need for animal testing.”	The report includes a framework that includes key questions to address in implementation. The areas and concerns covered include the following: Identify key pathways, including multiple pathways, leading to toxicity. Determine adverse effects associated with toxicity-pathway perturbations, and predictive patterns of perturbations for health outcomes. Capture perturbations of toxicity pathways associated with developmental timing or aging for high-throughput assays. Investigate how exposure duration affects biologic responses. Study the effect of small amounts of toxicants on a toxicity pathway in light of pre-existing human exposures. Analyze how people differ in their expression of toxicity-pathway constituents and predisposition to disease and impairment. Ensure adequate testing for metabolites in high-throughput assays. Use computational tools to predict chemical properties, metabolites, xenobiotic-cellular and molecular interactions, and biologic activity. Develop methods to facilitate the discovery of circuitry associated with toxicity pathways that best reflect in vivo conditions, and design tests for volatile compounds. Select assays that capture elucidated pathways and best reflect in vivo conditions, and design tests for volatile compounds.

Page 153 Cite Bookmark

European Chemicals Agency, 2016

In 2016, the European Chemicals Agency (ECHA) brought together 300 stakeholders for a scientific workshop, “New Approach Methodologies in Regulatory Science,” to address the use of NAMs to support chemical regulatory decisions. Among the workshop conclusions were that “[d]ata from NAMs were shown to support read-across as well as providing useful and usable information for screening and prioritization” and that “WoE [Weight of Evidence] frameworks are essential for the successful use of NAMs and integration of the information they provide.”

To stimulate uptake, the identified needs included the following:

Standardize new approach method (NAM) approaches to make them transferable and transparent.
Establish relevance for regulatory use and reliability.
Standardize reporting templates.
Standardize frameworks for using NAM data.
Develop an inventory of NAMs.
Address data quality and coverage.

OECD, 2017

In 2017, the OECD published guidance for describing “non-guideline in vitro test methods” whose purpose was to “harmonise the way non-guideline in vitro methods are described.”

This guidance includes a framework for describing a test method that outlines the type and level of information that would ideally be reported, and includes the following sections:

General information
Test method definition
Data interpretation and prediction model
Test method performance
Potential regulatory applications.

Key performance measures requested include information on within/between laboratory repeatability, transferability, and reproducibility; reference chemicals, performance measures/predictive capacity, and scope and limitations of the assay.

ICCVAM, 2018

In 2018, the U.S. federal agencies that comprised the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) published A Strategic Roadmap for Establishing New Approaches to Evaluate the Safety of Chemicals and Medical Products in the United States. The ICCVAM strategic roadmap encourages evaluating the robustness of NAMs without relying solely upon in vivo animal reference data to define performance and increasing focus upon human mechanistic understanding when establishing scientific confidence in NAMs.

Connecting end-users with the developers of NAMs
Fostering the use of efficient, flexible, and robust practices to establish confidence in new methods
Encouraging the adoption and use of new methods and approaches by federal agencies and regulated industries

Page 154 Cite Bookmark

Source	Description	Proposed Components (if any)
Parish et al., 2020	In 2020, a multi-stakeholder group of industry, academic, and regulatory experts proposed a framework with endpoint-agnostic criteria to evaluate a NAM’s fitness-for-purpose.	Determining the context of use Addressing the core principles of accuracy, transparency, understanding limitations, and domain of applicability Laying out criteria by which a NAM is evaluated that vary in their importance depending on context of use
Joint Research Center, 2021	In 2021, a survey of more than 200 stakeholders from industry, academia, government, and nongovernmental organizations was conducted by the European Union Reference Laboratory for alternatives to animal testing (EURL ECVAM), with a focus on how to assess and validate complex in vitro methods.	(Human) physiological and biological relevance Reliability (reproducibility, repeatability) Predictive capacity Relevant endpoint
U.S. Consumer Product Safety Commission, 2022	In 2022, the U.S. Consumer Product Safety Commission published guidance on the regulatory evaluation of NAMs and associated informational needs for the purpose of reducing the use of animals in tests for required labeling under the Federal Hazardous Substances Act. The evaluation of a proposed NAM is flexible, requires professional judgment, and may be for a specific use. The information requested includes a well-defined endpoint, how the test informs risk assessment, limits of use, as well as eight criteria for method evaluation that include accuracy (concordance), sensitivity, and reproducibility.	Independent scientific peer review Potentially relevant information for method evaluation with eight criteria Standard operating procedures Well-defined endpoint Well-defined applicability domain Generate data useful for risk assessment Limits of use Reduce, refine, and/or replace
Casati et al., 2018	In 2018, International Cooperation on Alternative Test Methods (ICATM) published a list of 12 criteria (adapted from the principles in OECD GD 34) for the evaluation of defined approaches for assessing skin sensitization. The ICATM framework was adapted and applied to the defined approaches in OECD Guideline 497 on Defined Approaches for Skin Sensitisation (OECD 2021a), as described in an accompanying annex (OECD 2021b).	Reproducible (no less than reference animal test) Relevance/predictive capacity (comparable to animal test) Level of information (for at least hazard identification) Mechanistically and biologically relevant, i.e., cover molecular initiating or key event of an existing adverse outcome pathway (AOP) Transparently described using the OECD template Amenable to independent evaluation and implementation Include one or more OECD TG methods Conflicting results with in vivo data described Uncertainty described

Page 155 Cite Bookmark

		Quality assured, independent scientific review Defined criteria for relevant applicability domain Predictions should be considered in the context of Integrated Approaches to Testing and Assessment (IATA)
EPA, 2018	Also in 2018, using the ICATM criteria as a foundation, the EPA Office of Pollution Prevention and Toxics (OPPT) published a strategic plan to promote the development and implementation of NAMs within the TSCA program which included a list of eight criteria to be used as a starting point for considering scientific reliability and relevance of NAMs used within the TSCA program.	Decision context defined Mechanistically and/or biologically relevant to the hazard being assessed Defined criteria for reference and training chemicals Reliability within context of intended use Transparently described, datasets publicly available (except CBI) Uncertainty described Independent review Accessible and usable by third parties
Patterson et al., 2021	Patterson et al. (2021) reviewed numerous validation frameworks for new approaches based on in vitro and computational models and delineated seven common “credibility factors” that could be used to facilitate communication and comparison across frameworks. One set of factors are deemed “data-based,” which are possible to implement when there are sufficient high-quality data. By contrast, the other set of factors are deemed “knowledge-based,” meaning they are derived from adequate understanding of underlying biology. The authors posit that while ideally, validation is based on both types of factors, they actually can be complementary. In particular, even in situations where existing data or knowledge are limited, a degree of scientific credibility can still be established by considering the strengths and limitations of each.	Confirmation of assumptions: Assumptions and their limitations identified and justified by empirical evidence Qualitative concordance: Extent to which predicted behavior matches observations Quantitative concordance: Extent to which predictions represent observational data Explanatory power: Explaining phenomena, behavior, and situations, including data not used in “training” or “calibrating” the model Internal consistency: Lack of logical contradictions External consistency: Ability to predict similar effects/behavior as alternative approaches Simplicity: Minimum level of complexity necessary
van der Zalm et al., 2022	van der Zalm et al. (2022) proposed a framework for establishing scientific confidence in NAMs based on five elements. According to the authors, the framework encourages a holistic assessment of NAMs that can reduce or eliminate reliance on direct comparison to currently used animal test methods, and thereby enable better biological coverage, increase confidence in appropriate use of NAMs, and thereby accelerate their uptake.	Fitness for purpose, such as regulatory requirements, end-user needs, manner of incorporation into assessment, and context of use (e.g., screening/prioritization, grouping, hazard identification, etc.) Human biological relevance with respect to human biological information (physiology, mechanisms of toxicity) and human in vivo data (if available)

Page 156 Cite Bookmark

Source	Description	Proposed Components (if any)
van der Zalm et al., 2022		Technical characterization, including accuracy, reproducibility/transferability, applicability domain, reference chemicals and controls, and limits of detection/quantification Data integrity and transparency with respect other elements, including uncertainties Independent review of other elements

Page 157 Cite Bookmark

REFERENCES

Casati, S., K. Aschberger, J. Barroso, W. Casey, I. Delgado, T. S. Kim, N. Kleinstreuer, et al. 2018. “Standardisation of Defined Approaches for Skin Sensitisation Testing to Support Regulatory Use and International Adoption: Position of the International Cooperation on Alternative Test Methods.” Archives of Toxicology 92(2): 611–617.

European Chemicals Agency (ECHA). 2016. New Approach Methodologies in Regulatory Science: Proceedings of a Scientific Workshop: Helsinki, 19-20 April 2016. Publications Office of the European Union.

EPA (Environmental Protection Agency). 2018. “Strategic Plan to Promote the Development and Implementation of Alternative Test Methods Within the TSCA Program,” EPA Document# EPA-740-R1-8004. https://www.epa.gov/sites/default/files/2018-06/documents/epa_alt_strat_plan_6-20-18_clean_final.pdf.

ICCVAM (Interagency Coordinating Committee on the Validation of Alternative of Alternative Methods). 2018. “A Strategic Roadmap for Establishing New Approaches to Evaluate the Safety of Chemicals and Medical Products in the United States.” https://doi.org/10.22427/ntp-iccvam-roadmap2018.

Joint Research Center, V. Zuang, A. Dura, L. E. Ahs, J. Barroso, L. S. Batista, E. Berggren, et al. 2021. Non-Animal Methods in Science and Regulation. Joint Research Center.

NRC (National Research Council). 2007. Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC: The National Academies Press. https://doi.org/10.17226/11970.

OECD (Organisation for Economic Co-operation and Development). 2017. Guidance Document for Describing Non-Guideline In Vitro Test Methods, OECD Series on Testing and Assessment, No. 211. Paris: OECD Publishing. https://doi.org/10.1787/9789264274730-en.

Parish, S. T., M. Aschner, W. Casey, M. Corvaro, M. R. Embry, S. Fitzpatrick, D. Kidd, N. C. Kleinstreuer, B. S. Lima, R.S. Settivari, D.C. Wolf, D. Yamazaki, A. Boobis. 2020. “An Evaluation Framework for New Approach Methodologies (NAMs) for Human Health Safety Assessment.” Regulatory Toxicology and Pharmacology: RTP 112 (April): 104592.

Patterson, E. A., M. P. Whelan, and A. P. Worth. 2021. “The Role of Validation in Establishing the Scientific Credibility of Predictive Toxicology Approaches Intended for Regulatory Application.” Computational Toxicology (Amsterdam, Netherlands) 17 (February): 100144.

U.S. Consumer Product Safety Commission. 2022. “Final Guidance for Industry and Test Method Developers: CPSC Staff Evaluation of Alternative Test Methods and Integrated Testing Approaches and Data Generated from Such Methods to Support FHSA Labeling Requirements.” CPSC 87 FR 18772: 2022–06825.

van der Zalm, A. J, J. Barroso, P. Browne, W. Casey, J. Gordon, T. R. Henry, N. C. Kleinstreuer, A. B. Lowit, M. Perron, A. J. Clippinger. 2022. “A Framework for Establishing Scientific Confidence in New Approach Methodologies.” Archives of Toxicology 96(11): 2865–2879. https://doi.org/10.1007/s00204-022-03365-4.