Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version (2022)
Chapter: Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee
Visit
NAP.edu/10766
to get more information about this book, to buy it in print, or to
download it as a free PDF.
Previous Chapter:
Appendix A: Committee Member Biosketches
Appendix B
Next Chapter: Appendix C: Open Session Agendas
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Appendix B
2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee
2020 Handbook Organization and Contentsa
| Assessment Development Stage | Chapter | Summary Description |
|---|---|---|
| Scoping | 1 | Defines the parameters of the assessment based on EPA needs. |
| Problem formulation and IRIS assessment plan (IAP) development | 2 | Describes health effects of potential interest and key science issues. |
| Systematic review protocol | 3 | Systematic review procedures for populations, exposures, comparators, and outcomes (PECO) criteria; literature identification; study evaluation; and data extraction. |
| Literature search, screening, and inventory | 4 | Describes methods for performing comprehensive literature search(es). Uses PECO criteria to identify relevant human and animal health effect studies. Identifies absorption, distribution, metabolism, and excretion (ADME) studies; models; and mechanistic information. Categorizes studies (e.g., by study type, health effect) and extracts cursory information to allow for organization by study design/mechanism. |
| Refined evaluation plan | 5 | Describes process for deciding whether and how to prioritize and group sets of related endpoints into health effect categories for review, focusing on those most likely to inform hazard identification. |
| Study evaluation | 6 | Describes study evaluation methods for individual human and animal health effect studies, pharmacokinetic models, and an approach for mechanistic studies. Study evaluation includes consideration of reporting quality, risk of bias, and sensitivity. |
| Organize hazard review | 7 | Discusses approaches to finalize the utility and organization of health effect categories and studies for hazard identification. These decisions are informed by study evaluation, toxicokinetic, and consideration of mechanistic information. |
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
| Assessment Development Stage | Chapter | Summary Description |
|---|---|---|
| Data extraction and display | 8 | Presents types of key health effect study information to collect in a database and examples of graphical and tabular displays. |
Evidence synthesis
|
9 | Discusses considerations and approaches to analyze results incorporating the strengths and limitations of the sets of health effect studies of exposed humans (controlled exposure or epidemiology) and animal toxicology experiments by health effect or other grouping. |
|
10 | Presents a process to conduct focused, step-wise analyses of the most relevant mechanistic evidence and summarize results by health effect or other grouping based on the unique needs of the assessment (e.g., key science issues) and considerations that arise from analyzing the human and animal evidence (e.g., questions of biological plausibility or human relevance). |
| Evidence integration | 11 | Describes the contents of the evidence integration narrative for hazard identification and a framework to determine overall summary conclusions. |
| Hazard considerations and study selection for deriving toxicity values | 12 | Describes the selection process to determine the most informative studies and outcomes for dose-response analysis based on study confidence and other considerations including hazard judgments and susceptibility. |
| Derive toxicity values | 13 | Describes dose-response modeling and methods to develop a quantitative estimate for each hazard of concern (cancer and noncancer). This includes the consideration of uncertainty and susceptibility and description of confidence in the estimates. |
ahttps://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=350086. Under “Downloads,” see “Draft Charge Questions for Reviewers.”
QUESTIONS FROM EPA TO THE COMMITTEE
Peer review advice on the following charge questions will be most useful when prioritized to indicate its relative importance during revision:
- Tier 1: Recommended Revisions—Highest priority recommendations the committee believes are critical to improve the scientific rigor and/or clarity of the document.
- Tier 2: Suggestions—Recommendations that EPA should consider to strengthen the document.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
- Tier 3: Future Considerations—Topic areas that may inform future developments. These recommendations are outside the immediate scope and/or needs of the current document under review.
Please comment on each question below, elaborating on the rationale and scientific evidence relating to each comment, and do not limit comments to “yes” or “no.” For Tier 1 and Tier 2 recommendations, please provide specific revisions or alternatives to improve the clarity of the presentation and increase the scientific rigor of the approach.
- Please comment on the overall organization of the handbook, in particular on whether the key aspects of the assessment process are represented.
- Are the systematic review approaches used by the IRIS Program (outlined in Chapters 1-5) clearly described and consistent with methodologies considered to be state of the science by experts in the field?
- Are the study evaluation methods in Chapter 6 for individual human studies (epidemiology and controlled exposure), animal studies, mechanistic evidence (pilot testing approaches), and pharmacokinetic models adequate; if not, how can they be improved (Chapter 6)?
- Given the broad questions considered in IRIS assessments (typically necessitating multiple systematic reviews and dose-response analyses to address different health effects, exposure scenarios, and potential susceptible populations or life stages), the handbook outlines approaches for refinement of the scope and analyses in the assessment. This relates to multiple stages of assessment development, primarily problem formulation (Chapters 2), inventorying the literature (Chapter 4), refinement of the evaluation plan (Chapter 5), and organizing the hazard review (Chapter 7). These processes are used to inform a variety of subsequent assessment decisions, such as which health outcomes to focus on during the hazard review, how to approach the evaluation of mechanistic evidence, and identifying scientific complexities related to application of pharmacokinetic models or dose-response analysis. Does the handbook clearly lay out a state-of-the-science approach to refinement? Are there specific areas for improvement (please indicate recommended alternatives)?
- Sections 2.2, 4.3.3, and 6.6 and Chapter 10 describe the systematic process for evaluating mechanistic data. Please provide your review and assessment of the handbook’s process for evaluating and integrating mechanistic data. Are there specific areas for improvement (please indicate recommendedalternatives)?
- Chapters 9-11 of the handbook outline approaches for applying expert judgment to synthesize and integrate the available evidence based on considerations related to conclusions about the likelihood of a biologically plausible causal relationship. Are the approaches to evidence synthesis described in Chapters 9 and 10 scientifically sound? Are the considerations sufficiently broad to allow for application to the wide range of scenarios that will be encountered when applied to individual assessments? Are the methods sufficiently clear in describing the
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
- intent to synthesize the relevant evidence, incorporating study evaluation conclusions, regardless of the study results? Are the approaches described in Chapter 11 scientifically sound and appropriate for integrating the various types of evidence relevant to investigating the potential for human health effects from exposure to environmental chemicals?
- Chapter 11 presents five categories for drawing evidence integration conclusions, which builds upon judgments regarding the available human, animal, and mechanistic evidence. This approach was previously reviewed by the NAS in NASEM (2018) and presented in systematic review protocols released for public comment during 2018 and 20191. More recently, the IRIS Program considered a three-category approach for evidence integration that was disseminated in systematic review protocols released for public comment in 2019 and 2020.2 Please comment specifically on which of these approaches is recommended, why it is recommended, and any specific refinements for improvement.
- Chapters 12 and 13 outline considerations and approaches for selecting studies, specific health effects, and endpoints for dose-response analysis, and for selecting toxicity values. They also provide an overview of methods for conducting dose-response modeling and deriving toxicity values based on more detailed, existing guidance and recommendations. Does the handbook provide appropriate considerations for identifying data sets for dose-response analysis based on systematic review conclusions? Are the basic methods for dose-response modeling and deriving toxicity values consistent with the current state of the science, and presented with sufficient clarity?
REFERENCES
EPA (U.S. Environmental Protection Agency). 2018. Systematic Review Protocol for the IRIS Chloroform Assessment (Inhalation) (Preliminary Assessment Materials). U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-17/486. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=338653.
EPA. 2019a. Systematic Review Protocol for the Hexavalent Chromium IRIS Assessment (Preliminary Assessment Materials). U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-18/155. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=343950#tab-3.
EPA. 2019b. Updated Problem Formulation and Systematic Review Protocol for the Inorganic Arsenic IRIS Assessment. U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-19/049. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=343951.
EPA. 2019c. Systematic Review Protocol for the PFAS IRIS Assessments. U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-19/049. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=345065.
___________________
1 The IRIS systematic review protocols released for public comment with the five-category approach included EPA (2018, 2019a,b).
2 The IRIS systematic review protocols released for public comment with the three-category approach included EPA (2019c,d, 2020).
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
EPA. 2019d. Systematic Review Protocol for the Polychlorinated Biphenyls (PCBs) Noncancer IRIS Assessment (Preliminary Assessment Materials). U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-19/201. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=237359.
EPA. 2020. Systematic Review Protocol for the Methylmercury (MeHg) IRIS Assessment (Preliminary Assessment Materials). U.S. Environmental Protection Agency, Washington, DC, EPA/635/R-19/243. https://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=345309.
NASEM (National Academies of Sciences, Engineering, and Medicine). 2018. Progress Toward Transforming the Integrated Risk Information System (IRIS) Program: A 2018 Evaluation. Washington, DC: The National Academies Press. https://doi.org/10.17226/25086.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Suggested Citation:
"Appendix B: 2020 IRIS Handbook Organization and Contents and EPA Questions to the Committee." National Academies of Sciences, Engineering, and Medicine. 2022. Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington, DC: The National Academies Press.
doi: 10.17226/26289.
Subscribe to Email from the National Academies
Keep up with all of the activities, publications, and events by subscribing to free updates by email.