Appendix E
Examples of Issues Identified by the Committee

This appendix details some examples of the issues that the committee identified with regard to the transparency and consistency of presentation in EPA’s 2022 Draft Assessment. While not exhaustive, they are provided as illustrative of the Tier 2 and 3 recommendations from the committee and as a guide to EPA’s revision. References are provided in the main chapters of the report (Chapters 2–5).

The issues are organized according to the steps of EPA’s review process (see Figure 1-3): literature identification, study evaluation criteria, synthesis and judgments including any mode of action considerations, overall hazard conclusions, and dose-response evaluation. A final section details some general issues identified by the committee.

LITERATURE IDENTIFICATION

Figure 2-3 needs to be modified to more accurately show representative ranges of outdoor and indoor formaldehyde concentrations levels.

For respiratory pathology, it is unclear how search terms were used (e.g., controlled vocabulary [MeSH] and keyword terms).

For sensory irritation, it is unclear if other MeSH terms like eye, ear, nose, or skin were used in the search because they were not reported in Table A-31. The number of studies (38 observational and 20 controlled trials) included in the 2022 Draft Assessment, as presented in Figure A-22, does not match the number of studies presented in Tables A-34 (13 residential studies), A-35 (1 school study), and A-36 (21 controlled trials). Neither of the numbers match with Tables 1-1 (14 controlled trials) and 1-2 (6 residential studies) in the Main Assessment.

Regarding the nervous system, Figure A-35 of the Supplemental Information indicates that 40 human studies were considered, but only 12 are described in the Main Assessment in (Tables 1-44 and 1-45), and 15 were included in the Appendices (Tables A-84 and A-85). The narrative indicates that studies that were evaluated as not informative were excluded from these tables; however, both the Kilburn (2000) and Schenker et al. (1982) studies are described despite their overall confidence evaluation of not informative. Thus, it is not clear why these studies were included in the table. Clear reasoning for exclusion from tables need to be provided or the literature described in Tables A-84 and A-85 need to be updated to be consistent with the information provided at the end of Figure A-35.

Table A-84 incorrectly describes the sample for the Bellavia et al. (2021) paper as “cancer cases”; however, they were amyotrophic lateral sclerosis (ALS) studies from registry data. The job exposure matrices (JEMs) for this study was developed for a previous cancer study; ALS cases were identified through Danish medical records without consideration of cancer status. The details in Table A-84 need to be reviewed for accuracy, and minor typographical errors need to be addressed.

STUDY EVALUATION CRITERIA

For sensory irritation, EPA needs to clarify how concurrent assessment ensures that exposure preceded health effects for studies in which the outcomes are considered to be acute effects.

On page A-267 column 2, EPA compared the reporting of asthma prevalence in Liu et al. (1991) (~4%) to the national prevalence of asthma at that time and concluded there was “minimal concern for selection bias” in that study. The committee has some concern that this conclusion may not be fully supported. For example, there are several sets of selection factors between the general US population and the study population that need to be considered:

• Total eligible sample—eligible sample who finally participated in the study (represented as S1) and eligible sample who were invited but did not participate (S2). S1 + S2 = total eligible random sample.
• Target accessible population (S3)—the larger universe of mobile homes in California from which the random sample was drawn.
• Target population (S3)—all mobile homes in the entire country.
• General population in the country (S4).

At best, the comparison that EPA described in page A-267 for the Liu et al. (1991) study makes a case for the generalizability (external validity) of the findings, if it is assumed that the Liu study sample is representative of the US general population. It does not make a case against selection bias. One argument against selection bias would be to demonstrate that those who participated (S1) have comparable characteristics with those who were invited but did not participate (S2). It is also not known how the randomly selected sample compares with the target to the accessible population. If it is not a true random sample, there could be additional potential for selection bias.

For pulmonary function, while the study evaluation criteria provided in Table A-43 appear to be appropriate, they do not overlap with the methodologic issues mentioned in the text.

EPA’s approach appears to contradict the expert panel’s advice for asthma. Several studies in infants and young children (<3 years) are examined (p. 1-17 and Table 1-20), where the EPA considers diagnoses in young children to likely be exacerbations of respiratory tract infections rather than representing a true “asthma” phenotype.

Potential updates are needed for some studies across the noncancer outcomes considered. For instance:

• Annesi-Maesano (2012) is rated high confidence, despite the vague description of the outcome in Table 1-12. EPA needs to indicate whether this study used an appropriate questionnaire, which would support the high confidence rating.
• Two high-confidence studies (Ozen, 2002, and Ozen, 2005) could be reassessed based on small sample size (N < 10) for reassignment to medium confidence. One medium-confidence study (Sapmaz, 2018) could be reevaluated for consideration as high confidence because the test article box was flagged gray, when it likely ought to be clear. However, the potential reassignment of these three studies would not affect synthesis judgements and the next assessment steps, since all medium and high studies were considered in the next steps.
• The study by Smedje and Norback (2001) is rated as low confidence in Table 1-12, page 1-91, contradictory to the text on pages 1-81 and 1-83, which states it is medium. This discrepancy needs to be rectified. The low confidence rating is justified by the large proportion of exposures below the limit of detection, and possible information bias and confounding; however, the “alternative evaluation” of medium confidence features strongly in recommendations and the confidence judgment needs to be consistent throughout the document to avoid confusion. The alternative evaluation (i.e., medium

• confidence) must be adopted throughout the document for consistency, and its being a prospective study needs to be a criterion in the initial evaluation, not in its alternative reevaluation. It would be helpful to check all the reference table confidence statements with their citations in the text of the documents. Additionally, the tabulated data in the main document need to be checked for consistency with the longer descriptions located in Appendix A.5.4 regarding confidence judgments.
• Pinkerton et al. (2013) was rated with high confidence, with a comment of “small number of cases” in reference to eight ALS deaths. However, other studies with a small number of cases were rated as medium.
• Kilburn (2000) and Schenker et al. (1982) were assessed as not informative with comments about limited or no exposure measures. However, it is important to note that two of the included manuscripts used some version of the same set of Danish registry data (Bellavia et al., 2020; Seals et al., 2017), both used JEMs derived from the Swedish JEM used in the Peters et al. (2017) manuscript as part of the Nordic Occupation Cancer Study, and each of these JEMs estimate exposure based on previous studies of biological samples, air and dust monitoring, and expert opinion, and are not direct exposure measures. Furthermore, the occupational codes used to determine exposures are not based on job title, but only on tax-recorded industry codes. Therefore, exposure assessment provided in these studies is not significantly better than that provided in the studies ranked with low confidence.
• In Table 1-2, where the entry for the Main and Hogan (1983) study shows two sources of biases with one box fully colored. The classification scheme in Figure II would suggest this study is of medium confidence, but a fully colored box likely means something more, which needs to be explained.

EVIDENCE SYNTHESIS AND JUDGMENTS

Regarding pulmonary function, Table 1-11 indicates there are multiple additional studies included in the evidence judgment for long-term effects, although none of these are referenced. The evidence cited in the table and narrative need to be revised to provide more informative statements. For instance, “concentration-related associations” would preferably be rephrased as “concentration-related decrements in lung function.”

Regarding Figure 1-5:

• EPA needs to order the studies within industry by formaldehyde exposure levels, or the exposure difference between groups.
• EPA needs to improve the labeling of the figure to provide clarity, such as by indicating that the numbers before the plotted results refer to sample sizes.
• The text refers to Figure 1-5 summarizing ten studies, while the figure caption notes there are eight.
• Clarity would be improved by ensuring that the comparison being reported (i.e., preshift pulmonary function differences in prevalence studies) is clearly stated in the caption and accompanying text.
• Reference information is needed for the study discussed on page 1-45, lines 11–15, and the study referred to as “this study” on line 15.

The pulmonary function section would benefit from a summary table that refers to all summarized studies and provides an organized distillation of the points made in the text.

Tables need to be formatted clearly to correspond to the information highlighted in the synthesis discussion (e.g., for pulmonary function, which occupational studies had employees that worked at least 5 or at least 10 years—see the Assessment Overview, p. 52, line 33; or indicate the main conclusions EPA reached for each group of studies).

EPA needs to reconcile information provided in the text and in the tables for animal studies of male reproductive toxicity.

EPA needs to document whether the evidence integration summary tables in the Main Assessment and Assessment Overview are identical.

DOSE-RESPONSE EVALUATION

Regarding sensory irritation, in Tables 1-1 and 1-2, several studies have been identified as high or medium confidence, but only six were included in the sensory irritation dose-response analysis (Table 2-1). It is unclear why only these six studies were chosen.

Regarding reproductive and developmental toxicity, EPA needs to double check the publication year for references in Tables 30 and 31. The testes endpoint is likely incorrectly cited, linking to Ozen (2005) when it ought to be Ozen (2002). For animal male reproductive toxicity studies, Ozen (2002) was appropriately considered to be the stronger of the two studies and was therefore used to derive the RfC.

Regarding the RfC derivation, specific examples of issues concerning consistency, accuracy, and lack of transparency include the following:

• EPA followed its guidelines (BMD; RfC) to conduct dose-response modeling for selected studies/endpoints when the data supported such reanalyses. In many cases where raw data were not available, EPA extracted secondary data from reported results. For example, EPA extracted eight distinct model-predicted means, one for each exposure concentration, from the Hanrahan et al. (1984) by reading a plot (Figure 1), and then refitted a logistic regression model with a polynomial of order 3. The resulting logistic regression model has artificially narrow standard errors for the model parameters compared with those fitted to the raw data because the model-predicted means failed to reflect the true data variation. As a result, the model-based BMDL (0.09 mg/m³, Table 2-2) is biased. Therefore, the use of the cRfC derived from Hanrahan et al. (1984) as the osRfC for sensory irritation needs additional justification.
• In deriving a POD based on the human studies of Kulle et al. (1987) and Andersen and Molhave (1983), EPA correctly recognized that the same group of volunteers were exposed to multiple concentrations, and the responses from individual volunteers were correlated across different exposure concentrations. EPA’s benchmark dose (BMD) modeling failed to account for such data dependence, and thus likely underestimated the variation of the BMC. EPA subsequently divided the benchmark concentration (BMC) by a factor of 2 to replace the model-based benchmark concentration lower bound (Table 2-2). However, it is uncertain whether a factor of 2 is greater than the true ratio of BMC/BMCL (i.e., whether it is sufficiently large to account for the data dependence). This practice appears inconsistent with EPA’s own guidelines and EPA did not provide a justification.
• Kulle et al. (1987) measured eye irritation using a four-point Likert scale for none, mild, moderate, and severe, and reported the mean score difference (standard error) between 180 minutes postexposure and baseline. EPA’s dose-response modeling of Kulle et al. (1987) and Andersen and Molhave (1983) was, however, for the fraction of affected

• (i.e., prevalence of eye irritation). EPA did not provide information in the Main Assessment or Appendices on the conversion of the Likert-scale in Kulle et al. (1987) to the dichotomous variable used to estimate prevalence.
• EPA used either a group median (e.g., current asthma, Krzyzanowski et al. [1990]) or a group mean (e.g., atopic eczema, Matsunaga et al. [2008]) as an estimate of lowest observed effect level / no observed effect level (LOAEL/NOAEL) when a range of exposure concentrations was reported within the group. EPA also seemed to have used either an arithmetic mean or a geometric mean to estimate a LOAEL/NOAEL. Dannemiller et al. (2013) reported the geometric mean of 54.0 and 34.4 ppb in the “very poor control” group and “all others” group, respectively. EPA seemed to have designated 34.4 ppb as the NOAEL (0.042 mg/m³, Table 2-4). (Note that the reference in the last row of Table 2-4 ought to be Dannemiller et al. [2013]). The study of Dannemiller et al. (2013) was dropped from the discussion for cRfC derivation thereafter. EPA did not give reasons for excluding this study. The text starting from page 2-16, line 19, appears corrupted.
• For pulmonary function, EPA relied on the linear mixed-effects model results reported by Krzyzanowski and colleagues (1990). The original regression model incorporated whether or not the children had asthma and whether or not the exposure measurements were taken in the morning. EPA’s derivation of the BMCL was based on children without asthma exposed at times other than the morning. This approach was inconsistent with EPA’s state-of-the-practice methods (i.e., using more vulnerable subpopulations for risk estimation). To its credit, however, EPA conducted a sensitivity analysis for children with asthma and morning exposures. The sensitivity analysis, however, was based on BMC, but would more appropriately be based on BMCL. The derivation of a BMCL for children with asthma can be achieved by first determining the standard errors of the combination of regression coefficients associated with the terms of formaldehyde concentration and concentration squared. Note that children with asthma had their own baseline prospective epidemiological risk factor (PERF) (348.09) and the corresponding 10 percent decrease of 34.8. The intercept in equation B-9 is incorrect.
• EPA reanalyzed the animal data of Kerns et al. (1983) (Level I of sagittal cross-section I only) and of Woutersen et al. (1989) (anterior, Levels I and II). EPA did not disclose whether it obtained raw data or extracted secondary data from the original report. Multiple models were fit to the datasets, and the one with the smallest Akaike information criterion (AIC) was chosen. Note that AIC is a relative criterion for comparison across models. Alone, AIC does not tell how well a model fits the data. EPA did not report goodness-of-fit tests, a practice inconsistent with the reanalysis of other studies in the Assessment.
• In the reanalysis of the data from Ozen et al. (2005), EPA stated that (p. B-23, lines 6–8): “If the BMDL estimates were ‘sufficiently close,’ that is, differed by at most xx-fold, the model selected was the one that yielded the lowest AIC value. If the BMDL estimates were not sufficiently close, the lowest BMDL was selected as the POD.” This criterion is vague and does not enhance transparency.
• The various model-based tests for the Ozen et al. (2005) study were labeled as “test 1,” “test 2,” etc. (Table B-17, p. B-29) without explanation.
• In conducting BMD modeling of Ozen et al. (2002) and Ozen et al. (2005), which had three and four distinct concentration levels (including the control), respectively, EPA fit models with three or four parameters, resulting in parameter saturation in the model or failure in model fitting.

• The osRfC for respiratory pathology was based on Woutersen et al. (1989). Kerns et al. (1983) was incorrectly cited in Table 2-11.
• Figure 2-2 displays variability and uncertainty across all osRfCs along three dimensions: confidence, uncertainty, and risk size. The committee found this graphic display to be an effective visual aid. The size of the plot symbol for each osRfC was determined by three factors: the level of the confidence in the study(ies) and health hazard identification; risk estimate(s) (EPA gives slightly greater weight to the risk estimate than other factors); and completeness of evidence database for each health outcome. However, EPA was not explicit about how these factors were weighted.
• Figure 2-3 is also interesting and informative. It graphically displays the uncertainties within a cRfC as well as variations and uncertainties between cRfCs.
• EPA presented a detailed discussion of uncertainties and variabilities, and noted that the osRfCs for asthma, pulmonary function, allergies, and sensory irritation were 0.006, 0.007, 0.008, and 0.009 mg/m³, respectively, reflecting the impact of formaldehyde on the respiratory system. EPA proposed the overall RfC of 0.007 mg/m³, but was not explicit about how it was chosen.

GENERAL ISSUES

In some cases, Tables and Figures included in the Appendices are not cited in other documents (e.g., Figures A-24 to A-26).

As an example of inconsistencies that EPA needs to address, Figure A-36 shows 20 human studies and 35 animal studies for inclusion. Table A-93, which summarizes the animal data for developmental and reproductive toxicity and animal studies, contains 29 rows, with one row containing two studies (Vosoughi et al., 2012, 2013), resulting in 30 total animal studies (not 35 as stated in Figure A-36).

REFERENCES