6

Recommendations to Guide the Use of Race and Ethnicity in Biomedical Research

This chapter provides the committee’s recommendations for improving the use of race and ethnicity in biomedical research. The chapter begins by outlining conceptual foundations that delineate harmful from appropriate use of race and ethnicity. The committee then presents recommendations for the intentional use of race, ethnicity, and related concepts throughout the research process and discusses their implementation. The chapter concludes with steps that other entities in the research ecosystem can take to support biomedical scientists in operationalizing and adopting these recommendations.

CONCEPTUAL FOUNDATIONS FOR ASSESSING APPROPRIATE USE OF RACE AND ETHNICITY IN RESEARCH SETTINGS

The recognition that race is a social construct raises questions about whether race and ethnicity data should be captured at all in biomedical settings and, if so, for what purposes. The position of this committee is yes—collecting information on race and ethnicity can serve the goals of fairness, inclusion, and equity in biomedical research. In addition, recording race and ethnicity data will continue to be necessary to examine health inequities and for legal reasons. However, despite continued use and measurement, race and ethnicity should never be construed as biological constructs or relied upon as a causal explanation for individual health outcomes. Indeed, questions about human diversity are always multidimensional, and race and ethnicity alone are insufficient to capture the complexity of information that contributes to health outcomes. This is a recurring theme also identified in another recent National Academies report that grappled with the use of race and ethnicity in genomics research (NASEM, 2023).

Describing race and ethnicity as social constructs gives rise to a fundamental tension—race and ethnicity are not suitable proxies for biological mechanisms, yet diverse representation in biomedical research is essential. It is also important to acknowledge how these constructs give rise to differing social realities and to respect the ways that people identify themselves. Although the underlying biology is the same, race and ethnicity, if used thoughtfully, can serve a number of purposes in research, such as to ensure scientific rigor with sample populations that represent a range of life experience and social contexts, to track health disparities, and to account for how individuals self-identify.

Harmful Uses of Race and Ethnicity in Biomedical Research

Here the committee describes three research practices that are harmful yet continue to appear in biomedical research. Avoiding these problems does not inherently mean that the use of race and ethnicity is appropriate. Assessing appropriate use requires careful decision making, and the remainder of this report is dedicated to this subject and provides recommendations and resources to assist researchers in making these decisions.

Incorrect Assumptions

Harmful uses of race and ethnicity in research can be subtle and hard to detect due to unrecognized, deep-seated assumptions or stereotypes about individuals and groups. An invalid presumption of innate difference between groups of people is at the root of many problematic uses of race and ethnicity in health and biomedical research. It is wrong to make assumptions about a patient or research participant based on their race or ethnicity. Although social determinants of health are often differentially distributed by race and ethnicity, presumptions based on such associations can pathologize people and groups and is misguided.

Causal Inferences from Observational Studies versus Experimental Research

There are key distinctions between experimental and observational research that make a difference when considering whether to use race or ethnicity as a variable in research. Experiments are designed to control variables and evaluate causal hypotheses—an independent variable is manipulated, and the effects on dependent variables are measured. Observational studies, in contrast, examine correlations between variables. Because observational studies do not manipulate isolated variables, they cannot discern causality. Neither race nor ethnicity can be isolated as independent variables in an experimental setting,¹ but these constructs often serve as a proxy for the true variable of interest. Thus, biomedical studies that use race and ethnicity as variables can only point to correlations and do not enable causal inference. Nevertheless, race- or ethnicity-related

___________________

¹ With the exception of controlled social science experiments intended to uncover evidence of racial and ethnic discrimination. For instance, racial and ethnic cues/classifications can be manipulated or otherwise varied in vignettes or audit studies designed to test for bias.

inferences sometimes go beyond claims of association or correlation in research. Besides investigators themselves making inferences, journal reviewers and editors ask researchers to speculate on the pathway between race or ethnicity and outcomes on the way to publication, highlighting the shift needed throughout the research ecosystem to eliminate biological essentialism of race and ethnicity.

In addition, epidemiological differences (e.g., observed differences in disease prevalence across racial and ethnic groups) can be taken out of context or wrongly interpreted to mean that an individual’s race or ethnicity is a causal explanation for disease. In clinical medicine, race and ethnicity are often characterized as risk factors for a disease or health outcome. Risk factors are attributes associated with an increased likelihood of developing a disease or a health outcome. The presence of a risk factor does not make a particular health outcome inevitable. In a classic example, in the early to mid-nineteenth century, it was observed that people living at low altitude were at increased risk of contracting cholera, leading many to believe that cholera was caused by bad air (miasma theory). John Snow, in a series of pioneering investigations, demonstrated that contaminated water was the source of cholera and that low altitude was associated with cholera only because people living at low altitude consumed more contaminated water than people living at high altitude (Bingham et al., 2004). As with bad air in this cholera example, without sufficient context, variables such as race and ethnicity can be incorrectly invoked in disease etiology as the cause of a health condition.

It is thus important to recognize that race or ethnicity may be correlated or associated with differences in disease prevalence, but differences in disease burden between populations do not unveil the specific factors underlying the disease. In fact, the biological or environmental mechanism may be unknown, and research may be needed to examine other explanatory variables or to better understand the fundamental biology.

Quantifying Racial and Ethnic Differences in Physiological Traits

It is also worth considering the supposed purpose of quantifying racial and ethnic differences in physiological traits in study design. The medical literature is replete with reports of differences in physiological or anatomical traits across racial and ethnic groups. When performed without a scientific rationale or hypothesis, such a search for racial and ethnic differences may appear to be a “fishing expedition” and does not make for sound science. If enough comparisons are made, between-group differences in traits are likely to be found, but within-group differences are also likely to be found. Occasionally, a noted difference in a given trait between racial or ethnic categories offers a potential lead for further hypothesis-driven investigations, which may prove to be illuminating and uncover information about the true drivers of disease. However, careful consideration should be exercised when reporting diseases, outcomes, or traits found to be associated with race or ethnicity by avoiding assumptions and discussing what the study limitations may be, given that it is not a causal relationship.

Racial categories are heterogeneous and have high within-group variation. Because a trait’s distribution can vary widely in a population, even if a small difference exists in the means between two groups of people, the difference is not necessarily meaningful

for individuals. A poor proxy for other factors that are at play, race is ill suited to tease apart effects due to various confounders, and other measures may be more fitting for the scientific question at hand (see Chapter 5). The same is true for ethnicity. Hence, it is inappropriate to use either race or ethnicity as an explanation for health or research outcomes, and it is necessary to consider the full context, including salient social, behavioral, or environmental factors.

Appropriate Use of Race and Ethnicity Is Context-Dependent

Decisions about the use of race and ethnicity in biomedical research require careful deliberation. Although some situations may be clear-cut, most are nuanced, involving balanced consideration of ethical, contextual, and scientific factors. Important ethical considerations include respect for individuals and communities, beneficence for all parties involved in the research and for the public at large, and justice throughout the biomedical research process (NASEM, 2023; U.S. Department of Health, 1979). Collecting race and ethnicity information for purposes such as recruitment, fairness, and equity may be largely appropriate from an ethical standpoint but still requires careful consideration of context. For example, collecting race and ethnicity data during recruitment can help ensure a diverse population of participants. This information can be used to engage the right population, increase accessibility of research studies, and disseminate research results to those who may benefit. However, even for well-intentioned purposes, the correct approach to using race and ethnicity depends on the research question of interest and the specific context (Quinones et al., 2024). Context can encompass a variety of biological, social, cultural, behavioral, and environmental factors, including social and historical background that may have contributed to the existing evidence base. Lastly, decisions to use race and ethnicity should uphold scientific validity given the research question of interest. It is important to consider whether race or ethnicity is best suited to the scientific purpose or whether another measure might better address the question.

The remainder of this chapter explores these considerations in more depth and offers recommendations and tools for scientists and others in the biomedical research ecosystem to use in assessing particular use cases and determining appropriate use of race and ethnicity in a context-dependent manner.

CONSIDERATIONS FOR THE USE OF RACE AND ETHNICITY THROUGHOUT THE RESEARCH PROCESS

As discussed in Chapter 2, biomedical research is governed by not only scientific principles, but also ethical principles. Given the breadth of types of biomedical research, the committee organized their analysis around the stages of the research process cycle (Figure 6-1). With some variation, this framework is useful for biomedical research broadly, across the translational spectrum from the bench to the bedside and out into the community. Although societal views and the definition of race and ethnicity will continue to change over time, as will biomedical study designs, this research process framework can be adapted to many scenarios.

The conceptual considerations described in the previous section may be most readily apparent in the early stages of the research process—during its conception and design, as well as during outreach and recruitment of study participants in studies where those phases are relevant. But in truth, issues of race and ethnicity recur throughout all stages of the research process. Because appropriate use is context dependent and the context will change throughout stages of a study, it is important to evaluate and make decisions about the use of race and ethnicity multiple times during the research process. Based on this reasoning and evidence presented in prior chapters,² the committee provides the following conclusions and recommendation:

Conclusion 6-1: Both deciding to use race and ethnicity and deciding to omit race and ethnicity can have advantages and disadvantages in biomedical research. It is important to evaluate potential implications, benefits, and risks not only of using race and ethnicity but also of forgoing collection of these data entirely.

Conclusion 6-2: Addressing the use of race and ethnicity at only one stage of a study fails to capture the unique factors and consequences that can emerge at subsequent steps of the process.

Recommendation 1: At every stage throughout the biomedical research process, researchers should scrutinize, evaluate, and decide whether the use of race and ethnicity is appropriate or inappropriate. Researchers should:

• Identify how the historical or social context, including prior uses of race and ethnicity in research, affects the underlying evidence base for the question of interest;
• Use race and ethnicity in ethical ways based on the context and research question, with a principled scientific rationale documented throughout the study;
• Understand distinct contexts and requirements for partnering with specific populations and communities, which could include American Indian or Alaska Native Tribes and their distinct political status as sovereign nations;
• Consider the benefits of collecting race and ethnicity information for research purposes, including promoting diverse representation and equity, when these constructs are not central to the research question;
• Refrain from making unsupported inferences from the analysis, such as relying on race and ethnicity as causal attributes that drive biomedical research outcomes in individuals; and
• Weigh the potential implications, limitations, benefits, or harms of using or not using race and ethnicity.

In publications, researchers should articulate their decisions about whether and how to use race and ethnicity in their research studies and reflect on the outcomes.

___________________

² See discussion in Chapters 3 and 4 about pulse oximetry, race correction, and clinical decision-making tools.

With the goal of eliminating racial and ethnic bias from both biomedical research and its applications in health care, there are four key considerations that researchers will need to address throughout the research process:

1. Assessing whether to include race and ethnicity and, if so, how to use them.
2. Forming enduring partnerships with communities.
3. Ensuring inclusion and equity for everyone involved in the study and those most affected by the study results.
4. Recognizing and characterizing the biases and limitations of datasets and study methodology.

Each consideration is explained briefly in the subsections below, followed by a list of questions to help researchers operationalize these recommendations at each stage of the research process (see “Questions to Assess the Key Considerations”). Subsequent sections of the report provide recommendations and tools for effectively addressing each consideration.

Assessing the Use of Race and Ethnicity

The role of race and ethnicity in a given study should be considered as early as the conceptualization of the research question. From the outset, it is important to consider whether the use of race or ethnicity would be appropriate or inappropriate in this context and for what reasons. For purposes extrinsic to the research question, such as inclusive recruitment, the use of race and ethnicity is generally appropriate. (See “Conceptual Foundations for Assessing Appropriate Use of Race and Ethnicity in Research Settings” above.) When using these constructs, they should be contextualized. It is equally important to consider the benefits and potential limitations or consequences of using race or ethnicity. Working with racial and ethnic communities can help researchers identify some of these potential limitations or unintended consequences.

Forming Partnerships with Communities

If considered at all, community outreach is often addressed only once in a study—typically in conjunction with recruitment. However, considerations for building effective community partnerships should be embedded at each stage of the research process to assist with determining whether the use of race and ethnicity is appropriate. In developing a research program, investigators can start by assessing whether community engagement is suited to the specific aims of the study and, if so, determining what type of community partnerships will be most effective for accomplishing the research goals and understanding community views on how race and ethnicity may be used (see “Building Community Partnerships” below). If the decision is made to use race and ethnicity, it is necessary to consider how the data are most appropriately collected. Another important consideration is the time frame for engagement,

including appropriate planning of the steps and time required for successful community outreach. In addition, early in the process the study team should ensure that it incorporates the expertise needed to accomplish the four considerations under discussion here. The study team may benefit from including experts in community engagement or community leaders and liaisons.

As a study is completed, it is important to keep its potential impact on the community in mind by considering how the study results may be used to improve health and not reinforce racial or ethnic stereotypes. Thus, efforts to involve communities in the dissemination of the results should be considered. After a given study, reflecting on the process during an evaluation stage is helpful. Researchers can reflect on how community expertise influenced or altered the research process details or the trajectory of the research. Because sustained investment builds trust among researchers and communities, investigators might also consider ways to maintain a relationship with the community after the specific research study has concluded. Building trust is a continuous process that serves to increase the quality of the data collected and thus support stronger scientific conclusions.

Ensuring Equity and Inclusion in the Research Process

To promote equity and the inclusion of historically underserved racial and ethnic populations at multiple points throughout the research process, study teams should start by considering how diversity is defined in the study and what methods will be used to recruit and include a diverse sample of participants. However, these considerations do not stop with recruitment. It is equally important to consider rigorous ways to design scientific analyses and interpret results to foster inclusion throughout the process. Researchers should also keep in mind how the research could benefit or negatively affect the participants and the communities who contributed to the work. Where relevant, those seeking to evaluate potential effects of their work on health equity might consider the Health Equity Impact Assessment Tool³ and adapt the questions to their study context. In addition, equity and inclusion are key aspects to examine during the re-evaluation phase of the research process when study teams can assess how to address any challenges encountered in future studies as well as reflect on potential social impact of their work.

Evaluating Datasets and Study Methodology

In the process of designing and executing a research study, it is important for researchers to explicitly define how race and ethnicity data will be collected and used. For this reason, researchers should contemplate two points. First, what is the source of the data: primary/original data collection, secondary data—e.g., legacy study data, data derived from electronic health records (EHRs), or financial claims data—or a

___________________

³ https://www.camh.ca/en/professionals/professionals--projects/heia/heia-tool (accessed August 20, 2024).

combination of the two? (See Chapter 3, section “Race and Ethnicity in Secondary Data Analysis” for more detailed descriptions.) Second, how could the source of the data affect the study? Although investigators have less control over the information within secondary datasets, it is still important to recognize how the data were assembled, state what is known about data provenance, assess potential bias, and acknowledge limitations of the datasets used. Of note, recruiting requirements necessarily differ across small and large trials. The committee recognizes that many studies, including Phase I trials, are conducted with small sample sizes. The committee also maintains that the principles of transparency, openness, and reproducibility apply wherever race and ethnicity data are collected—regardless of study size. The committee provides the following conclusion and recommendations:

Conclusion 6-3: Continued use of some legacy datasets may be more harmful than beneficial, and some legacy race and ethnicity data should no longer be used in future biomedical research. Because knowledge and reporting has changed over time, combining legacy with current datasets is problematic. Particularly common and problematic in legacy data are issues of missing data and aggregating data into an “Other” category.

Recommendation 2: Whether conducting primary research or secondary data analysis, biomedical researchers should provide an operational definition of race and ethnicity, if used, in all grant applications, manuscripts, and related products. Within these products, researchers should explain their rationale and the limitations of their approach as well as describe attributes of data provenance, such as:

• Which race and ethnicity categories were used for enrollment and/or scientific analyses and why (e.g., which version of the Office of Management and Budget categories was used);
• How race and ethnicity data were reported (e.g., self-identified or socially assigned);
• When data were collected;
• Whether any subcategories were aggregated, including whether samples were relabeled, combined, or harmonized across various sources;
• Whether any race and ethnicity data were derived (e.g., imputation, estimation), and how; and
• Whether bias may exist due to the way categories were defined and handled (e.g., sampling, classification, method of data collection, completeness of data).

Data-related considerations are critical for biomedical studies that rely on secondary datasets. For instance, the development of biomedical and health technologies usually relies on secondary datasets rather than original data collection, underscoring the importance of characterizing the existing data to understand bias

and limitations. Tools such as the Bias Elimination for Fair and Responsible AI in Healthcare (BE FAIR) framework (Cary et al., 2024) and the Racial Bias in Data Assessment Tool (Burkhardt et al., 2021) can be useful resources for evaluating datasets for bias. Moreover, limitations identified need to be disclosed. In addition to accounting for potential bias when designing the technology, model or device performance should be evaluated for bias by conducting comprehensive subgroup analysis (e.g., assessing performance metrics across racial and ethnic groups). Even tools that do not explicitly include race or ethnicity as an input can have differential effects for different subgroups. Technology such as algorithms is often applied in the clinic and then monitored. The performance can shift over time and contexts—known as “model drift”—and requires ongoing monitoring and evaluation.

Recommendation 3: Researchers should operate with transparency at every stage in the development, application, and evaluation of biomedical technology that may influence health (e.g., clinical algorithms, artificial intelligence [AI] models and tools, medical devices). Researchers should assess and report the performance of biomedical technology across a range of racial and ethnic groups.

Questions to Assess the Key Considerations

At each stage of the research process, these four considerations (assessing the use of race and ethnicity, forming partnerships with communities, ensuring equity and inclusion throughout the research process, and considering data limitations and study methodology) raise questions for investigators and study teams to consider. For a compiled checklist of questions that researchers can ask themselves throughout the research process, see Box 6-1. As they move through different phases, study teams are encouraged to revisit these questions as the context changes.

APPROACHES AND METRICS TO STRENGTHEN INQUIRY ABOUT RACE, ETHNICITY, AND RELATED CONCEPTS

In biomedical research and clinical care, race and ethnicity are often relied upon for purposes that could be better served by using more specific approaches, concepts, or measures. One of the factors perpetuating this problem is the use of the Office of Management and Budget (OMB) race and ethnicity categories beyond their initially intended uses, which are to track the inclusion of U.S. population groups in federally funded activities and provide fairer access to and allocation of federally funded goods and services. Though the OMB categories are frequently required by funders and sponsors for reporting recruitment statistics, the same categories are often used in the design of the study, to structure data for analysis, and to draw inferences. These latter uses are not required, can be scientifically unsupported, and are often less informative than approaches conceptualized based on the research questions. Parsing racial and ethnic categories in more granular ways or in ways that focus on more meaningful variables of health and disease, such as social determinants of health or underlying biological mechanisms, will enhance the discovery process.

Race is often used in place of a variety of distinct, albeit related, concepts—ethnicity, indigeneity, ancestry, and more—which conflates these concepts and collapses multidimensional information about identity and experience. Using a single self-identified measure of race can also contribute to conflating dimensions of race by overlooking interactive or relational aspects that are a part of the social experience of race. Recognition of the multidimensional aspects of race and multilevel aspects of racism can facilitate disaggregating these ideas in ways that are more useful and precise for research. In addition, race and ethnicity are sometimes used as a proxy for unexplained variance, which can lead to misattributing differences—which may be due to a variety of biological, social, or environmental factors. As demonstrated in Chapter 5, there are myriad ways to measure and interrogate the mechanisms at work. Using targeted approaches to tease apart the underlying mechanisms and phenomena will improve understanding and make for better science. Given these considerations, the committee offers the following conclusions and recommendation:

Conclusion 6-4: The concepts of race and ethnicity, among others, are defined, used, and misused differently across various domains of biomedical research, which has contributed to confusion and misunderstandings. However, there is no single unifying measure for the concepts of race and ethnicity across contexts and use cases.

Conclusion 6-5: Race and ethnicity conflate many concepts and collapse multidimensional information about people’s experience and identity. There is a need for disaggregation of related concepts and for increased granularity in the data collected to better capture the information for which race has been a proxy. Greater methodological specificity will be required to disentangle the various concepts that are often collapsed into a single “race or ethnicity” descriptor or variable.

Recommendation 4: Researchers should strive to identify which concepts often conflated with race or ethnicity (e.g., environmental, economic, behavioral, and social factors, including those related to racism) are relevant to their study. Based on those concepts, researchers should select applicable measures and do the following:

• Researchers should not rely solely on self-identification with OMB race and ethnicity categories.
• To the greatest extent possible, researchers should incorporate multiple measures in study design, data collection, and analysis to allow for comparison or combination.
• If using a single measure, researchers should articulate a clear scientific justification for why it was chosen and discuss its limitations.

To assist researchers in operationalizing Recommendation 4, Table 6-1 lists key social and biological concepts for which race frequently serves as a proxy. Instead of categorizing groups of people based solely on broad race categories and using race as the framework and basis for data analysis and drawing conclusions, researchers can refer to this table periodically throughout the research study process to assess what other concepts may be salient to incorporate in the study context. If researchers engage communities during the earliest stages of the research cycle and continue to partner with them throughout the process (see section below, “Building Community Partnerships”), community members can aid in identifying what concepts may be most apt in the specific study. Some of these measures may involve disaggregation or categorization that requires larger sample sizes to statistically power some of these analyses. This may increase the cost of some studies to recruit appropriate sample sizes for the targeted measures.

Incorporating People Who Are Members of Small Populations and Who Identify as Multiracial or Multiethnic

Fostering inclusion in the research process goes beyond the early stages of recruitment. The promise of inclusion may not be fulfilled if participants are recruited and then their data are summarily excluded from analysis due to issues such as small sample size. For instance, the multiracial population represents an increasing percentage of the U.S. population, but people who are multiracial are often left out of analyses. Sometimes their data are not used; other times the data are aggregated into a group labeled “Other.” As demonstrated in Chapter 5, there are many approaches taken to analyzing data from people who identify as multiracial or multiethnic. There are advantages and disadvantages to the various methods, but more work is needed to determine best practices for different applications or types of research. Currently, there is no definitive best practice, and the optimal methodology will depend on the context and research question of interest.

Many considerations for including multiracial and multiethnic people highlight concerns with how smaller racial and ethnic populations are treated in biomedical

TABLE 6-1 Race, Ethnicity, and Associated Concepts^†

^† In the United States, the OMB categories are often used interchangeably with race. The 2024 update to the OMB standards combines race and ethnicity categories under a single question. Ethnic categories are often conflated with race categories and sometimes also used as a proxy for concepts described in the table.

NOTES: Race and ethnicity are often used as proxies for other concepts or measurements. Targeted approaches, such as those described in this table, measure or interrogate race and ethnicity more directly. The concepts and measures above can be exposures, mediators, or moderators depending on the context. Additional detail and references can be found in the corresponding subsections of Chapter 5.

¹ OMB (2024). ² Roth (2016). ³ López et al. (2018). ⁴ Adkins-Jackson et al. (2022). ⁵ Brown and Homan (2024). ⁶ Dean and Thorpe (2022). ⁷ Gee and Ford (2011). ⁸ Blankenship et al. (2023). ⁹ Yearby (2020). ¹⁰ Ford and Harawa (2010). ¹¹ Afshari and Bhopal (2010). ¹² Massey and Denton (1993). ¹³ Wight et al. (2011). ¹⁴ Asad and Clair (2018). ¹⁵ Gartner (2023). ¹⁶ Huyser and Locklear (2023). ¹⁷ Liebler (2018). ¹⁸ Dixon and Telles (2017). ¹⁹ Gordon et al. (2022). ²⁰ Monk (2023). ²¹ Morning and Saperstein (2018). ²² NASEM (2023). ²³ Djuric et al. (2008). ²⁴ Lawrence et al. (2022). ²⁵ Aronson and Ferner (2017).

research more generally. In samples designed to be nationally representative, there can be too few individuals who identify as American Indian or Alaska Native, Native Hawaiian or Pacific Islander, Middle Eastern or North African, or Asian to make valid statistical comparisons. Data from these individuals are often excluded or lumped together into a single residual category from which no meaningful inferences can be drawn. Ensuring diverse samples requires revisiting these past practices and making positive efforts toward respectful and equitable inclusion in

biomedical research. Therefore, the committee provides the following conclusions and recommendations:

Conclusion 6-6: Many people are left out of research analyses either due to missing data or because none of the available categories reflects their background. More granular categories may be aggregated, potentially obfuscating missing data or a misalignment of participants’ identities with the available categories. “Other” is a category label sometimes used to aggregate data—combining race and ethnicity categories that are too small for separate analysis, individuals with missing data, and individuals who do not identify with the available race and ethnicity categories.

Conclusion 6-7: There is an increase in multiracial identification in the United States, but there is no standard way to account for multiracial or multiethnic people in biomedical research. Even if they are recruited, many people who are multiracial or multiethnic are left out of analysis, often because of small sample sizes or uncertainty about how to conduct the analysis. There is a need to include people with mixed ancestry or multiple identities in biomedical research and to appropriately incorporate them in analyses to the greatest extent possible to ensure a diverse sample population.

Recommendation 5: At each stage of the research process, all racial or ethnic category inclusions and exclusions should be based on a clear scientific rationale motivated by the research question.

Researchers should:

• Consider oversampling for smaller populations to ensure adequate power for analysis.
• Describe and characterize all recruited populations, even if some cases cannot be included in an analysis due to limits of small sample size.
• Articulate the purpose of aggregating categories, deriving missing data, or omitting cases.
• Use aggregate category labels that are motivated by the research question (e.g., “Members of minoritized racial and ethnic groups”) or reflect the analytical approach (e.g., “Remaining participants”).
• Justify the choice of reference population.

Researchers should not:

• Combine categories solely to improve statistical power.
• Make inferences about residual categories.
• Aggregate participants into the nonspecific category labels “Other” or “non-White” because they can be isolating and reinforce one category as the norm.

Recommendation 6: Researchers should consider the inclusion and analysis of multiracial and multiethnic participants at each stage of the research process, especially when developing research questions and designing the study.

Throughout the course of a study, researchers should:

• Identify relevant concepts (e.g., ancestry, self-identification);
• Ensure that respondents can select multiple races, ethnicities, or ancestries during data collection;
• Report granular data for multiracial or multiethnic respondents to the greatest extent possible, while respecting confidentiality concerns; and
• Identify a plausible classification scheme for including multiracial and multiethnic people in analysis, based on the research question or context; or provide a comparison of results using alternate approaches.

Researchers should design their studies in advance with small racial and ethnic populations in mind, including incorporating oversamples as needed to ensure statistical power for relevant comparisons. When this is not possible, including when using legacy data, all data aggregation should be scientifically justified based on hypothesized mechanisms of interest. For instance, two small populations may experience the same exposure or environment, which could warrant combining the categories. Increasing statistical power is not inherently sufficient justification to aggregate categories, and researchers should consider whether pursuing statistical significance or power will actually improve the quality of the results. Researchers should also avoid using terminology such as “Other” or “non-White” when labeling population groups. To “other” individuals or groups is dehumanizing and is particularly inappropriate when the data were not collected using that term. Using “non-White” implies that White people are the reference group against which everyone else should be compared. Labeling terminology should be respectful of people’s identities and relevant to the research question. To be sure, even when offered more granular options, some participants may still prefer to have the opportunity to write-in another racial or ethnic identity because their identify is not captured among the finite number of racial and ethnic categories. Recommendation 5 specifically recommends not assigning participants to the category “Other” as a means of dealing with small group sizes when the participants themselves did not select it.

Some racial and ethnic groups may have shared history or experiences of discrimination, and the term “non-White” has sometimes been used in historical and social research to emphasize solidarity among some minoritized populations. Even so, the term is not well suited to biomedical research because it homogenizes experiences across diverse racial and ethnic groups. Recommendation 5 does not preclude the possibility that there could be a valid reason to compare racial and ethnic groups to White populations; however, researchers should question assumptions that emphasize a division between White/non-White and avoid defaulting to approaches that reinforce White as the norm. For example, if studying consequences of White supremacy, there may be a need to distinguish White populations; however, from a scientific standpoint, researchers should consider whether comparing White and non-White is in fact the most effective methodology because there could be heterogeneous effects across groups, which challenges the practice of modeling all other racial and ethnic groups as one homogeneous population. Regardless, “non-White” is not an appropriate term for aggregating small populations for analysis. A number of other terms could be used instead, and Recommendation 5 offers two alternatives for researchers to consider.

The terms chosen should be specific to the methodology and research context. For example, “Members of underserved racial and ethnic groups” may be relevant in the context of health care access.

As with considering appropriate category labels, inclusion of multiracial and multiethnic respondents is not only relevant at the data collection stage. Accounting for mixed race, ethnicity, or ancestry should also be built into study designs to better understand both within and between population variations. Throughout study design and analysis, researchers should think critically about assumptions related to group “homogeneity” or “admixture”—that is, ideas in which groups of people are categorially described as one or the other (NASEM, 2023; Shim et al., 2014). For data collection, researchers should consider whether self-identification as multiracial is most relevant to their research question or whether measures of ancestry (self-reported, genealogical, or genetic similarity) should also be collected or assessed. At the analysis stage, how multiracial and multiethnic people are accounted for should be scientifically justified and related to the specific research question and potential mechanisms (Yao et al., 2021). For example, a study investigating contemporary racial discrimination as the mechanism of interest for a health disparity might consider using a categorization scheme that best reflects the diversity of experiences faced by multiracial people of different racial backgrounds (Franco et al., 2021; Gay et al., 2022; Harris, 2016). On the other hand, a study to investigate racial identity affirmation and belonging might group all multiracial individuals together, regardless of racial background, because threats to affiliation and belonging are consistently reported among multiracial individuals with a variety of backgrounds (Albuja et al., 2019, 2020; Franco et al., 2021; Nalven et al., 2021; Sanchez, 2010; also see Forthal, 2024 in Appendix C). In the absence of a specific hypothesized mechanism, or when multiple processes may be at play, researchers should consider comparing results using different categorization schemes. How studies define, include, or exclude their multiracial participants can change outcome estimates for other racial categories as well (Facente et al., 2022; Klein et al., 2019; Saperstein, 2009; Yao et al., 2022). For example, in a study of men living with HIV, whether estimates of the Black–White disparity in experiences of stigma were statistically significant depended on how multiracial participants were categorized (Facente et al., 2022; also see Forthal, 2024 in Appendix C).

To implement this approach and determine appropriate categorization schemes for their own work, researchers can consider the following questions:

• What mechanism may be driving the outcome of interest? How is multiracial ancestry or multiracial identity relevant to the research context?
• What is known about this mechanism or context for individuals with multiracial ancestry or identity? What does existing evidence suggest about which aspects of multiracial participants’ identity may be most salient?
• Which monoracial category or categories would likely have the most similar outcomes and experiences?

It is also important to weigh the drawbacks of past strategies for including multiracial or multiethnic participants (see Table 5-1). These include asking people who identify as multiracial to select their single “best race” to facilitate analysis and grouping

participants into subcategories defined by having or not having a White identity (e.g., White–Nonwhite, Nonwhite–Nonwhite). Treating White people as the norm against which all others should be compared has long been an unquestioned default in research across disciplines, but the selection of a reference category should be done intentionally and responsibly because it affects the interpretability of results (e.g., Johfre and Freese, 2021). Similarly, rather than asking multiracial people to simplify their self-identity down to a “best race”—which may cause discomfort and be less precise for analysis (Giebel, 2023; Jackson, 2023)—researchers should consider collecting additional measures such as perceived or reflected race (see Table 6-1) that may better align with their research question or hypothesized mechanism of interest. Whenever possible, it is preferrable to use or develop methods for including data from multiracial or multiethnic participants that preserve how they chose to identify themselves (see, e.g., the additive categorization scheme, which sorts respondents into every race or ethnicity category they select).

BUILDING COMMUNITY PARTNERSHIPS

Evaluating the Need for Community Engagement

Community partnership is essential to improving the appropriate use of race and ethnicity in biomedical research. However, the type of engagement best suited to a study will depend on the type of study, line of scientific inquiry, and community context. One way to categorize biomedical research is based on where it falls within the operational stages of translational research, as illustrated (Figure 6-2) in a report from the Institute of Medicine and its many antecedents (IOM, 2013). Much of the research at the T0 stage is basic and preclinical work using model organisms and cell and tissue lines, and the focus is on defining fundamental biological mechanisms, molecules, and pathways involved in health and disease. Biological mechanisms are universal among all humans (and often nearly universal among many types of organisms). Therefore, race and ethnicity are less directly relevant to basic science and preclinical research, and T0 studies have relatively low need for community engagement. Consideration should also be given to whether these study teams have the resources or expertise to support meaningful engagement that will ensure benefit sharing with the community participants. Although there are circumstances when early-stage research would benefit from and need community engagement, a lack of standard resources for and expertise in community engagement among basic science study teams could result in short-lived, superficial attempts at engagement that exacerbate rather than resolve problems. Some have characterized the related phenomenon called “health equity tourism” in which researchers pivot into health equity without the requisite depth of knowledge and commitment and, thereby, risk producing misguided or substandard work (Lett et al., 2022). For these reasons, the committee assessed how research contexts differ across the translational spectrum.

For types of biomedical research that fall in stages T1 through T4, community partnership needs to be considered and, in many cases, should be an integral part of the process throughout the research cycle (see Figures 6-1 and 6-2). At the T1 stage, there is a moderate need for community engagement. Careful consideration should be given to whether social questions, including race or ethnicity, may be relevant. Attention should be

given to recruiting diverse groups of participants to the extent possible even when working with small sample sizes. At stages T2–T4, community engagement or partnership is essential, as these types of research deal directly with human populations and questions that are likely to intersect with issues of race and ethnicity. Moving up the translational spectrum from T0 to T4, the type and dimensionality of community engagement will change, and the level of investment needed for meaningful engagement will increase.

Conclusion 6-8: Basic, preclinical, and proof-of-concept studies that seek only to interrogate a biological mechanism can, but need not, invoke questions of race and ethnicity. Regardless of this choice, representing human biological diversity, including in early-stage research, is essential to assure generalizability. Biomedical studies that involve human populations and that hold social and clinical implications necessitate a high degree of cooperative community engagement or partnership.

In addition to the guidance provided in Figure 6-2, researchers can answer the following questions to further evaluate the need for community engagement in their study. Answering “yes” to these questions indicates that community collaboration would benefit the study by integrating community-identified needs and priorities and helping to clarify the social and environmental determinants that provide the context for the study. Considering these questions can also help study teams begin to identify who comprises the community relevant to the research context.

• Is the disease or condition of interest disproportionately represented in some population (defined by exposure, geography, racial or ethnic background, etc.)?
• Do the specific aims of the study involve racial or ethnic communities who could benefit from the research?
• Are there plans to investigate social issues related to the biomedical topic at hand, either now or possibly in the future?
• Could these data be (re)used to address research questions related to health disparities in racial or ethnic populations?

The Value of Community Engagement and Partnerships

The existing guidance for engaging and partnering with communities, which was presented in Chapter 4 and Table 4-1, is a solid foundation for biomedical researchers to work from. Importantly, community engagement is a collaborative process between the research team and the people who will be affected by the study and its outcome. It is rooted in a set of scientific and ethical principles (see Chapter 2, section “Guiding Principles”) that guide the interactions, communication, decision making, and organization of the partnership. Given how race and ethnicity are intertwined with people’s identity and experience, collaborative engagement at every stage of the research process is essential, as it can provide researchers a key perspective and deeper understanding about context that is vital for the use of race and ethnicity in research.

Information gathered by the committee also emphasized the importance of true partnership throughout the entire research cycle. Partnership in research means “we [community members and researchers] make decisions together. Community members have

a voice at the table. They are part of the decision-making. It’s a collaborative where it’s not just me getting an email, and the claim being made that I’m part of the team. It’s real work. We’re academics, folk in practice, and community. And when I say community, I’m talking grassroots folk” (Ella Greene-Moton in remarks to the committee on March 14, 2024). Members of the community can play a role in identifying and prioritizing research questions that are important to them. A speaker emphasized, “I think that at the beginning of research, even the questions, let’s formulate them together” (Gladys Vega in remarks to the committee on March 14, 2024). Community members and representatives can also give valuable input on designing protocols that meet their needs and that account for the barriers they face to participation and provide insight and networks to aid in the dissemination of research results.

Forming partnerships with community leaders and members requires patience, time, funding, and expertise. It is important to understand the pros and cons of different approaches to community engagement (see Table 4-1) and to recognize that more extensive community engagement plans may require a longer study timeline. Research teams may need a community engagement expert who can facilitate meetings between community leaders and research team members, identify challenges, develop strategies for building trust and respect, develop an engagement or partnership plan, manage that plan, and handle logistics.

Some communities or populations may have unique needs, preferences, or requirements. Approaching Tribal nations could involve more formal contact and entrée for research purposes than typically needed in working with other communities (see Chapter 4, section “Collaborating with Indian Tribes on Biomedical Research Studies”). A liaison could help navigate these complexities.

To maintain existing relationships, as well as build new ones, research entities must be involved in ongoing relationship-building activities to cultivate a consistent presence in surrounding communities of interest. Funders are also key partners in creating opportunities and support for in-depth community participation in research. They can structure funding allocations and reporting timelines that provide the money and time needed to build effective and lasting partnerships. However, as discussed in Chapter 4, funding models are frequently not supportive of sustained community engagement. Further, many funders do not allow for community partners to serve as co-Principal Investigators (PIs) on studies, and instead involve them as sub-awardees on grants. This model inherently does not allow for an equitable partnership between researchers and community members. Although outside of the scope of this report, this is an area that requires careful consideration and change to advance this important work. The committee makes the following recommendation:

Recommendation 7: Researchers collecting and using race and ethnicity data in biomedical research with human populations should identify and partner with specific communities relevant to the research context. Researchers should collaborate with community engagement experts and organizations and, to the greatest extent possible, partner directly with community members to optimize authentic, continuous, and sustained researcher–community member engagement undergirded by mutual trust.

• From the earliest stages of the project, these partnerships should be established to inform hypothesis development and study design, including how

• race and ethnicity information should be collected and used, through results interpretation and dissemination.
• Research teams should communicate potential benefits to community partners from project initiation through results dissemination.
• In the case of secondary data use, researchers should consult documentation or original investigators from participating studies to understand how communities were involved in the process.

IMPLEMENTATION OF THE REPORT’S RECOMMENDATIONS

Aware of the broad scope of biomedical research, the committee developed an approach that can be applied in many biomedical contexts and outlined general considerations for the use of race and ethnicity across the research process. However, implementing the foundational concepts and recommendations will differ across disciplines and studies, as the specific context varies. To help operationalize these recommendations, the committee presents a few hypothetical research scenarios that contextualize the recommendations. Following these examples are some unique considerations applicable to basic science, industry (e.g., pharmaceutical development), artificial intelligence, and the development of medical devices. Closing out the chapter are strategies for operationalizing the recommendations that are directed to everyone in the biomedical research community.

Research Scenarios

The following are hypothetical scenarios that researchers may encounter. They serve as a thought exercise for how the recommendations could apply in different contexts.

Scenario 1

MK is an established PI at a large research center at an urban R1 university, with a network of many potential collaborators. One of her colleagues has expertise in qualitative research methods and community outreach and is her co-PI on a new National Institutes of Health (NIH) grant about a health condition. Given the specific aims to study this health condition, the team wants to ensure the work will benefit the local community. Before enrolling participants, MK’s co-PI holds a series of listening sessions at local churches and other community centers. As a result, they modify their study objectives and decide to also measure social determinants of health and environmental exposures to better understand and contextualize the disease of interest.

MK’s prior research has primarily been funded through NIH grants, and in the past her team has primarily used the NIH planned enrollment table (based on the 1997 OMB categories) to determine their data collection strategy. For this new study, the team decides a priori that they intend to make some between-group comparisons among racial and ethnic groups, so instead of collecting a representative sample, the team plans to oversample smaller groups to ensure sufficient statistical power. For data collection, MK’s team is planning to use the 2024 OMB categories, including the more detailed subcategories. In addition to collecting self-identified categorical race and ethnicity data for NIH grant reporting, MK’s co-PI assesses other dimensions of diversity, including

age, sex and gender identity, income, and educational attainment, and will include open-ended prompts for qualitative analysis such as: What words or terms come to mind when I ask you to identify yourself? To study interpersonal experiences with racism, they also collect reflected race.

Throughout the study, MK and her team continue to assess how race and ethnicity are being used and to update community members. Afterward, the team shares results with the community and integrates lessons learned for the next study.

See: Recommendations 1, 2, 4, 6, 7; Table 6-1; Table 6-2; Box 6-1

Scenario 2

DW is a new assistant professor in biostatistics at a mid-size research institution. With their team, DW would like to create an algorithm-based risk calculator that uses risk factors to guide decision making about additional testing in patients with a suspected disease condition. Evidence in the literature suggests that race may be a (noncausal) risk factor for the condition.

DW is using an EHR dataset from a local hospital system, and they have access to clinical indicator data as well as demographic information. Before starting to develop the algorithm, DW conducts some initial analysis to become familiar with the dataset, understand its limitations, and evaluate potential biases. They find a number of limitations, including a high proportion of missing race and ethnicity data, that they note for any later manuscripts. They also identify a group of multiracial individuals and test out different categorization schemes that could be most appropriate for including them in analysis.

To optimize the algorithm, DW tests various parameter sets, both with and without race and ethnicity. In early versions, it appears that including race and ethnicity variables may improve the performance of the algorithm, but they find that another parameter set without race or ethnicity performs equally well. In this case, DW concludes that it is unwarranted to use race and ethnicity as proxies for unknown social factors and decides against using them in the algorithm.

Even so, DW conducts a thorough subgroup analysis for the racial and ethnic groups included in the dataset, including a multiracial group, and finds that the algorithm performs similarly across different groups. When DW publishes the results of their algorithm, they detail the rationale for their decisions regarding race and ethnicity in the manuscript and also makes the information available to other researchers via a repository.

See: Recommendations 1, 2, 3, 6; Box 6-1

Scenario 3

JR is a postdoctoral fellow analyzing factors associated with disease severity. The existing dataset has information about relevant physiological traits as well as demographic data, including sex, age, and self-reported race. JR finds a correlation that suggests race could be a risk factor for disease severity. He knows that causal inferences cannot be made from a correlation with race and that caution should be exercised in

reporting race as an association in this context. In discussions with his PI, the two of them decide to include the result in their manuscript and acknowledge the limitations, stating the following:

Our results suggest an association between self-reported race and disease severity. Nonetheless, this result should be interpreted with caution. While race may be correlated with increased disease severity, there could be a variety of plausible explanations, including undetermined physiological factors or social influences such as structural racism, and further research is needed to elucidate the underlying mechanism. We hypothesize that XYZ could be contributing to the result. Of note, a limitation of the dataset is a lack of information about social and environmental factors to better contextualize the results.

When JR goes on to present his findings at a conference a few months later, he makes sure to clearly articulate the limitations of their study design and to emphasize that race and ethnicity are not causative factors for the disease outcome they examined. In his future studies, he learns to collect and use datasets with more nuanced measures and stays away from searching for arbitrary correlations without rationale.

See: Recommendations 1, 2, 4; Box 6-1

Basic Science and Early-Stage Biomedical Research

Driven by curiosity about how biological systems work, basic science expands the foundation of scientific knowledge and sometimes leads to unexpected breakthroughs and applications in medicine. Basic science research seeks to reveal fundamental biological mechanisms and can give rise to hypotheses about health and disease that can inform applied research and may translate into clinical treatments. Basic biomedical research commonly uses a variety of informatic, imaging, biochemical, biophysical, molecular biology, and immunology techniques, along with cell culture for in vitro studies and animal models (e.g., mouse) for in vivo work. Early-stage preclinical studies are designed to show preliminary proof-of-concept to begin to translate discoveries from the bench to humans. These studies are generally small in scale and can look quite different from clinical trials, which must meet specific criteria to test treatments or interventions in humans. Even small Phase I clinical trials prospectively assign participants to different groups to evaluate the effect of an intervention on a health-related outcome (NIH, 2017). In contrast, basic discovery research is not subject to the same criteria. Preclinical studies sometimes involve secondary research with human tissue, such as harvested cells, biopsies, or post-mortem samples, to explore a biological mechanism, characterize a condition, demonstrate feasibility, or validate a hypothesis.

In considering the translational spectrum of biomedical research (Figure 6-2), the committee acknowledges that race and ethnicity may be less directly and immediately relevant to basic science or early-stage research that seeks only to examine biological mechanisms (see Conclusion 6-8). Indeed, fundamental molecular and cellular mechanisms are often shared across species and are the same across racial and ethnic groups. Though there are epidemiological differences in disease prevalence, and in the

many potentially salient social or environmental factors that may contribute to them, this does not mean that the underlying biological mechanisms necessarily differ across groups. For example, rates of cardiovascular disease may vary across populations, but the cellular mechanisms and biological sequelae are generally the same for everyone, though genetic variation may exist.

Although it could be easy to rely on the universality of biology to say that race and ethnicity have no role in the context of basic science or preclinical studies, it is worth considering what may be missed by failing to consider race and ethnicity at all in early-stage research. It is nonetheless important for basic science investigators to consider how race and ethnicity may have influenced past evidence that was gathered in this area and whether race and ethnicity could be relevant to applications of this line of research in the future. Even in basic science, decisions about whether to use race and ethnicity should be deliberate, and if a future application of this research could invoke social context, it is important for researchers to start thinking about the potential implications of these social dimensions early in discovery science. Building awareness of potential implications of race and ethnicity in early-stage research could catalyze progress in subsequent translational stages of research.

One approach is employing racism-conscious research (see Chapter 5, section “Moving Toward Racism-Conscious Research”) early in research. At study design, investigators can ask how race and ethnicity may influence or intersect with some attribute(s) of this research study, either now or in the future (Figure 6-1). It may not be apparent how race and ethnicity could be relevant when working with animal models; however, for example, most animal models have low-pigment skin, so using these animals to design optical sensors or study mechanisms that could be influenced by skin pigmentation could lead to bias when the results are translated to humans with highly pigmented skin. As another example, mouse models are often modified with human genes for mechanistic studies. Even though the studies are in mice, differential distributions of alleles in human populations that correlate with ancestry may be relevant when genetically modifying mice. Even if race and ethnicity are not immediately pertinent, it is valuable to understand potential implications early on.

Many basic science study teams are small and have limited grant-based funding, posing additional challenges to adopting some of this report’s recommendations. These smaller scale, discovery-based studies may lack the funding or expertise to develop extensive community engagement plans or to address questions about social context, but investigators can reach out to collaborators and other partners for input and guidance. These challenges also highlight the importance of assembling diverse, interdisciplinary study teams from the beginning of a project to help identify potential blind spots in the research question or study design early in the process. Despite the practical limitations of small-scale basic science, investigators can be thoughtful about study design choices and acknowledge the limitations of their approach. To be successful, efforts to root out the effects of biased data and analysis will need to be included in the earliest stages of research, not only at clinical applications. Building awareness of possible connections between basic science and race and ethnicity can train investigators to pause before beginning a study and to be watchful for biased assumptions and evidence.

Considerations for Industry

Recommendations in this report also apply to industry-sponsored research. For industry, engaging with communities and advocacy groups remains an important way to build trust, enhance awareness, and provide education and outreach about the drugs and treatments they are developing. Understanding the populations most likely to be affected with a disease could help identify potential barriers to participating in clinical trials so that the trial design could better serve communities by addressing contextual and socioeconomic factors. For example, patients may be willing to participate in a study, but their work schedule for their livelihood could prevent them from attending follow-up visits. Alternatively, time off work may not be an option for some participants, or they may not have transportation to a distant trial site. In response, industry is increasingly undertaking decentralized trials, which have the potential to increase engagement among underserved communities. Technology may facilitate decentralized clinical trials, making recruitment more efficient and easing the burden of follow-up for participants; however, these strategies are not without drawbacks. Care should be taken to ensure that patients are not excluded due to lack of access to internet or mobile devices, digital illiteracy, rural residence, or socioeconomic status.

A couple of factors make industry-sponsored research unique. First, industry-sponsored research is regulated in the United States by the U.S. Food and Drug Administration (FDA) (see Chapter 4). Historically, private companies, though they may be committed to addressing race and ethnicity in research, have not been able to resolve such issues as a lack of diversity and inclusivity alone. To overcome this challenge, systemic support will be needed across regulatory bodies, industry, and public–private partnerships. The committee concluded:

Conclusion 6-9: FDA is a powerful regulator and, via industry guidance, heavily influences the use of race and ethnicity in industry-sponsored research for inclusion purposes.

Second, industry often undertakes research on a global scale. Although this report focuses on U.S.-based constructions of race and ethnicity, it is important to recognize the complexity of these issues in an international context. Racial and ethnic descriptors vary greatly among countries. In different countries, race and ethnicity labels are frequently conflated with each other and with categories like nationality, citizenship, caste, tribe, or dialect group (Morning, 2008). In addition, similar terms have different meanings to people in different countries. Large-scale clinical trials are typically international, so issues of race and ethnicity cross borders. Data collection practices and privacy laws vary by country, creating variance that may be a barrier to the harmonization of racial and ethnic group categories. The extent to which major international regulatory authorities can work together will facilitate progress.

Although not unique to industry, mistrust is a particular barrier that must be overcome to improve inclusion in private–sector biomedical research and development. Among racial and ethnic minority groups, there is often mistrust of the health care system and medical research due to past harms, safety issues, or unclear public

health communication (Pahus et al., 2020; Scharff et al., 2010). The biopharmaceutical industry faces further mistrust due to public perceptions of the profit motive inherent in drug development and perceived conflicts of interest. These dynamics create a negative cycle where lack of diversity in clinical trials compounds existing mistrust, increasing reluctance to participate in trials among racial and ethnic minority populations.

It should be noted that some evidence suggests the contrary—that underrepresented populations are just as willing to participate in research as those who identify as White (Wendler et al., 2006), but some racial and ethnic groups may be underrepresented in trials because they are not as frequently invited to participate, have more limited access to certain health care resources and institutions, or face greater obstacles (e.g., childcare and transportation) to taking part (Fisher and Kalbaugh, 2011; Wendler et al., 2006). Supported by sustained investment across the health care landscape, proactive communications about the benefits of and safeguards relating to clinical trial participation could begin to overcome trust deficits. Gaining community trust through partnership could build a foundation for improving representation in industry research and development.

Considerations for Biomedical Applications of AI

Efforts to employ clinical algorithms and other decision-making tools across the health care ecosystem in a way that maximizes benefits and reduces potential harms are hampered by the need for more guidance on the development, assessment, and implementation of these tools. As discussed in earlier chapters,⁴ there are some guidelines available from government organizations and professional societies regarding both statistical and AI-based clinical algorithms. However, the existing AI-specific guidance provides little commentary directly on the use of race and ethnicity in algorithms. In addition, much of this available guidance is based on assessments of algorithms that are already in clinical use for the purpose of mitigating deleterious effects on health outcomes, particularly for racial and ethnic minority groups. As a result, the existing guidance is often insufficient to address the numerous pitfalls that can accompany both the design and application of these tools in clinical practice (Cary et al., 2024; Jain et al., 2023).

The application of clinical algorithms, especially AI-based ones, in medicine is rising, and the need to ensure they are safe, fair, and transparent is increasingly recognized. While more is needed in terms of standardized policies, particularly at the federal and state level, to identify and implement best practices for developing, testing, and implementing clinical algorithms to promote equitable care, new guidelines provide some standards for algorithmic developers. FDA issued guidance broadening oversight of clinical-decision support tools, including algorithms (FDA, 2022). In 2023, the agency published a draft guidance on AI and machine learning-enabled device software functions (FDA, 2023); updated guidance is anticipated in 2024 (Morris and Sharma, 2024). In addition, a recent rule issued by the U.S. Department of Health and Human Services underlines the importance of addressing racial bias in clinical algorithms as part of broader nondiscrimination protections in health care. The rule stipulates that health

___________________

⁴ For an overview of existing guidance for AI in clinical algorithms, see Chapter 4, section “Guidance for Race and Ethnicity in Clinical AI Algorithms.”

care entities must actively identify and mitigate discriminatory effects, including race-based bias, when using AI and other decision-support tools (HHS, 2024a; HHS, 2024b).

Analytic strategies targeted at various points of the algorithmic life cycle are emerging to mitigate racial and ethnic bias in AI algorithms (Cary et al., 2024). Strategies include collecting higher-quality data from representative patient groups and carefully characterizing any biases in secondary datasets, which are subject to limitations of missingness, inaccuracy, and bias (see Chapter 3, section “Race and Ethnicity in Secondary Data Analysis”). When developed from datasets with these limitations, AI algorithms can have cascading negative consequences for racial and ethnic minority groups by increasing the potential for misdiagnosis or late diagnosis and restricting access to lifesaving medical interventions. For this reason, it is important for researchers to carefully consider the provenance of the data they use in their studies and assess any potential sources of bias along with other limitations (see Box 6-1 and Recommendation 2). However, one obstacle to assessing bias when dealing with AI tools is that algorithms are often proprietary, preventing individual researchers from digging into their inner workings without employing creative workarounds (Obermeyer et al., 2019). Lack of access to an algorithm’s training data and detailed methodology also limits opportunities to understand the mechanisms behind any disparities in its performance, especially across different racial or ethnic groups (Obermeyer et al., 2019).

Several tools can be used to assess bias in datasets. The METRIC framework is a framework for assessment of the quality of biomedical data comprising 15 dimensions that enable AI algorithmic developers to know about biases that may affect fairness of algorithms (Schwabe et al., 2024). The Racial Bias in Data Assessment Tool provides an evidence-based assessment tool for assessing the risk of racial and ethnic bias in datasets for secondary analysis (Chapin Hall, 2024). Algorithm design and development strategies focus on the selection of the predictor variables, the outcomes and methods used for deriving the algorithm. If a dataset is assessed to be biased, mitigation techniques may include data weighting or sampling methods to make it more representative of the population in which it would be used. Derivation of the algorithm can be optimized for fairness by adjusting the method to incorporate some aspects of fairness in addition to statistical fit to the data, such as ensuring that error rates across racial groups are similar when minimizing overall error (Ghassemi, 2024). After an algorithm is implemented, monitoring measures include its impact on treatment allocation and health outcomes over time to ensure they do not deteriorate. More generally, development of clinical algorithms should investigate and track anticipated pitfalls of using race and ethnicity throughout the algorithmic life cycle from the design phase to implementation and monitoring (see Chapter 4, section “Guidance for Race and Ethnicity for Clinical AI Algorithms”).

Algorithmic biases sometimes stem from a lack of understanding of domain-specific clinical and social aspects, and one solution is to enhance statistical and AI expertise with social science, health disparities, and clinical expertise to bring a multidisciplinary approach to algorithmic development (Cary et al., 2024). Social scientists typically consider bias as inclinations in human thinking and reasoning that must be addressed during algorithmic design. Health disparities researchers and clinicians see bias as a contributing cause to health care inequities in access, allocation, and outcomes.

For example, an algorithm analyzed by Obermeyer and colleagues (2019) had racial biases because differential access to health care means that health care cost is a poor proxy for health. This is well-known in health disparities research, and algorithmic bias could be mitigated if health disparities researchers were involved in developing this algorithm. Thus, a multidisciplinary approach could help to identify, assess, and mitigate biases more effectively (e.g., Joyce et al., 2021).

The guidance provided in this section is necessarily brief since the application of AI in biomedical research is an emerging and dynamic area. However, the committee acknowledges that AI technologies, especially generative AI that creates new content (e.g., text, images, and audio) that mimics human-generated content, are rapidly evolving and will certainly be used in biomedical research and clinical algorithm development. Some are optimistic that AI can be used to promote equity (Pierson et al., 2023). Yet, the deep learning technology that underlies recent advances in generative AI has significant drawbacks—notably a lack of interpretability, which can be problematic in applications where understanding the reasoning behind decisions is crucial, such as in health care (Clusmann et al., 2023; Tian et al., 2023). Furthermore, the huge volume and variable quality of data utilized in the development of generative AI are likely to conceal racial and ethnic data biases (see Box 4-3), which can be difficult to detect and overcome (Omiye et al., 2023).

Development of Medical Devices

Medical devices are approved through different regulatory pathways than traditional pharmaceuticals. The FDA has long used the 510(k) pathway to bring new medical products to market, and this has been the dominant pathway for medical devices (Aboy et al., 2024). In recent years, most medical devices with an AI or machine learning component have been cleared through the 510(k) pathway (Benjamens et al., 2020; Muehlematter et al., 2023; Reuters, 2023). The 510(k) pathway provides clearance for devices that have proven to be “substantially equivalent” to a device that has already been approved, also known as the predicate (Muehlematter et al., 2023). One challenge with this approval pathway has been the variable interpretation of substantial equivalence, leading to the approval of several generations of devices with claims of equivalence to each other, thereby straying further from the original predicate and creating safety concerns for patients (Muehlematter et al., 2023).

Suggestions to improve this complex pathway for all parties have long been documented. A 2010 workshop hosted by the Institute of Medicine, for example, provided perspectives from industry associations as well as academic and patient organizations on improvements to this clearance process to benefit patient safety (IOM, 2010). Suggestions provided by workshop participants included better educating the American public about the premarket approval process, creating robust guidance for the FDA to better exercise its regulatory authority, and mandating the reporting of adverse events when using these new devices (IOM, 2010). With the increased use of AI methods in these products, there is also a need for more scrutiny on these components when determining substantial equivalence with older predicates to improve care and mitigate patient harm (Muehlematter et al., 2023). Furthermore, many of the predicates against which new devices are assessed are often built on legacy data that may now be considered outdated or

problematic. Thus, there is a need for more guidance (see Recommendations 2 and 3) on how to reconcile with the use of these inequitable devices that have not yet been recalled.

Another persistent challenge accompanying the development of many new medical devices is the issue of small sample sizes (see Recommendation 5) for safety group testing, which often lacks representativeness and results in insufficient statistical power to do robust subgroup analyses. For example, when evaluated with separate equal-sized subgroups of lightly and highly pigmented skin, many pulse oximeters exhibit inaccuracies detecting hypoxia in people with darker skin (Bickler et al., 2005; Feiner et al., 2007). These discrepancies and inequitable device performance have contributed to disparities in health outcomes, such as during the Covid-19 pandemic (Fawzy et al., 2022). Legal scholars have also argued that continuing to utilize these devices in clinical care settings may open health care institutions up to potential legal liability by potentially violating a recently finalized health care antidiscrimination law⁵ (Kupke et al., 2023). The equitable design of medical products is compromised when population subgroups within safety test groups are too small to have statistical power, such as those currently used for device testing. Therefore, the committee concluded:

Conclusion 6-10: There is a need for more guardrails to ensure that health technologies meet federal safety thresholds for all clinically relevant subgroups, not only a given assessment group overall.

One potential way to enforce such standards is to have payers reimburse for only the devices and algorithms that meet these requirements. A recent UK report emphasized the importance of transparency around the performance of medical devices across subgroups with different skin tones (UK Department of Health and Social Care, 2024). Moreover, fostering expertise on the collection and use of race and ethnicity data among device engineers, regulators, and industry partners in this space could also be useful when it comes to better integrating this information into the development of these devices (Ferryman et al., 2023). Given that equity issues encoded into medical technologies have many facets, efforts to understand and address them will benefit from sustained and in-depth conversations across disciplines and among industry, academia, and the general public.

Strategies for Operationalizing the Recommendations

Adopting this report’s recommendations will require coordinated efforts across the biomedical research ecosystem as well as a willingness to be patient and flexible. This report is only the start, and the conversation must not stop here. Addressing the complex issues inherent in how to use race and ethnicity thoughtfully in biomedical research will require sustained and in-depth conversations across disciplines and sectors. It will also involve increasing awareness of the problem itself and the potential solutions. Those from academia, industry, regulatory bodies, funders and sponsors, and

___________________

⁵ Federal Register. 2024. Nondiscrimination in health programs and activities. https://www.federalregister.gov/documents/2024/05/06/2024-08711/nondiscrimination-in-health-programs-and-activities (accessed August 28, 2024).

communities will have to work together to implement the recommendations. Unfortunately, few spaces for this collaborative dialogue currently exist, but creating crosstalk across sectors and disciplines can promote adoption of these recommendations.

The National Academies population descriptors report (NASEM, 2023) used similar implementation strategies and serves as a good case study. Since that report was published, journal editors from seven major journals⁶ published a joint statement advocating the recommendations in the National Academies report and providing 10 precepts of their own built from the National Academies report’s findings, conclusions, and recommendations (Feero et al., 2024). Since its release, the report has generated conversation among key actors in the field of genomics as it has been discussed at conferences, including the annual meeting of the American Society of Human Genetics, and a recent NIH workshop.⁷ In late 2023, 11 institutes and two offices at NIH posted an ethical, legal, and social implications research funding opportunity, which cited the population descriptors report and called upon “applicants who propose to address or analyze race, ethnicity, genealogical ancestry or genetic ancestry are strongly encouraged to review” the 2023 National Academies report when designing their research strategy (NIH, 2024).

Recommendations 1 through 7 in this report focus on actions that researchers should take. This section speaks to other actors in the biomedical research ecosystem. Notably, many of these actions should be done in collaboration with other actors and in partnership with researchers for sustained implementation of the report recommendations.

Journal Editors

To disseminate and build awareness of this report’s recommendations, journal editors could develop special issues with invited articles about topics such as illustrative examples of implementing reporting recommendations, fostering community partnerships in biomedical research, and measuring social and environmental variables that affect health and disease. Journals can also take action by implementing guidelines for what is expected in manuscripts submitted for publication. The American Journal of Human Genetics, for example, stated its support for the recommendations in the NASEM population descriptors report on its “Information for Authors” webpage⁸ describing its policies and manuscript requirements. As another example, JAMA provided guidance on reporting race and ethnicity in medical and biomedical journals a few years ago (Flanagin et al., 2021). Based on this article, professional societies, like the American Society of Nephrology, have adapted the JAMA guidance and created checklists for researchers, clinicians, and editors to use when submitting and reviewing manuscripts. Checklists can be a helpful way to operationalize publishing guidelines, but such tools need to be disseminated to both authors and reviewers and used consistently to have the most

___________________

⁶ Journal of the American Medical Association (JAMA), Genetics in Medicine, American Journal of Medical Genetics, Nature Genetics, Human Genetics and Genomics Advances, Journal of Genetic Counseling, and American Journal of Human Genetics.

⁷ https://obssr.od.nih.gov/news-and-events/events/population-descriptors-legacy-genomic-data-challenges-and-future-directions (accessed July 26, 2024).

⁸ https://www.cell.com/ajhg/authors (accessed July 26, 2024).

impact. American Heart Association Journals, for example, include a checklist organized by article subsection (e.g., methods, results) for reporting racial and ethnic disparities.⁹

While it is helpful for individual journals to have policies and guidance for prospective authors about race and ethnicity in articles, true change will be effected the more that journals can come together to develop consistent reporting guidelines, including across disciplines. For example, the International Committee on Medical Journal Editors (ICMJE), which is a group of medical journal editors that convenes to improve the quality of medical science and reporting, could provide more information or host a convening for interpreting their guidance on race and ethnicity. ICMJE recommendations currently state that authors should define how they determined race or ethnicity and provide a justification for this use. Their recommendations also state that “race and ethnicity are social and not biological constructs; authors should interpret results associated with race and ethnicity in that context” (ICMJE, 2024). Although this recommendation is in alignment with the messages of this report, more detailed guidance and illustrative examples of how to do this successfully could be helpful for a broader understanding and adoption of this report’s recommendations.

Some journals have had guidelines around the use of race and ethnicity for some time, but the guidance and level of detail vary across publishers and disciplines. In addition, having a policy is not sufficient if it is not consistently applied and monitored. As discussed at length in Chapter 4, despite some existing guidelines, race and ethnicity are not reliably defined in publications or used in consistent, rigorous ways (see, for example, Martinez et al., 2023). More accountability and consistent enforcement of guidelines, throughout the publication process, are needed to make additional progress.

Professional Societies

To increase awareness of the problems with many current uses of race and ethnicity and to help biomedical scientists more appropriately use race and ethnicity in their research, professional societies can serve an important role as conveners and educators. Professional societies can hold workshops, provide online training modules, commission surveys and reports, and write articles that highlight the ways in which biomedical research can make lasting changes. For example, the Society of General Internal Medicine (SGIM) published an article on how to integrate anti-racist principles into the research process, which includes recommendations for using race data throughout the research life cycle (Gonzalez et al., 2024). Moreover, professional societies can also develop example case studies and offer sample questions for researchers asking participants about their racial and ethnic identification, which can serve as educational tools and provide best practices for those engaging in this work. (See section, “Research Scenarios,” for examples of case studies).

Professional societies can also revise existing courses and training modules specific to proper study design and conducting reproducible and rigorous science to include information about this report and ways to implement the report’s recom-

___________________

⁹ See https://www.ahajournals.org/disparities-research-guidelines and https://www.ahajournals.org/pb-assets/policies/AHAJournals_DisparitiesGuidelines_Checklist.pdf (accessed August 21, 2024).

mendations, such as additional research scenarios like those described in this chapter. For example, the Collaborative Institutional Training Initiative (CITI) Program has a course on research study design that provides an overview of research study design and approaches to optimize the reproducibility of research results.¹⁰ CITI could ensure that this or similar modules and courses provide a framework and tools for researchers to learn about that will help them understand how to use race and ethnicity appropriately in their research.

Professional societies can also act as partners in disseminating the report by hosting panel discussions, webinars, workshops, and podcast episodes about the recommendations and including presentations in upcoming conferences to ensure that the conversation continues. These types of meetings and conferences also provide a venue for bidirectional exchange of ideas among those who are more senior in the field and earlier career individuals, which is key for setting the foundation for sustained change.

Funders of Biomedical Research

Funders of biomedical research can host workshops and conferences to raise awareness about the considerations for using race and ethnicity in the research that they support. For example, the Doris Duke Foundation has co-hosted meetings with the National Academy of Medicine in this regard to foster discussions among their clinical algorithm grantees, and NIH has hosted workshops with scientists working on population descriptors to share and discuss approaches to using population descriptors for legacy datasets.

Funders could also incorporate material from this report into available or required training opportunities. For example, key messages of this report could be incorporated into responsible conduct of research (RCR) training, which is a required course for all NIH intramural investigators and for all trainees on an NIH institutional research training grant or fellowship. RCR courses include instruction on proper data acquisition and ethical data use, which could include the principles outlined in this report on appropriate collection and use of race and ethnicity data.

Funders can examine the guidance in this report and change or update their policies on how they would like to see grantees think about and use (or not use) race and ethnicity in research. Requesting articulated scientific rationales and approaches could ensure that researchers are held accountable for considering how race and ethnicity are used at the study design phase. This increased transparency may be particularly salient for funders of health technologies, which, as discussed earlier in this chapter, commonly rely on secondary datasets. Funders can also change policies to hold investigators accountable for properly engaging communities in the research life cycle and ensuring that researchers are building partnerships with relevant communities to inform their research design.

Finally, for researchers to build relationships and do the necessary community engagement outlined in Recommendation 7, funders of biomedical research need to recognize the increased time and resources it takes to engage and build lasting relationships with

___________________

¹⁰ https://about.citiprogram.org/course/research-study-design-rsd/ (accessed July 26, 2024).

racial and ethnic minority populations in biomedical research. As described in a recent National Academies report, “Investment of time and resources are needed to build and restore trust with underrepresented and excluded communities” (NASEM, 2022, p. 7). Federal research awards typically are for a period of 3 to 5 years, which is meant to encompass all aspects of the research life cycle. However, as discussed in Chapter 4, these timelines often do not align with the time and resources it takes to build and sustain relationships with racial and ethnic minority communities. This may be particularly true for research conducted with Tribal nations, which, as discussed in Chapter 4, requires approval and partnerships with sovereign Tribal governments. The committee provides the following conclusion and recommendations for funders and publishers of biomedical research:

Conclusion 6-11: Funding timelines and publishing pressure often do not account for, and are thus misaligned with, the time required for outreach to and partnership with racial and ethnic minority populations for participation in research. Reservation-based research, for example, requires longer timelines for approvals to navigate the unique legal and political status of Tribal nations.

Recommendation 8: Funders, sponsors, publishers, and editors of biomedical research should provide consistent guidelines to assist researchers in developing and examining their work and to promote the thoughtful use of race, ethnicity, and related concepts to enhance adoption of these recommendations.

• Journal publishers and editors, research funders, and sponsors should require researchers to provide a scientific rationale for their use of race and ethnicity, describe data provenance, and acknowledge limitations of their use.
• Journal editors and funding agencies should provide reviewers with specific guidelines for reporting race and ethnicity that should be used to assess publication and funding decisions.
• Funders of research to develop health technologies should require researchers to report results across racial and ethnic groups and encourage researchers to provide datasets, algorithms, and code in an open-source format to the greatest extent possible.

Funders, sponsors, publishers, and editors of biomedical research should periodically evaluate their policies on the use of race and ethnicity to assess the extent to which the policies are followed and upheld, monitor progress, consider the need for updates, and ensure the guidelines reflect current best practices.

Recommendation 9: To support partnerships between communities and research teams, funders and sponsors should require as appropriate a community engagement plan as part of the application. Funders should provide resources and timelines that encourage researchers to build and sustain collaborations. Research institutions, medical centers, and other biomedical research organizations should develop and support lasting, equitable relationships with community partners.

This report offers ways to change how race and ethnicity are used, analyzed, and reported in biomedical research. When implemented, these changes have the potential to improve the scientific rigor of biomedical research, mitigate bias that continues to affect research and health care, and build lasting trust among the scientific community and racial and ethnic communities. However, issues of race and ethnicity remain complicated and challenging to address. As this report has emphasized, appropriate or inappropriate use of these constructs is context dependent. Because next steps will differ depending on the context of the research, all participants in the biomedical research ecosystem have a role to play in operationalizing the recommendations. Individual researchers and clinicians can do their part by scrutinizing their decisions to use—or not use—race and ethnicity throughout the research process, assessing whether study results and clinical tools were developed based on robust evidence and making thoughtful inferences and interpretations of the evidence.

While putting these recommendations into practice across the biomedical research ecosystem will take time and effort, the report represents a vision of the future where the biomedical community moves beyond the limits of focusing on race to adopt practices that will facilitate an understanding of the true factors of disease.

Conclusion 6–12: The biomedical research enterprise has long emphasized race at the expense of exploring other concepts such as racism and discrimination that are known to have more direct effects on health. Much of the existing evidence base has deep-rooted bias and requires reexamination. Rebuilding the evidence to examine the role of racism and other associated concepts beyond race and ethnicity categories will require investment from funders and sponsors of biomedical research.

Moving forward starts with recognizing and acknowledging assumptions, biases, and flaws in the existing evidence base. Yet making progress does not have to be daunting. It is an exciting time for the biomedical research community to chart a path forward to improve the use of race and ethnicity for better science and better health.

REFERENCES

Albuja, A. F., D. T. Sanchez, and S. E. Gaither. 2020. Intra-race intersectionality: Identity denial among dual-minority biracial people. Translational Issues in Psychological Science 6(4):392–403.

Aronson, J. K., and R. E. Ferner. 2017. Biomarkers—a general review. Current Protocols in Pharmacology 76(1):9.23.21–29.23.17.

Asad, A. L., and M. Clair. 2018. Racialized legal status as a social determinant of health. Social Science & Medicine 199:19–28.

Benjamens, S., P. Dhunnoo, and B. Meskó. 2020. The state of artificial intelligence-based FDA-approved medical devices and algorithms: An online database. NPJ Digital Medicine 3(1).

Bickler, P. E., J. R. Feiner, and J. W. Severinghaus. 2005. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology 102(4):715–719.

Bingham, P., N. Q. Verlander, and M. Cheal. 2004. John Snow, William Farr and the 1849 outbreak of cholera that affected London: A reworking of the data highlights the importance of the water supply. Public Health 118(6):387–394.

Blankenship, K. M., A. Rosenberg, P. Schlesinger, A. K. Groves, and D. E. Keene. 2023. Structural racism, the social determination of health, and health inequities: The intersecting impacts of housing and mass incarceration. American Journal of Public Health 113(S1):S58–S64.

Blumberg, R. S., B. Dittel, D. Hafler, M. Von Herrath, and F. O. Nestle. 2012. Unraveling the autoimmune translational research process layer by layer. Nature Medicine 18(1):35–41.

Brown, T. H., and P. Homan. 2024. Structural racism and health stratification: Connecting theory to measurement. Journal of Health and Social Behavior 65(1):141–160.

Burkhardt, T., L. A. Huang, R. Kakuyama-Villaber, A. Pacheco-Applegate. 2021. Racial Bias in data assessment tool. https://www.chapinhall.org/wp-content/uploads/Racial-Bias-in-Data-Assessment-Tool_Chapin-Hall_INTERACTIVE.pdf (accessed November 17, 2024).

Cary Jr., M. P., S. Bessias, J. McCall, M. J. Pencina, S. D. Grady, K. Lytle, and N. J. Economou-Zavlanos. 2024. Empowering nurses to champion health equity & BE FAIR: Bias elimination for fair and responsible AI in healthcare. Journal of Nursing Scholarship 1–10.

Chapin Hall. 2024. Tool to assess survey data for racial bias: Applying a racial equity lens to research. https://www.chapinhall.org/project/new-tool-to-assess-survey-data-for-racial-bias/ (accessed September 4, 2024).

Clusmann, J., F. R. Kolbinger, H. S. Muti, Z. I. Carrero, J.-N. Eckardt, N. G. Laleh, C. M. L. Löffler, S.-C. Schwarzkopf, M. Unger, G. P. Veldhuizen, S. J. Wagner, and J. N. Kather. 2023. The future landscape of large language models in medicine. Communications Medicine 3(1):141.

Dean, L. T., and R. J. Thorpe. 2022. What structural racism is (or is not) and how to measure it: Clarity for public health and medical researchers. American Journal of Epidemiology 191(9):1521–1526.

Dixon, A. R., and E. E. Telles. 2017. Skin color and colorism: Global research, concepts, and measurement. Annual Review of Sociology 43(1):405–424.

Djuric, Z., C. E. Bird, A. Furumoto-Dawson, G. H. Rauscher, M. T. Ruffin Iv, R. P. Stowe, K. L. Tucker, and C. M. Masi. 2008. Biomarkers of psychological stress in health disparities research. The Open Biomarkers Journal 1(1):7–19.

Facente, S. N., T. Lam-Hine, D. N. Bhatta, and J. Hecht. 2022. Impact of racial categorization on effect estimates: An HIV stigma analysis. American Journal of Epidemiology 191(4):689–695.

Fawzy, A., T. D. Wu, K. Wang, M. L. Robinson, J. Farha, A. Bradke, S. H. Golden, Y. Xu, and B. T. Garibaldi. 2022. Racial and ethnic discrepancy in pulse oximetry and delayed identification of treatment eligibility among patients with COVID-19. JAMA Internal Medicine 182(7):730.

FDA (U.S. Food and Drug Administration). 2022. Clinical decision support software: Guidance for industry and Food and Drug Administration staff. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software (accessed August 28, 2024).

FDA. 2023. Marketing submission recommendations for a predetermined change control plan for artificial intelligence/machine learning (AI/ML)-enabled device software functions: Draft guidance for industry and Food and Drug Administration staff. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial (accessed August 28, 2024).

Feero, W. G., R. D. Steiner, A. Slavotinek, T. Faial, M. J. Bamshad, J. Austin, B. R. Korf, A. Flanagin, and K. Bibbins-Domingo. 2024. Guidance on use of race, ethnicity, and geographic origin as proxies for genetic ancestry groups in biomedical publications. JAMA 331(15):1276–1278.

Feiner, J. R., J. W. Severinghaus, and P. E. Bickler. 2007. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: The effects of oximeter probe type and gender. Anesthesia & Analgesia 105(6 Suppl):S18–S23.

Ferryman, K., M. Mackintosh, and M. Ghassemi. 2023. Considering biased data as informative artifacts in AI-assisted health care. New England Journal of Medicine 389(9):833–838.

Fisher, J. A., and C. A. Kalbaugh. 2011. Challenging assumptions about minority participation in U.S. clinical research. American Journal of Public Health 101(12):2217–2222.

Flanagin, A., T. Frey, S. L. Christiansen, and AMA Manual of Style Committee. 2021. Updated guidance on the reporting of race and ethnicity in medical and science journals. JAMA 326(7):621–627.

Ford, C. L., and N. T. Harawa. 2010. A new conceptualization of ethnicity for social epidemiologic and health equity research. Social Science & Medicine 71(2):251–258.

Franco, M., M. Durkee, and S. McElroy-Heltzel. 2021. Discrimination comes in layers: Dimensions of discrimination and mental health for multiracial people. Cultural Diversity and Ethnic Minority Psychology 27(3):343–353.

Gartner, D. R., C. Maples, M. Nash, and H. Howard-Bobiwash. 2023. Misracialization of indigenous people in population health and mortality studies: A scoping review to establish promising practices. Epidemiologic Reviews 45(1):63–81.

Gay, T. M., O. T. O. Farinu, and M. Issano Jackson. 2022. “From all sides”: Black–Asian Reddit communities identify and expand experiences of the multiracial microaggression taxonomy. Social Sciences 11(4):168.

Gee, G. C., and C. L. Ford. 2011. Structural racism and health inequities: Old issues, new directions. Du Bois Review: Social Science Research on Race 8(1):115–132.

Ghassemi, M. 2024. The pulse of ethical machine learning in multimodal health data. Presentation at Committee on the Use of Race and Ethnicity in Biomedical Research Meeting #3. January 31, 2024.

Giebel, S. 2023. “As diverse as possible”: How universities compromise multiracial identities. Sociology of Education 96(1):1–18.

Gonzalez, C., S. Krishnamurthy, F. Rollin, S. Siddiqui, T. Henry, M. Kiefer, S. Wan, and H. Weerahandi. 2024. Incorporating anti-racist principles throughout the research lifecycle: A position statement from the Society of General Internal Medicine (SGIM). Journal of General Internal Medicine 39(10):1922–1931.

Gordon, R. A., A. R. Branigan, M. A. Khan, and J. G. Nunez. 2022. Measuring skin color: Consistency, comparability, and meaningfulness of rating scale scores and handheld device readings. Journal of Survey Statistics and Methodology 10(2):337–364.

Harris, J. C. 2016. Toward a critical multiracial theory in education. International Journal of Qualitative Studies in Education 29(6):795–813.

HHS (U.S. Department of Health and Human Services). 2024a. HHS issues new rule to strengthen nondiscrimination protections and advance civil rights in health care. https://www.hhs.gov/about/news/2024/04/26/hhs-issues-new-rule-strengthen-nondiscrimination-protections-advance-civil-rights-health-care.html (accessed September 4, 2024).

HHS. 2024b. Section 1557 of the Patient Protection and Affordable Care Act. https://www.hhs.gov/civil-rights/for-individuals/section-1557/index.html (accessed September 4, 2024).

Huyser, K. R., and S. Locklear. 2023. Reversing statistical erasure of Indigenous peoples: The social construction of American Indians and Alaska Natives in the United States using national data sets. In M. Walter, T. Kukutai, A. A. Gonzales, and R. Henry (eds.), The Oxford Handbook of Indigenous Sociology. New York: Oxford University Press. Pp. 247–262.

ICMJE (International Committee of Medical Journal Editors). 2024. Preparing a manuscript for submission to a medical journal. https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html (accessed July 1, 2024).

IOM (Institute of Medicine). 2010. Public health effectiveness of the FDA 510(k) clearance process: Balancing patient safety and innovation: Workshop report. Washington (DC): National Academies Press.

IOM. 2013. The CTSA program at NIH: Opportunities for advancing clinical and translational research. Washington, DC: The National Academies Press.

Jackson, K. F. 2023. A critical scoping review of mental health and wellbeing research with multiracial subsamples 2012–2022. Journal of Racial and Ethnic Health Disparities. https://doi.org/10.1007/s40615-023-01811-2 [Online ahead of print].

Jain, A., J. R. Brooks, C. C. Alford, C. S. Chang, N. M. Mueller, C. A. Umscheid, and A. S. Bierman. 2023. Awareness of racial and ethnic bias and potential solutions to address bias with use of health care algorithms. JAMA Health Forum 4(6):e231197.

Johfre, S. S., and J. Freese. 2021. Reconsidering the reference category. Sociological Methodology 51(2):253–269.

Joyce, K., L. Smith-Doerr, S. Alegria, S. Bell, T. Cruz, S. G. Hoffman, S. U. Noble, and B. Shestakofsky. 2021. Toward a sociology of artificial intelligence: A call for research on inequalities and structural change. Socius: Sociological Research for a Dynamic World 7:237802312199958.

Klein, D. J., M. N. Elliott, A. M. Haviland, P. A. Morrison, N. Orr, S. Gaillot, and R. WeechMaldonado. 2019. A comparison of methods for classifying and modeling respondents who endorse multiple racial/ethnic categories: A health care experience application. Medical Care 57(6):e34–e41.

Kupke, A., C. Shachar, and C. Robertson. 2023. Pulse oximeters and violation of federal antidiscrimination law. JAMA 329(5):365–366.

Lawrence, J. A., I. Kawachi, K. White, M. T. Bassett, N. Priest, J. G. Masunga, H. J. Cory, C. Mita, and D. R. Williams. 2022. A systematic review and meta-analysis of the everyday discrimination scale and biomarker outcomes. Psychoneuroendocrinology 142:105772.

Lett, E., D. Adekunle, P. McMurray, E. N. Asabor, W. Irie, M. A. Simon, R. Hardeman, and M. R. McLemore. 2022. Health equity tourism: Ravaging the justice landscape. Journal of Medical Systems 46(3).

Liebler, C. A. 2018. Counting America’s first peoples. The Annals of the American Academy of Political and Social Science 677(1):180–190.

López, N., E. Vargas, M. Juarez, L. Cacari-Stone, and S. Bettez. 2018. What’s your “street race”? Leveraging multidimensional measures of race and intersectionality for examining physical and mental health status among Latinxs. Sociology of Race and Ethnicity 4(1):49–66.

Martinez, R. A. M., N. Andrabi, A. N. Goodwin, R. E. Wilbur, N. R. Smith, and P. N. Zivich. 2023. Conceptualization, operationalization, and utilization of race and ethnicity in major epidemiology journals, 1995-2018: A systematic review. American Journal of Epidemiology 192(3):483–496.

Massey, D., and N. Denton. 1993. American apartheid: Segregation and the making of the underclass. Cambridge, MA: Harvard University Press.

Monk, Jr., E. 2023. The Monk skin tone scale. https://doi.org/10.31235/osf.io/pdf4c.

Morning, A. 2008. Ethnic classification in global perspective: A cross-national survey of the 2000 Census round. Population Research and Policy Review 27(2):239–272.

Morning, A., and A. Saperstein. 2018. The generational locus of multiraciality and its implications for racial self-identification. The Annals of the American Academy of Political and Social Science 677(1):57–68.

Morris, T., and S. Sharma. 2024. Preparing for FDA’s new guidance for AI/ML medical devices. https://www.exponent.com/article/preparing-fdas-new-guidance-aiml-medical-devices (accessed September 4, 2024).

Muehlematter, U. J., C. Bluethgen, and K. N. Vokinger. 2023. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. Lancet Digital Health 5(9):e618–e626.

Nalven, T., N. S. Spillane, and J. S. Rossi. 2021. Racial discrimination, racial identity affiliation, and heavy alcohol use among multiracial individuals. Alcoholism: Clinical and Experimental Research 45(8):1653–1663.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2022. Improving representation in clinical trials and research: Building research equity for women and underrepresented groups. Washington, DC: The National Academies Press.

NASEM. 2023. Using population descriptors in genetics and genomics research: A new framework for an evolving field. Washington, DC: The National Academies Press.

NIH (National Institutes of Health). 2017. Does your human subjects research study meet the NIH definition of a clinical trial? https://grants.nih.gov/ct-decision/index.htm (accessed July 1, 2024).

NIH. 2024. Department of Health and Human Services part 1. Overview information. https://grants.nih.gov/grants/guide/pa-files/PAR-23-293.html (accessed July 1, 2024).

Obermeyer, Z., B. Powers, C. Vogeli, and S. Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366(6464):447–453.

OMB (U.S. Office of Management and Budget). 2024. Revisions to OMB’s Statistical Policy Directive No. 15: Standards for maintaining, collecting, and presenting federal data on race and ethnicity. Federal Register. https://www.federalregister.gov/documents/2024/03/29/2024-06469/revisions-to-ombs-statistical-policy-directive-no-15-standards-for-maintaining-collecting-and (accessed August 5, 2024).

Omiye, J. A., J. C. Lester, S. Spichak, V. Rotemberg, and R. Daneshjou. 2023. Large language models propagate race-based medicine. npj Digital Medicine 6(1):195.

Pahus, L., C. M. Suehs, L. Halimi, A. Bourdin, P. Chanez, D. Jaffuel, J. Marciano, A.-S. Gamez, I. Vachier, and N. Molinari. 2020. Patient distrust in pharmaceutical companies: An explanation for women under-representation in respiratory clinical trials? BMC Medical Ethics 21(1):72.

Pierson, J., A. Kerr, S. Robinson, R. Fanni, V. Steinkogler, S. Milan, and G. Zampedri. 2023. Governing artificial intelligence in the media and communications sector. Internet Policy Review 12(1).

Quinones, N., L. Fuentes, A. Hassan, A. K. Hing, G. Samari, and M. McLemore. 2024. Society of family planning research practice support: Strategies and considerations for addressing race and racism in quantitative family planning studies. Contraception 110534.

Reuter, E. 2023. 5 takeaways from the FDA’s updated list of cleared AI/ML medical devices. MedTech Dive. https://www.medtechdive.com/news/5-takeaways-from-the-fdas-updated-list-of-cleared-aiml-medical-devices/697570/ (accessed July 26, 2024).

Roth, W. D. 2016. The multiple dimensions of race. Ethnic and Racial Studies 39(8):1310-1338.

Sanchez, D. T. 2010. How do forced-choice dilemmas affect multiracial people? The role of identity autonomy and public regard in depressive symptoms. Journal of Applied Social Psychology 40(7):1657–1677.

Saperstein, A. 2009. Different measures, different mechanisms: A new perspective on racial disparities in health care. Research in the Sociology of Health Care 27:21–45.

Scharff, D. P., K. J. Mathews, P. Jackson, J. Hoffsuemmer, E. Martin, and D. Edwards. 2010. More than Tuskegee: Understanding mistrust about research participation. Journal of Health Care for the Poor and Underserved 21(3):879–897.

Schwabe, D., K. Becker, M. Seyferth, A. Klaβ, and T. Schaeffter. 2024. The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review. npj Digital Medicine 7(1).

Shim, J. K., K. W. Darling, M. D. Lappe, L. K. Thomson, S. S. Lee, R. A. Hiatt, and S. L. Ackerman. 2014. Homogeneity and heterogeneity as situational properties: Producing—and moving beyond?—race in post-genomic science. Social Studies of Science 44(4):579–599.

Tian, M., Z. Shen, X. Wu, K. Wei, and Y. Liu. 2023. The application of artificial intelligence in medical diagnostics: A new frontier.

UK Department of Health and Social Care. 2024. Equity in medical devices: Independent review. https://www.gov.uk/government/publications/equity-in-medical-devices-independent-review-final-report (accessed August 5, 2024).

Wendler, D., R. Kington, J. Madans, G. V. Wye, H. Christ-Schmidt, L. A. Pratt, O. W. Brawley, C. P. Gross, and E. Emanuel. 2006. Are racial and ethnic minorities less willing to participate in health research? PLOS Medicine 3(2):e19.

Wight, V. R., M. Chou, and Y. Aratani. 2011. Who are America’s poor children? The official story. National Center for Children in Poverty, Columbia University.

Yao, E. S., K. Meissel, P. Bullen, T. Clark, S. Morton, and P. Atatoa Carr. 2021. Classifying multiple ethnic identifications: Methodological effects on child, adolescent, and adult ethnic distributions. Demographic Research 44(21):481–512.

Yao, E. S., P. Bullen, K. Meissel, J. Tiatia, T. Fleming, and T. C. Clark. 2022. Effects of ethnic classification on substantive findings in adolescent mental health outcomes. Journal of Youth and Adolescence 51(8):1581–1596.

Yearby, R. 2020. Structural racism and health disparities: Reconfiguring the social determinants of health framework to include the root cause. Journal of Law, Medicine & Ethics 48(3):518–526.

This page intentionally left blank.