2

Committee’s Approach to Its Statement of Task

This chapter describes the committee’s approach to, considerations of, and methods for reviewing the Department of Veterans Affairs (VA) pre-decisional presumption decision process document (see annex in Chapter 1). The first section describes how the committee interpreted its Statement of Task (see Chapter 1, Box 1-1), including what aspects were considered to be outside the scope. This is followed by a description of the committee’s information-gathering process; as this assessment is primarily a document review, it relied on information gathered through open sessions, information requests, and other supplemental documentation rather than a synthesis of systematically identified literature. The committee then defines specific terms used in the VA presumption decision process document and in this report, characteristics the committee believed would be desirable for the process, statistical and decision-threshold concepts, and other considerations that the committee kept at the forefront when reviewing and assessing each section of the document.

INTERPRETATION OF THE STATEMENT OF TASK

The Sergeant First Class Heath Robinson Honoring Our Promise to Address Comprehensive Toxics (PACT) Act of 2022 (PL 117-168) required VA to enter into an agreement with the National Academies to conduct an assessment of the presumption decision process (Section 202 §1176) that it uses to determine whether a “specific medical condition is or is not associated with an environmental exposure” (VA, 2022). The legislative language specified three elements for the assessment:

1. To determine whether the process is in accordance with current scientific standards for assessing the link between exposure to environmental hazards and the development of health outcomes,
2. To assess whether the criteria are fair and consistent, and
3. To provide recommendations for improvements to the process.

Other parts of Section 202 (see Appendix A) detail required elements of the VA presumption decision process, such as

• The requirement that the public have an opportunity to comment on the “military environmental exposures and adverse health outcomes” that may be considered for presumption decisions (§1172);
• Establishment of a VA working group to assess suspected and known toxic exposures occurring during active military service that are not recognized under existing presumptions of service connection (§1172);
• Types of information to be reviewed and considered for each health condition recommended by the VA working group and the use of four categories of association based on the estimated likelihood that a positive association exists (§1173);
• The process to be used by the VA Secretary to establish or modify a presumption of service connection based on the working group’s recommendations (§1174); and
• The authority of VA to modify the process (§1175).

In its interpretation of the charge and at the urging of VA, the committee focused its assessment on the scientific attributes of the process. VA stressed that the process is a population-based model—it is not intended to be applied to individuals—and is meant to “improve VA’s method for evaluating conditions and exposures for presumptions that [is] scientifically sound and legally defensible with the goal of creating a new evaluation model that considers all relevant data and information, is transparent, and reduces the time it takes to determine presumptions of service connection” (VA, 2023b). Using a population-based model for presumption decisions and tying it to an accepted exposure connected with military service removes the burden of proof on individuals to show they were exposed.

The presumption decision process document is eight pages long and incorporates both VA-led and external paths that diverge and converge at different points along the process, which may offer some flexibility but adds to potential complexity as well. Therefore, to use consistent language when discussing the process as a whole or focusing on specific components, the committee adopted the following terminology, which it uses throughout its report:

• The phrase “presumption decision process” denotes the entire VA presumption decision process as presented in the eight-page pre-decisional document provided to the committee, from the preparatory phase (condition or exposure identification) through the ultimate decision of presumption by the VA Secretary, and includes the potential to use both VA-led internal and external paths as appropriate for the condition(s) under consideration.
• The phrase “medical condition(s)” as used in the presumption decision process document was interpreted by the committee to be those physical and mental health conditions that have associated diagnostic codes (e.g., International Classification of Diseases or Diagnostic and Statistical Manual of Mental Disorders). Because VA cannot make presumptions based on individual symptoms¹ (VA, 2023a), phrases such as “health outcome” as used in the PACT Act are likewise ambiguous, and the committee adopts VA’s wording of “medical condition” or simply “condition.” The conditions may be common or rare, be physical or psychologic, have short or long latency periods, and have different levels of etiologic complexity (e.g., single versus multiple causes, single insult versus confluence of multiple factors).
• The phrase “VA-led” denotes internal process components. This includes monitoring or surveillance activities of VA data, VA-based research, or epidemiologic studies and possibly internal literature reviews that may be highly focused.
• The term “external” denotes components of the process that are performed by an agency or organization outside of VA. These components might include large-scale studies or reviews, such as National Academies consensus reports.

Aspects Outside the Statement of Task

Several aspects of the presumption decision process were outside the committee’s Statement of Task. The committee was not tasked with designing, redesigning, or developing a new framework for the presumption decision process. It was also specifically instructed not to apply the proposed process to an example condition as a case study or vignette. Similarly, the committee was not to use or apply the process to readjudicate any existing presumption decisions. Moreover, as the process is not specific to any one era or cohort of veterans, the committee assessed it broadly. The committee

___________________

¹ Some veterans may receive disability compensation for chronic disabilities resulting from undiagnosed illnesses or medically unexplained chronic multi-symptom illnesses defined by a cluster of signs or symptoms (VA, 2015).

was not tasked with performing a comprehensive assessment of the scientific evidence regarding any associations between exposures that may have been encountered during military service and medical conditions, nor would it draw conclusions regarding the strength of evidence for any associations. Furthermore, although the 2008 National Academies report that served as the basis for several components of the revised VA presumption decision process was considered, the committee was not tasked with assessing the extent to which the recommendations in that report were accepted or implemented by VA, nor was the committee bound to agree with, support, or endorse any aspect of that report. Finally, the use of the presumption decision process to make a presumption of exposure (not specifying a medical condition) was outside the committee’s charge.

The committee’s assessment of the VA presumption decision process is not intended to set or change the statutory standards VA is required to use regarding the process or any specific presumption decision. Any VA policy decision to maintain, expand, or reduce benefits, presumptions, or covered conditions will necessarily be based on a range of considerations, including congressional interest and action and VA’s priorities. The relative weights of those considerations and priorities are not necessarily equal, may change over time, and are outside of the Statement of Task.

One consideration in this assessment, and indeed for any protocol or process document, is the balance between brevity and the detail required to explicate the intended process. VA has stated that an earlier version of the revised presumption decision process was 26 pages, which was deemed unacceptable, and so it was shortened to the eight pages that the committee reviewed and assessed. The 26-page draft was provided to the committee as background information. In reducing the document by two-thirds, some details of the process had to be omitted, such as when and how Veterans Benefits Administration data would be used. However, the purpose and background sections of the eight-page document fill nearly one and a half pages of the limited space, reducing the area where important and critical details may be provided. Therefore, one of the considerations that the committee kept at the forefront, given the brevity of the VA document, was the need for additional detail that would allow an understanding of and ensure the consistent performance of the presumption decision process.

INFORMATION GATHERING

The committee used a variety of information-gathering activities in its deliberations, including invited presentations, information requests to VA, requests for public comment, and peer-reviewed scientific literature. The report Improving the Presumptive Disability Decision-Making Process for Veterans provided additional background (IOM, 2008). Although the

committee was told that the process it was to review is being implemented, no data from its actual use were available to examine for potential sources of biases, representativeness, validity, and reproducibility.

Given the short time frame the committee had in which to deliberate and produce its report, it held one virtual, public information-gathering session to learn more about the legislative origins of the presumption decision process and VA’s and Congress’s expectations for its review. During that public session, it also heard an overview of the 2008 National Academies report, which had a different Statement of Task, including a request for proposing a science-based process for determining presumption decisions. Several elements of that 2008 proposed framework were included in the PACT Act (PL 117-168) or appear in the VA pre-decisional eight-page presumption decision process document. Following the invited presentations by VA and the chair of the 2008 National Academies committee, veterans, veteran service organizations, family members, and other veteran advocates were invited to make oral comments related to the committee’s Statement of Task or provide information on the presumption decision process. Nearly 100 individuals attended this public session, and they were encouraged to submit written comments and materials throughout the committee’s deliberations (through May 2023).

Technical requests for information were submitted to VA’s Health Outcomes Military Exposures staff. The committee’s questions and VA’s responses are available in the public access file,² as are other directives and documentation that the committee received from VA.

DESIRED CHARACTERISTICS OF THE PRESUMPTION DECISION PROCESS

One overarching principle that the committee applied to its assessment of the presumption decision process is the need for it to be inclusive of a broad range of environmental exposures and health conditions, including mental health conditions. After hearing a presentation from VA on the Statement of Task, engaging in discussion with VA representatives and professional staff of the Senate and House Committees on Veterans’ Affairs, and hearing public comments, the committee clarified some key goals for and desired characteristics of the presumption decision process. Specifically, it should be fair, consistent, transparent, timely, and veteran centric to account for and be consistently applied to all potential medical conditions. Other characteristics and attributes may also be included within, implied by, or common across these five desired characteristics. For

___________________

² The committee’s public access file is available by request at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-22-10.

example, an overarching premise of the presumption decision process is that it is feasible, flexible, and has the appropriate elements to fulfill its purpose (determining recommendations and the associated approval process regarding conditions qualifying for a presumption). Feasible means that it can be achieved by VA staff (or contractors) and balances scientific rigor with an efficient process. Flexible means that it can accommodate variable volumes of evidence for each medical condition considered and different numbers and types of conditions (physical and psychologic).

Fairness

Fairness in this context means a process that is free from prejudice or prejudgment to the greatest extent possible. The presumption decision process is designed to be fair to veterans across all eras and cohorts, a priority for VA. It also includes considering the same types of information (provided such information exists) and applying the same threshold for granting a presumption for all eligible veterans. For example, in Section 202 §1173 of the PACT Act and codified in 38 U.S.C., Congress has provided the elements (evidence, data, and factors) for VA to consider when determining whether a presumption is warranted. The goal is to apply these elements fairly and consistently (see next paragraph) in every review, including how conditions are identified for presumption review (VA, 2023c).

Consistency

The presumption decision process is also designed so that the elements of the process are applied consistently and reproducibly to veterans of all eras and cohorts. Consistency needs to be considered and maintained across multiple dimensions, including dates of service, the conditions (physical or mental) under consideration, and the minimum quantity and quality of evidence required to make a recommendation for a presumption.

Transparency

Transparency of the presumption decision process allows veterans and the general population to have visibility into it and verify that it is conducted without bias and in a manner supporting other goals, such as fairness and consistency. The committee assumed that the end users of the presumption decision process document are VA personnel, but to help ensure that the process is transparent, it is important that documentation is available to other interested parties and written at a level that can be understood by those who will be affected by it (the veteran population). Having a

transparent process is not synonymous with ensuring that all stakeholders will agree with the decisions that are reached.

Timeliness

A timely process does not unnecessarily lengthen or delay any step or the time between evidence evaluation and decision determination. Timeliness also supports the goals of fairness and consistency. In response to a committee information request, VA stated that this goal is primarily related to promoting and acting on a recommendation for a presumption (VA, 2023c).

Veteran Centric

Being veteran centric includes fairness, transparency, and timeliness but also the guiding principle of reaching presumption decisions that favor veterans when the available information regarding an association between an exposure and the condition of interest is equivocal. This may occur, for example, when all sources of confounding cannot be exhaustively addressed or experimental data demonstrating causality cannot be reasonably or ethically obtained. Taken together, these components are consistent with the goals of providing maximum benefit to veterans with potentially service-connected physical or mental health conditions and minimizing burdens and harms of delays, such as from advancing pathology and disability.

CAUSATION VERSUS ASSOCIATION

The committee considered in depth the definitions for and congressional intent of the terms “association” and “causality” or “causal relationship,” in both the setting of their usual technical meanings in epidemiologic and statistical inference and the presumption decision process. In clarifying the intent of the statutory requirements for the presumption decision process and its implementation by VA, it is useful to distinguish between conclusions regarding the relationship between an exposure and adverse condition in a population versus in an individual. Congress intended the presumption decision process and its recommendations regarding that relationship between an exposure and a condition to be applied on a population level regardless of whether it can be shown for an individual veteran. To address the intent of statutory language (which applies nonstandard use and definitions of several terms of association), the committee uses the terms “association,” “positive association,” and “individual causality” and defines their meanings in the context of, and as used in, this report.

Association

An association is a statistical finding of a relationship between an exposure and an outcome (in this case a medical condition), commonly quantified by a non-zero correlation coefficient or equivalent metric (e.g., a relative risk when the outcome is dichotomous), so that the presence or absence of the exposure provides information on the distribution or risk of the condition in a defined population. An association is a characteristic observed in a population, as it reflects a comparison of the health outcomes among persons with and without the exposure; there is no assessment for whether this correlation may be due to other factors (confounding). Associations may exist because the exposure causes the condition or, alternatively, when both of them are associated with one or more confounding factors or a bias (e.g., selection or information bias or differential measurement error), which leads to spurious associations.

Evidence of an association may be based on observational study designs, cross-sectional study designs, or other designs that do not account specifically for temporality between the exposure and condition. These designs may not include statistical methods for addressing the potential for confounding to create or contribute to the observed association. The committee uses “association” when a non-zero correlation has been established between the exposure and condition, with the required level of certainty; however, the possibility that this association is the result of confounding factors has not been reasonably ruled out.

Positive Association

Several terms have been used in public law and policy documents that attempt to denote association as the standard to be applied to a relationship between a specified exposure and health condition. These have included “statistical association,” “positive association” (e.g., PL 117-168, §1173), and “link” (e.g., PL 117-168, §1176). From a statistical and epidemiologic standpoint, these terms are not synonymous nor interchangeable, but all refer to the same concept: assessment of the relationship of the medical condition to the exposure. Although not in the presumption decision process document, the term “positive association” is used in the PACT Act Section 202 §1173, which describes elements of the process. This term is not standard, but the committee chose to adopt “positive association,” which it uses to mean that the condition is caused by the exposure in at least some members of the population. “Positive” in positive association is not intended to imply a beneficial effect, direction of effect, or confidence of the relationship between an exposure and health condition.

In some situations, the condition may also occur in unexposed individuals (e.g., lung cancer occurs at a low rate in nonsmokers), so that a positive association may not definitively demonstrate that it was caused by the exposure in any particular individual. Instead, it implies that the exposure causes the condition in at least some individuals in the eligible population. Demonstrating a positive association from observational data usually requires statistical adjustments for potential confounders or similar sources of bias (e.g., multivariate regression [Holmberg and Andersen, 2022; Lewis and Ward, 2013; Rothman et al., 2008], inverse probability weighting [Cole and Hernán, 2008; Mansournia and Altman, 2016; Rothman et al., 2008], propensity score analysis [Guo et al., 2020; Haukoos and Lewis, 2015; Rothman et al., 2008], and, in some cases, more sophisticated and modern approaches, such as target trial emulation [Hernán et al., 2022]).

Demonstrating a positive association generally requires both an association between the exposure and the condition—with the condition occurring after the exposure—and evidence that confounding factors and potential biases are unlikely to be the cause of the association using the applicable scientific and statistical methods and thresholds (e.g., considering the plausibility of the causal relationship, existence of other similar positive association relationships, or multivariable analyses of observational data to adjust for known or plausible confounders).

Individual Causality

“Causality” is a general term denoting that the exposure results in the medical condition in at least some individuals and, conversely, that it would not occur without the exposure in at least some individuals. To prove that the condition resulted from an exposure at an individual level would require evidence that it could not have otherwise occurred, and for most conditions, this proof is nearly impossible to obtain. For example, because of advances in genomics and genetics it has been established that certain individuals are more or less susceptible to a particular condition. Additionally, many conditions are multifactorial, meaning either gene-environment interactions or multiple exposures result in the condition, and rarely is any one exposure the sole “cause” for that condition, especially cancer. As such, an exposure may contribute to the development of a condition either solely or in conjunction with other exposures. Demonstrating causality at the individual level is not required or expected of the presumption decision process, which is applied on a population level.

Individual causality is not the standard set by law for VA to use in the presumption decision process, but carefully considering causality serves as a basis for thinking through the exposure–condition question. It allows for a structured discussion regarding the weight of evidence and whether

a population-level positive association has been established with sufficient likelihood, without having to prove causation for an individual.

Guidance for Assessments of Positive Associations

Factors to guide assessments of the likelihood of causal associations have been proposed for many decades. An early example was provided by Sir Austin Bradford Hill (Bradford Hill, 1965), nearly 60 years ago (see Appendix A of the Chapter 1 Annex). Although the Bradford Hill factors, and similar approaches, were developed to address questions of causal relationships, the committee has adopted “positive association” to denote the situation in which the condition is caused by the exposure in at least some individuals, without any implication of or requirement for causality being demonstrated on an individual level, and to be consistent with legislative language.

The Bradford Hill factors are strength of association, consistency of evidence through replication of findings and other knowledge, specificity of the association, temporality (i.e., exposure must have occurred before the condition), dose–response relationships, biologic plausibility, coherence of evidence, consideration of experimental evidence, and consideration of alternate explanations. These factors are to be considered in total and not used as a checklist that requires each to be satisfied. However, to assess a positive association with rigor, the temporal sequence must be established. While many of the factors are still valid, scientific advancement of genetics, genomics, epigenetics, omics, and pathophysiology makes many of them outdated and incomplete for assessing association.

Scientific standards for assessing positive associations have advanced since Bradford Hill first published these factors. Modern methods and criteria are more sophisticated and have advanced interpretation of observational data (Hernán et al., 2022). The nine Bradford Hill factors were a step forward when they were first published, and they are still used appropriately as examples of concepts to consider for such assessments. However, these factors are structured to be broad and somewhat vague, are generally applied algorithmically, and do not represent the current state of science for determination of a positive association. It is beyond the committee’s Statement of Task to delve into the complexities of assessing the probability of a positive association from observational data. The committee considers the Bradford Hill factors as only a basic foundation for examining the likely origins of observed associations but not as defining a scientifically valid process for assessing the likelihood of a positive association.

In the presumption decision process, VA is required to evaluate evidence that can be used to evaluate the likelihood of a positive association and, in contrast, not to require demonstrating individual causality. The threshold

for granting a presumption, set by statute, is that the evidence supporting a positive association is equal to or greater than the evidence against it, which is termed “equipoise” in the legislative language of the PACT Act and the VA presumption decision process document.

Equipoise

“Equipoise” is used in the PACT Act and in the VA presumption decision process document for situations in which the evidence for and against a positive association exactly balances. This is the threshold at which VA is required to grant a presumption: the likelihood of a positive association is equal to the likelihood of no positive association, or a probability of 50%.

The committee uses statements regarding the probability or likelihood of an association of 50%, or similar phrasing, to denote the threshold for making a recommendation for presumption and generally avoids “equipoise” except when referring to specific language within applicable regulations and documents. This terminology is not intended to convey any alteration in the threshold for granting a presumption. Equipoise is discussed at greater length in Chapter 4.

Assessing the Likelihood of a Positive Association Versus Estimating the Magnitude of a Positive Association in a Population

The recommendation to grant a presumption is based on the likelihood of the existence of a positive association between an exposure and an adverse medical condition, with a decision threshold that a positive association is more likely than not. A distinct goal in the evaluation of a potential association is estimating its magnitude (i.e., the quantification of the change in risk of the condition associated with exposure). Examples of metrics used to capture the magnitude include relative risks, odds ratios, and hazard ratios, with associated measures of uncertainty, such as confidence intervals. As noted by other National Academies committees (NASEM, 2016, 2018, 2020a,b, 2022), a variety of challenges can occur in estimating the magnitude of an association, including random error (i.e., chance variation), selection bias, exposure and medical condition measurement error, unexplained variation in exposure or medical condition data, and confounding. The probability that a positive association exists is only indirectly related to its magnitude, in that larger magnitude associations are less likely to be the result of unmeasured confounding or bias. The magnitude of the positive association is not a factor, in and of itself, in the decision to recommend a presumption; however, the committee notes that VA’s decision to not explicitly include magnitude of

association or attributable fraction³ as factors in presumption decisions may result in classification errors affecting which veterans may be eligible for a presumption.

VA uses a decision process rather than an estimation process to reach a presumption recommendation, which yields a dichotomous decision to grant or not grant a presumption. The distinction between an estimation process and a decision process is important because in some situations (e.g., a rare condition), uncertainty may be large regarding the magnitude of an association, but a positive association is highly likely.

REFERENCES

___________________

³ Attributable fraction quantifies the proportion of disease burden among exposed people in a population that is caused by the exposure. In other words, it is the fraction of all affected individuals who would not have been affected had no one in the population been exposed to a particular exposure. When specifically applied to military personnel, the service attributable fraction is used to determine the fraction of new and emerging cases of a condition associated with an exposure that would not have occurred except for military service (in garrison or deployment), and that additional risks and effects from exposure will remain consistent over age and across subgroups in the exposed population of military personnel.

Rothman, J., S. Greenland, and T. Lash. 2008. Modern Epidemiology, 3rd ed. Philadelphia, PA: Lippincott Williams & Wilkins.

VA (Department of Veterans Affairs). 2015. Federal Benefits for Veterans, Dependents and Survivors: Chapter 2 Service-connected Disabilities. https://www.va.gov/opa/publications/benefits_book/benefits_chap02.asp (accessed July 5, 2023).

VA. 2022. Presumption Decision Process. Provided by Dr. Patricia Hastings, Chief Consultant, Health Outcomes Military Exposures, VA. June 14, 2022. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-22-10.

VA. 2023a. Charge to the Committee to Review the Department of Veterans Affairs Presumption Decision Process. Presentation by Dr. Patricia Hastings, Chief Consultant, Health Outcomes Military Exposures. March 7. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-22-10.

VA. 2023b. VA Presumption Model and Proof of Concept. Provided by Dr. Patricia Hastings, Chief Consultant, Health Outcomes Military Exposures, VA. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-22-10.

VA. 2023c. Response to the Committee to Review the Department of Veterans Affairs Presumption Decision Process information and data request. Provided by Dr. Patricia Hastings, Chief Consultant, Health Outcomes Military Exposures, VA. April 19, 2023. Available from the project public access file at https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BPH-22-10.

This page intentionally left blank.