Appendix D

Methods for the Structured Evidence Review

OVERVIEW

The committee conducted a structured evidence review of clinical services to prevent pregnancy-related cardiovascular morbidity and mortality among women of reproductive age (see Chapter 1). It employed a systematic, transparent, and reproducible approach to identify eligible studies, extract data, and assess quality while maintaining the flexible process necessary for a National Academies of Sciences, Engineering, and Medicine consensus study. The committee followed established protocols to conduct limited (not full/formal) systematic reviews to ensure methodological rigor and consistency across all nine research questions addressing the prevention of pregnancy-related cardiovascular morbidity and mortality. To help structure the elements of reviews, the committee adapted concepts from the analytic framework developed by the U.S. Preventive Services Task Force where appropriate (see Figure D-1).

SEARCH STRATEGY AND SCOPE

Since the nine research questions cover preventive clinical services across the prepregnancy, prenatal, postpartum, and interpregnancy periods, the committee developed separate search strategies with targeted inclusion/exclusion criteria to identify the relevant studies for each question (see Appendix C for a summary of each strategy). However, certain criteria remained the same for all searches. The committee limited searches to English-language publications from 2010 onward, studies conducted in the

United States or other countries classified as Very High Human Development Index settings (UN Development Programme, 2025), and randomized controlled trials (RCTs), cohort and case-control observational studies, cross-sectional studies, and systematic reviews. It included case series only when particularly relevant to an intervention or population. It excluded conference abstracts, dissertations, single case reports, protocol-only papers, and preclinical/animal studies.

Abstract Screening

For each search, all titles and abstracts underwent an initial screening for relevance using criteria tailored to each research question. Reviewers assessed the following:

• Study design: inclusion of eligible designs;
• Population: inclusion of the intended target group (e.g., prepregnancy, pregnant, postpartum, or interpregnancy women at risk for cardiovascular disease [CVD]);
• Intervention: relevance to the preventive service under consideration (e.g., telehealth, enhanced postpartum hypertension [HTN] management, and Life’s Essential 8);
• Outcomes: reporting of CVD, pregnancy, use, or harms outcomes, including maternal morbidity and neonatal outcomes; and
• Setting, language, and publication date: consistency with inclusion requirements.

A single reviewer conducted the initial screening of titles and abstracts for each question, with a second reviewer evaluating a random selection of 50 abstracts for each question to ensure consistency. If the two reviewers differed in their evaluations, they jointly reviewed 25 additional abstracts for that question and refined the inclusion criteria iteratively until both reviewers applied the screening criteria consistently. All screening was conducted within PICO Portal,¹ which supported tracking, documentation, and reproducibility.

The committee excluded studies using a hierarchical set of criteria in the following order: non-English language, outside publication date, wrong population, wrong intervention, ineligible design, or no relevant outcomes. Since the committee often excluded studies based on multiple criteria, a single reason was assigned for each study to prevent double-counting. The

___________________

¹See https://picoportal.org/ for more information (accessed December 7, 2025).

committee could still reference excluded studies, such as scoping reviews, for background information or to guide conceptual rationale.

Full-Text Review

For studies that passed title and abstract screening, one reviewer conducted a full-text review to confirm eligibility. Reviewers assessed each article to verify that it addressed the appropriate population, evaluated an intervention or clinical service relevant to the research question, and reported outcomes of interest, including maternal cardiovascular outcomes, pregnancy-related morbidity, service use, and intervention-related harms. Full-text review also confirmed that sufficient methodological and analytic detail was reported to support risk-of-bias assessment. When eligibility was uncertain, a second reviewer independently evaluated the study to support inclusion or exclusion decisions.

Study Quality

All studies that met eligibility criteria had their quality (good, fair, poor) evaluated using the National Heart, Lung, and Blood Institute (NHLBI) Study Quality Assessment Tools (NHLBI, 2021).

NHLBI Quality Assessment Tools

The NHLBI Quality Assessment Tools assign a quality score of good, fair, or poor by assessing a study’s risk of bias using one of five design-specific instruments:

1. Controlled intervention studies (RCTs),
2. Cohort and cross-sectional studies,
3. Case-control studies,
4. Before–after (pre–post) studies with no control group, and
5. Systematic reviews and meta-analyses.

The NHLBI tools were well suited to this review because they

• Provide design-appropriate criteria, which was essential given the heterogeneity of studies;
• Emphasize the transparency of judgments and explicit sources of bias;
• Support qualitative synthesis; and
• Provide interpretations for good, fair, poor ratings.

To evaluate the appropriateness of the NHLBI Quality Assessment Tools for this review (NHLBI, 2021), three committee members used them to independently assign quality scores for different types of studies and found consistent application and concordant quality scores across reviewers.

Interpreting Good, Fair, and Poor Study Ratings

Based on the NHLBI Quality Assessment Tools (NHLBI, 2021), a good score indicates a study with minimal risk of bias, strong methodological rigor, and results that could reasonably be considered valid. For example, a randomized trial with concealed allocation, low and balanced attrition, clearly defined outcomes, and appropriate analytic methods would receive a good score. A fair score indicates a study with some methodological limitations but none that fundamentally compromised its internal validity. For example, a study may have moderate attrition, limited adjustment for confounding, incomplete blinding, or some missing population details but still provide useful evidence. Studies were rated poor if they had significant risk of bias—including one or more fatal flaws, rendering their findings unreliable. The committee defined a fatal flaw as a methodological, execution, or analytic issue that so severely compromised internal validity that the results could not reasonably be considered credible evidence. Examples include high or differential attrition without appropriate handling, failure to preserve randomized assignment in an RCT, outcome measures that were undefined or inconsistently applied, or analytic choices that made effect estimates uninterpretable. Poor-quality studies were excluded from the evidence base used to assess effectiveness but included rarely for contextual insights (e.g., implementation or feasibility considerations) if higher-quality evidence was lacking.

The committee notes that fair and poor ratings do not necessarily reflect shortcomings in investigators’ work. Many limitations stem from constraints inherent to study populations, interventions, or ethical considerations, especially in evaluating clinical services in pregnant patients.

• Good studies had the lowest risk of bias and were given the most weight.
• Fair studies had some methodological concerns but were still considered informative.
• Poor studies were generally excluded from the evidence base but could be used for contextual information (e.g., implementation challenges, feasibility, cost).

Dual Review and Adjudication

The committee used a structured dual-review process to assign a quality score for each study using the NHLBI quality assessment tool to evaluate risk of bias. The reviews were conducted by a PICO Portal methodologist and a committee member with subject expertise.

1. The methodologist assigned the initial quality score based on the risk-of-bias assessment.
2. A committee member with subject-matter expertise independently assigned a second quality score based on their assessment.
3. If quality ratings differed by only one level (good ↔ fair or fair ↔ poor), the committee member’s determination was final after additional review by National Academies staff for consistency.
4. If quality ratings differed by two levels (e.g., good ↔ poor), the committee chair adjudicated the final rating, with discussion among the committee as needed.

All risk-of-bias assessments were documented within the PICO Portal.

Data Extraction

For all eligible studies, the committee extracted data using a PICO Portal standardized template for study design, demographic/clinical/contextual characteristics, study analyses, and outcomes. The committee then summarized findings across all studies for each question. Extracted information included the following:

• Study identification and characteristics;
• Citation, year, country, and study design;
• Sample size;
• Reproductive period addressed (prepregnancy, prenatal, postpartum, interpregnancy);
• Intervention type, timing, intensity, and implementation setting;
• Comparator or control conditions;
• Population characteristics; and
• Inclusion/exclusion criteria.

Where available, reviewers also captured key demographic, clinical, and contextual characteristics for patients, including the following:

• Age,
• Race and ethnicity,

• Socioeconomic indicators (education, income, insurance status),
• Parity,
• Body mass index,
• Key comorbidities (e.g., hypertension, diabetes),
• Smoking status,
• Prior pregnancy complications, and
• Geographic region or health care system features.

Additional variables (e.g., marital status, language, immigrant status, postpartum definition) were abstracted when relevant to generalizability.

Outcomes, Analyses, and Findings

For all reported outcomes, reviewers extracted the following:

• Outcome definitions, classifications (primary, secondary, exploratory), and measurement time points;
• Effect size estimates (means, proportions, relative risks/odds ratios/hazard ratios);
• Precision measures (confidence intervals, standard errors);
• Statistical significance;
• Attrition and missing data handling;
• Adjusted variables in multivariable models; and
• Relevant intervention fidelity or implementation information.

Maternal outcomes included mortality; CVD events; blood pressure, glycemic, lipid, and weight outcomes; pregnancy outcomes; service use; process measures; and harms. Neonatal outcomes included mortality, preterm birth, birth weight–related outcomes, neonatal intensive care unit admission, infant health complications, and harms.

The committee created a streamlined evidence table for the final report that summarized key characteristics and findings for each included study (see Chapters 4 and 5).

Evaluation of Strength of Evidence

The committee evaluated the overall strength of evidence for each preventive clinical service targeted by the nine research questions and considered the following:

• Study Design and Rigor: RCTs and high-quality cohort studies were weighed more heavily than cross-sectional or descriptive studies.

• Sample Size and Precision: Studies with larger samples and adequate power contributed more confidence. Small samples, underpowered designs, wide confidence intervals, or borderline p-values as indicators of uncertainty were noted.
• Effect Size and Clinical Relevance: Magnitude and direction of effects were considered alongside statistical significance.
• Follow-Up Duration.
• Generalizability: The committee evaluated whether study populations reflected the racial, ethnic, socioeconomic, and geographic diversity of U.S. populations and whether settings were representative of typical clinical practice.
• Methodological Limitations: Key limitations affecting confidence included missing demographic data, potential confounding, incomplete adherence/fidelity information, and limited longitudinal follow-up.

The committee included all relevant studies, regardless of whether findings indicated benefit, harm, or null effects.

Totality of Evidence Framework

The committee used a five-tier framework to characterize the overall strength of evidence for each clinical service (see Chapter 1 for more details):

1. Conclusive evidence of effectiveness,
2. Substantial evidence of effectiveness,
3. Moderate evidence of effectiveness,
4. Limited evidence of effectiveness, or
5. No or insufficient evidence of effectiveness.

This framework reflects the current state of evidence. For clinical services considered as having no or insufficient evidence because of few studies or limited-scope studies, this should not be interpreted as that clinical service lacking in effectiveness.

Synthesis

Given the heterogeneity of study designs, outcomes, and follow-up windows, the committee used narrative synthesis rather than pooled quantitative analysis to summarize findings (see Chapters 4 and 5). For each question and clinical service, it integrated findings across studies, weighting evidence by risk of bias, design, generalizability, and consistency. For each

research question, it also discusses available evidence on potential benefits and harms, cost and cost-effectiveness, population considerations, and other factors.

REFERENCES