5

Social and Health Policies

CHALLENGES AND SOURCES OF DATA

Jason Fletcher, Vilas Distinguished Achievement Professor of Public Affairs with appointments in Applied Economics and Population Health Sciences at the University of Wisconsin–Madison, started his talk by discussing a few methodological challenges. First, the work is highly interdisciplinary, and it is important to have the right mix of disciplines at the table. Second, the available data often are not representative of a population of interest (i.e., they are often based on convenience samples). Third, there may be a mismatch between the nature of the current data and the importance of capturing nuances and complexity. In particular, there is a tradeoff between collecting deep phenotyping of the outcome and having adequate sample sizes. Fourth, Fletcher said datasets need to be large enough to measure the types of effects that one hopes to find. Fifth, it is important to have strong data sharing and reproducibility protocols; Fletcher said social scientists seem to do this more than researchers in the medical sciences.

Outside of education, Fletcher said there are not a lot of social policies directed at cognitive decline or dementia. Programs such as SNAP have other purposes, and any impacts they may have on cognition are unintended and likely to be small. Looking for a silver bullet in these areas to prevent dementia may not pay off.

Fletcher said one potentially valuable source of data is Medicare claims reports. Medicare datasets are large enough (millions of people) that the standard errors are small, and even small impacts can be statistically significant. An example is an article by Bishop et al. (2023) that causally estimates

the impact of small changes in air pollution with small increases in the probability of receiving a dementia diagnosis.

Another valuable source of data, according to Fletcher, is large surveys, such as the UK Biobank, which allows one to examine cognition in up to 500,000 people (Davies et al., 2018). In that survey, data on AD diagnosis will become increasingly available as the cohort ages.

Fletcher said using vital statistics (death records) is another option, focusing on dementia mortality rather than on diagnosis. Still, the death records have limited information and do not encompass the life course, unless one is able to create linkages with large-scale surveys. One example of the latter is the Mortality Disparities in American Communities dataset, which is large (>3 million), though the data currently are restricted use. There are stalled plans to release a public use file. Fletcher also discussed the Diet and Health Study, a large-scale survey (400,000 people and more than 1,000 deaths from AD) whose data are used, among other things, to examine variations in mortality across different geographic locations, such as the “Stroke Belt” (Topping et al., 2021a,b, 2023). These data also have restricted access.

Fletcher recommended devoting additional funding to expand data linkages between the historical census, Medicare files, Social Security Administration files, and other federal databases. He acknowledged that such linkages would necessarily raise issues concerning access but felt they would be the closest we could come to having a registry system.

Fletcher listed several ongoing challenges:

• He said medical studies form the majority of dementia studies, and they tend to be too small.
• HRS is representative, he said, but also too small and lacks phenotyping. Similarly, the Wisconsin Longitudinal Survey is representative of the 1957 high school class in Wisconsin but lacks deep phenotyping. When only modest relationships between social exposures and dementia outcomes are anticipated, such surveys are too small.
• The UK Biobank offers more cases, but according to Flecther it is dependent on the respondents’ reports, resulting in less precise measurement. It has a low response rate of 5 percent.

Rather than multiple people independently buying access to the same data (including government data), Fletcher recommended that one organization purchase the access rights once and then make the data available more broadly.

In response to a question from the online audience, Fletcher agreed that one way of increasing the sample size is to add siblings to the HRS, Add

Health, Future of Families and Child Wellbeing Study, and other surveys. However, the sample sizes still are likely to be too small.

METHODS AND DATA FOR EXAMINING SOCIAL POLICIES

David Rehkopf, Associate Professor of Epidemiology and Population Health [Division of Primary Care and Population Health], Health Policy, Pediatrics and Sociology at Stanford University, and Director of the Stanford Center for Population Health Sciences, started by discussing the dominance of both proximal and early life factors in theory. Looking at what immediately happens in exposures and outcomes is useful for many applications but less so for much of chronic disease etiology. For most risk factors, the evidence is stronger for risk accumulating over time than for sensitive or critical periods early in the life course (Dunn et al., 2018; Lyu & Burr, 2016; Singh-Manoux et al., 2004).

Many frequently used datasets are too small to uncover even modest effect sizes. As an example of this, Rehkopf discussed analyses of the effects of the Works Projects Administration (WPA), which employed about 3 million individuals at its peak and employed almost 9 million in total. The Wisconsin Longitudinal Study collected data on WPA participation and linked these data to the 1940 census data to get information about exposures. The study had 10,000 participants, seemingly large enough to measure impacts from the WPA, but after excluding those who were not expected to benefit from the WPA (e.g., certain ages and people in rural areas) the number of observations for the analysis was ultimately around 2,700 (Modrek et al., 2022). Similarly, in a study that linked the HRS with the 1940 census data, the sample size was about 5,000 and only very large effect sizes could be detected (Lee et al., 2022a).

Rehkopf also discussed the use of electronic health records (EHRs), which have larger sample sizes as compared to most traditional cohort studies. These data present challenges, however, in terms of selection bias and measurement. Rehkopf wondered whether they provide enough opportunities to serve as useful complements to traditional cohort studies. These data are becoming more organized and accessible through a number of data aggregation projects, for example the Cosmos Epic EHR (Epic) platform. Because of its dominance, the Epic platform has a sample size of 280 million patients. Epic mainly offers coded data, but other sources have unstructured data that offer a lot of potential, with millions of books’ worth of notes from physicians talking about the barriers patients are confronting in their lives. As another example, Stanford maintains a relatively small EHR dataset with about 8 million patients, including about 25,000 American Indian and Alaska Native people. This inclusion is another advantage that this type of data has over traditional cohorts, where you

are very unlikely to have enough of a sample size to look at important subgroups of the population.

Rehkopf was also asked about using EHR data for subpopulations that traditionally have small sample sizes, whether the EHR data capture the social and structural determinants of health, and whether the data can be enhanced through data linkages. Rehkopf responded that for some subpopulations, such as American Indian and Alaska Native people, the Stanford data are quite good. One also has the option of looking at samples for particular geographic areas, such as people from Anchorage versus the Aleutian Islands.

EHR data tend to be extremely limited with regard to structured data on social determinants. Some screeners are beginning to be mandated to collect such data, but only about 4 percent of the sample has those data. Regarding data linkages, one possibility is to link to Census Bureau data at the individual level, and another is to use geographic overlays, such as three-digit zip codes or, in some cases, more refined geography like the census tract level. There may also be information from the notes that could be pulled out. Atheendar Venkataramani (Associate Professor in the Department of Medical Ethics and Health Policy at the Perelman School of Medicine, Staff Physician at the Penn Presbyterian Medical Center, and Director of the Penn Opportunity for Health Lab) added that EHR data increasingly break out Asian subgroups, such as Chinese, Philippine, or Indian origin. There may be measurement error, sometimes with 10 percent switching from one time to another, and it is not clear who is recording the assignment (e.g., whether self-reported, reported by a clinician, or reported by someone else), but typically this is thought of as an advantage of the data that allows fluidity of identity with respect to race and ethnicity.

In response to a question about how to deal with linkages to the EHR data and confidentiality issues, Rehkopf responded that a lot of his work is on state policies and that identifying people by state does not make them too identifiable and allows him to overlay current state policies and related information. If one can also access more detailed information on the location of a person’s residence, one can potentially create linkages with the 1940 or 1950 federal decennial censuses; in shorter time frames, one can still accumulate those places where people live over time.

Rehkopf said another potential data source is computed tomography (CT) scans. Approximately 80 million CT scans are performed every year in the United States and are sitting on a server somewhere. These data are not easy to get, and researchers would need to find a way to obtain or otherwise deal with the issue of patient consent, but they could potentially be used for studies of cognitive decline.

Rehkopf closed by discussing measurement difficulties. Data linkage is essential and complex, but a lot of the issues have been solved. Probabilistic

linkage can be performed on decennial census data that have been made public after 72 years, so the 1950 data are now available.

There also is the possibility of linking multiple datasets within a surrogate outcomes approach, Rehkopf said. Prentice (1989) set out a statistical framework for testing surrogate outcomes: the data needed in his framework are the exposure that is measured, true exposure of interest, surrogate endpoint, and actual endpoint of interest, all within the same dataset. Typically, however, the data are not in a single dataset, making this a challenge in practice. Recently, however, ideas have been put forward for using two separate datasets to evaluate surrogate outcomes. A paper from Athey et al. (2024) provided a user-friendly way of doing this. The true exposure may be in the experimental dataset, the long-term outcome in an observational dataset, and a surrogate and pretreatment variables in both datasets.

In later online discussion, Rehkopf was asked if the surrogate outcome approach was dependent on such high correlations that the two datasets would need to be virtually identical. Rehkopf responded that one could create a surrogate index that combines a large number of measures as surrogates. This helps to avoid the need of finding a single extremely strong surrogate. He also compared the analytic approach to one of using propensity scores to extrapolate out, saying a researcher could test the overlap of key covariates and cross-sections of those covariates between the samples.

Jennifer Weuve asked Rehkopf to elaborate further on the uses of EHR, noting that they are subject to social determinants that affect who seeks and receives healthcare, though she still felt the data are of value. Rehkopf agreed that there is a critical type of selection bias in the EHR data and commented that all data collection approaches are subject to some type of selection bias—for example, who agrees to be in surveys—and EHR data can be used to provide balance, triangulating among the various sources of bias in different types of data. He said that the way to appropriately use the data would depend on how the EHR data had been collected. For example, if the data were for primary care, then one might look at those conditions that are serious enough to lead to seeing a primary care physician but not serious enough to lead to an emergency room visit. A value of the Epic data is that they include emergency room visits, primary care, and specialty visits.

Venkataramani added that it might be useful to model what is not in the EHR, such as what is not being said, who is missing appointments, who is late, and which notes are shorter. This type of information could help in understanding the selection process for diagnosis in general, particularly if other administrative data sources are available. Paola Gilsanz (Research Scientist II at Kaiser Permanente Northern California Division of Research) added that the presence of EHR data could lessen the need to ask the participant for some types of information, reducing participant burden, since such information was already available.

COMMENTS ON THE SESSION

Speaking as the moderator for the session, Atheendar Venkataramani, Associate Professor in the Department of Medical Ethics and Health Policy at the Perelman School of Medicine, Staff Physician at the Penn Presbyterian Medical Center, and Director of the Penn Opportunity for Health Lab, commented that he would like to see if there could be a large cohort infrastructure with retrospective and prospective components that included surveys, some deep dive for biomarkers, and integration with administrative samples. Such data could be useful not just with regard to cognitive decline or dementia but with aging in general. Such a design might include component observational studies, randomized controlled trials that could test social science interventions as well as biomedical interventions which are interested in ADRD in particular, and fusion designs to look at different outcomes and different surrogates and make population inferences when there are small samples for any one of those outcomes.

Venkataramani commented on ways that non-health data could also be useful. Nicholas et al. (2021) found that one starts to see a failure to pay bills and a reduction in credit scores about six years preceding a diagnosis of AD. Such data might be useful where there is not full access to healthcare.

Given the cost and difficulty of conducting longitudinal studies over long periods of time, Venkataramani said one potential substitute is to use large cross-sectional data (i.e., based on a large sample) when the data include measures of cognitive performance or decline along with measures of age and state of residence (Deaton, 1985). There are measurement issues to deal with, but there are ways of dealing with these concerns. An example of using this approach is Schwandt and von Wachter (2018), though that study was on the mortality of graduates during a recession, not on cognitive decline or dementia.

Venkataramani commented on the value of using EHR data. Though the data have problems, including missing data, deep learning can be used to take information from physicians’ notes, adding to the value of the other data collected.

Venkataramani suggested that concepts such as the trajectories of cognitive decline, the accumulation model, and cognitive reserve could be put into simple mathematical models with few parameters using a methodology similar to that used by Lleras-Muney and Moreau (2022) in a demography paper looking at mortality. Such models are useful when data are incomplete, because they provide a theoretical structure for approaching questions of interest. With this model, Deryugina and Reif (2023) were able to look at the mortality penalty from quasi-experimental shifts in pollution exposure on the order of days, and then calibrate the model for the five parameters that they wanted to estimate, using the model to predict the

impact of lifetime exposure to pollution on mortality. Venkataramani felt researchers are pretty close to being able to do this with cognitive decline given all the epidemiologic insights discussed in this workshop. If there are sensitive periods in midlife, causal machine learning and looking at treatment-effect heterogeneity can allow researchers to examine that in a data-driven way, even without a theory to lean back on.

Amelia Karraker of NIA commented that NIA faces a difficult challenge finding the right balance among providing appropriate nuance in the data, making the data accessible, and protecting data security and human subjects. Maintaining trust in science so that people are willing to participate is important, and legal constraints also exist. Karraker also wanted people to be aware that it maintains the NIA LINKAGE Program,¹ which provide Centers for Medicare & Medicaid Services data linked with other NIA-funded datasets, including HRS, the National Health and Aging Trends Study, and the Understanding America Study.

Rehkopf raised the prospect of using synthetic versions of the data as a way of providing data access while protecting confidentiality.

A workshop participant noted that Meng (2018) looked at the tradeoff involved in choosing between smaller samples and large ones that are not randomly selected. This might be a useful tool for examining the potential tradeoffs for NIA funding.

___________________

¹ https://www.nia.nih.gov/research/dbsr/nia-data-linkage-program-linkage

This page intentionally left blank.