SUMMARY

Medicine is currently undergoing a paradigm shift from evidence-based practice to a personalized approach. A shortcoming of evidence-based medicine (EBM) is that it lacks precision by applying broad-based group data to the treatment of an individual. Yet, each patient is unique, and treatment responses differ from one person to the next. This variability in treatment response is called heterogeneous treatment effects (HTE), the study of which is essential for doctors to effectively tailor treatments for their patients to maximize the benefits while minimizing the harm.

On May 31, 2018, the National Academy of Medicine, in conjunction with the Predictive Analytics and Comparative Effectiveness (PACE) Center at the Tufts Medical Center, held a workshop in Washington, DC, to discuss approaches to examining HTE to personalize and improve patient care. Funded by the Patient-Centered Outcomes Research Institute (PCORI), the day-long discussion centered on the following motivating questions:

• Potential: How can clinical trial data be analyzed to yield reliable patient-centered treatment effect estimates? What are the state-of-the-science methods for assessing treatment heterogeneity?
• Risks: How can we be sure personalizing evidence will improve decision making, as compared with the default of relying on overall average treatment effects? What are the evidentiary standards for implementing changes to clinical practice to personalize care based on evidence of HTE?
• Lessons learned: What can be learned from the challenges of genomics-based personalized medicine? What can be learned from the efforts of previous clinical trialists to understand more personalized treatment effect estimates?

• Strategies: How should clinical research and clinical practice be redesigned to support the generation and the dissemination of patient-centered evidence?

This publication summarizes the remarks and the insights of workshop participants consisting of patients and patient advocates, physicians, medical researchers, research funders, and health insurers, as well as representatives from pharmaceutical companies, federal agencies, professional associations, and medical journals. The conversation began with a discussion of the promise of exploiting HTE to personalize care for patients, the related key concepts and considerations, examples of the types of analyses that have been conducted, and challenges for the field. One challenge with modeling treatment effects is identifying an appropriate reference class or group of patients with a similar set of characteristics to reflect the target patient. For many reasons, it is now recognized that conventional subgroup analyses that examine how treatment effects vary across characteristics “one variable at a time” are of extremely limited value for informing care decisions. Defining subgroups based on outcome risk has emerged as a useful (if imperfect) approach to separating the patients most likely to benefit from a treatment from those unlikely to benefit or those most likely to experience net harm. Ultimately, the goal is to develop sophisticated composite risk scores that reflect a range of patients’ personal variables such as comorbidities, functional status, mental health status, and the various social determinants of health.

As researchers and clinicians search for ways to best deal with and take advantage of HTE, they must consider patients’ needs and preferences. For instance, patients need to understand the relationship between the average treatment effect described in clinical trials and their own individual situation. Additionally, it would benefit patients if studies of a specific condition were conducted in a uniform way that enabled results to be compared across studies—and for trials to be pooled to provide the statistical power needed to describe variations in treatment effect. Given these priorities, a push toward patient-centered care will undoubtedly alter the traditional relationship between patients and health care systems, with patients playing a more active role in their care.

The transformation is not exclusive to patients. Regulatory agencies and health insurance companies have to rethink their assessment of medical treatments. The U.S. Food and Drug Administration (FDA) looks for a variety of differences in how people respond to drugs based on variables such as demographic differences, genomic characteristics, and disease severity. For payers, the assessment of medical treatments and the examination of treatment heterogeneity have reimbursement implications, especially in the current environment in which payers are relying

more frequently on value frameworks to determine which treatments to cover for which cohort of people.

To deal effectively with HTE, there are several methods and models for predicting how individuals will respond to different treatment approaches. Using multiple genes to predict predisposition to a disease, polygenic risk scores have been used since the late 1990s to identify high-risk groups for targeted interventions. In recent years, however, the predictive performance of polygenic risk scores has come under question, with numerous studies proving their inability to clearly distinguish which groups of people will likely develop a disease from those who will not. Machine learning offers an additional set of analytical tools. With advancements in computing power, machine learning methods (e.g., penalized regression, regression tree-based methods, Bayesian nonparametric models, ensemble learners) make it possible to spot correlations in data that are beyond human capacity. Yet, despite the theoretical appeal of these methods, applications of these tools in general practice have been limited.

This is just one of the many barriers to implementing HTE prediction models and techniques in routine clinical care. Apart from ensuring clinical validity, HTE predictive models need to demonstrate clinical utility and workflow advantages. Prediction tools must be able to integrate seamlessly into a medical records system so as to provide clinicians with near-real-time results and improve decision making. Addressing these issues is necessary to impress provider confidence in these tools. Without physician acceptance, HTE models will be meaningless and will fall short of their potential to improve the value of care.

As highlighted by participants’ remarks, the field of HTE is still in its infancy. It must not only address outstanding methodological questions, but also determine best practices for implementing risk models and predictions tools in clinical practice. Therefore, key directions for the field include

• Developing guidance on approaches for assessing the effectiveness or the validity of predictive and prognostic models;
• Understanding the comparative performance of supervised machine learning methods that can be applied to understand HTE;
• Facilitating collaboration and leadership across various sectors of the research ecosystem to create prioritized opportunities for large trial re-analyses or collaborative individual patient data analyses to examine HTE most likely to impact population health;
• Describing approaches to implementing risk models in clinical care and providing guidance on which approaches are most effective at informing decisions both at the point of care and at the level of the health care system;

• Considering approaches for integrating data related to the social determinants of health into risk-prediction models;
• Determining the role for observational data and when it is appropriate to combine randomized controlled trials and observational data;
• Reforming the predominant fee-for-service payment system in the United States to one that rewards value and population health improvements;
• Promoting dissemination of innovative trial designs, including those sampling larger and broader populations to enrich patient heterogeneity; and
• Establishing or extending research reporting guidelines to promote the conduct of predictive HTE analyses.

Addressing these priorities will require deliberate coordination among a wide range of stakeholders, including researchers, clinicians, payers, regulators, health delivery organizations, and medical journals, with the ultimate goal of serving the patient. The individuality of the patient should be at the core of every treatment decision. One-size-fits-all approaches to treating medical conditions are inadequate; instead, treatments should be tailored to individuals based on heterogeneity of clinical characteristics and their personal preferences.