3

IMPACT OF HEALTH SERVICES RESEARCH

A session on the impact of health services research on policy and practice revealed a wide range of ways in which knowledge generated by the field has affected health care. It also highlighted the potential role and value of health services research in different contexts and for different audiences, sparking an important discussion of the ways in which the field can serve both policy and practice in the future and laying the groundwork for examination of this topic later in the workshop.

POLICY IMPACT OF HEALTH SERVICES RESEARCH

Health services research is as complicated as the health system that it tries to elucidate, said David Blumenthal, president and chief executive officer of the Commonwealth Fund. As a result, translating its results into terms that are useful to policy makers can be difficult. Nevertheless, health services research has made important contributions to policy in such areas as cost sharing, quality, payment models, and patient safety, Blumenthal observed. In particular, he identified 10 studies that have influenced policy (Box 3-1) and briefly described several of them.

In their discussion of how rates of utilization varied from town to town and county to county in Vermont, Wennberg and Gittelsohn (1973) raised the issue of practice variation that continues to be studied and have implications for the public and private sectors today.

McGlynn et al. (2003) showed that only about 55 percent of encounters with physicians resulted in the receipt of recommended care and has been an important influence on work on health care quality ever since.

Brook et al. (2006) described the RAND Health Insurance Experiment, which was “probably the largest and most influential single randomized control trial that has ever been done—and may ever be done—in health services research,” said Blumenthal. It documented the effects of cost sharing on the consumption

of care, on the use of appropriate and inappropriate services (showing that cost sharing reduces the use of services but does not reduce the use of inappropriate services), and on health. The result has been an important conversation that helped inform the creation of the Patient Protection and Affordable Care Act.

Campbell et al. (2007), in a project with which Blumenthal was involved, looked at interactions between academia and industry and contributed to passage of the Sunshine Act part of the Patient Protection and Affordable Care Act, which made it possible to learn if a physician is accepting donations or gifts from the pharmaceutical industry.

Girosi et al. (2009) developed one of only a very limited number of microsimulation models for health care available to predict the effects of new policies. Of the others, one has been developed by the Urban Institute, and another, developed by the Congressional Budget Office, is not available to health services researchers.

Song et al. (2014) evaluated the alternative quality contract, which was important in informing the accountable care organization model and demonstrated the potential to save money and improve quality by providing global budgets to physician groups.

Sommers et al. (2017) documented that expansion of Medicaid improved the health and well-being of low-income populations compared with states that did not expand Medicaid, a finding relevant to recent discussions of block granting for Medicaid.

Many more studies could be listed, Blumenthal said, but these 10 are enough to demonstrate “that this work deeply affects policy and also deeply affects the way the health system operates.”

QUALITY AND SAFETY IMPACT OF HEALTH SERVICES RESEARCH

As a specific example of the impacts of health services research, the president and chief executive officer of the Leapfrog Group, Leah Binder, noted that her organization, which represents employers and other purchasers of health benefits, relies heavily on health services research. “It gives us the insights we need to understand where the problem is, what are the best methods for solving it, and how to hold people accountable for results.”

Binder focused first on the problem of detecting and preventing mistakes. Health services research has provided key tools to identify errors, provide insights into how they happen, and determine how they can be prevented. Hospitals can now be held more accountable both internally, in terms of physicians and clinicians holding each other accountable for results, and externally, through businesses and other purchasers of health care gaining more accountability. “We can look at

results across the board and compare among health systems to better understand who is reaching and achieving at the highest levels and who is not and who needs to work on it.” This is information that would have been impossible to envision in the 1980s, yet now is available routinely because of health services research.

Binder also cited the progress that has been made through the development of patient safety indicators. “There’s been a lot of criticism of them, but they are extraordinarily effective.” Patient safety indicators have been endorsed by the National Quality Forum, extensively validated and tested, and utilized in research and in efforts to improve accountability. “We rely heavily on them, and we’ve seen hospital systems embrace them and make real changes that have been very effective in saving lives.”

She highlighted the annual survey that Leapfrog conducts, in which about half of US hospitals voluntarily make data available on patient care. As an example of the survey’s value, it includes data on the results of a Computerized Provider Order Entry (CPOE) assessment, which was developed with funding from AHRQ, that evaluates a hospital’s CPOE system. Provided with a set of dummy patients and dummy orders, the hospital determines if the system alerts to common errors embedded in the orders. “Most hospitals, when they take this test for the first time, are shocked because they thought their system was working just fine, and they usually find out it’s not,” said Binder. Based on her experience, about a third of the orders that they tested each year—and about one in six orders that would result in fatalities—were not alerted. But when hospitals take the test for a second time, they generally improve. “To me, [that] has been an example of where excellent health services research can make a difference both in quality improvement and accountability and has saved lives.”

Looking toward the future, Binder suggested that people think of measures and measurement not just as a noun, but as a verb. Sometimes, instead of a new measure, new strategies to achieve measurement are needed, she said. For example, the use of social media makes it possible to interact with consumers in ways that can improve their health. This work is in its infancy, she noted, but already other industries provide models that could be adopted in health care. In addition, she lauded the requirement by PCORI to incorporate patient views into the research it funds.

Binder concluded by warning that the battles over the status of AHRQ could be discouraging the next generation of researchers from entering the field. This “would be a terrible tragedy,” she said, “because so many of the problems in our health care system can be solved only through excellence in health services research.”

HEALTH SERVICES RESEARCH IN A LARGE HEALTH CARE SYSTEM

Following passage of the Patient Protection and Affordable Care Act, Partners Healthcare founded a large accountable care organization to manage risk-based reimbursement models. On January 1, 2012, Partners transitioned from having financial risk for the costs of its 100,000 employees and dependents to having financial risk for the cost of an additional 350,000 commercially insured patients and 100,000 Medicare patients. Then, on March 1, 2018, the at-risk population expanded to more than 85 percent of all patients seen in primary care, because the system agreed to accept financial risk for the cost of care for all its Medicaid patients. “If you haven’t heard about what’s going on in Massachusetts, it’s a landmark event,” said Timothy Ferris, a practicing primary care physician and chief executive officer of Massachusetts General Physicians Organization.

As senior vice president for population health at Partners Healthcare, Ferris was faced with the task of replacing the delivery of existing services with services that produce higher value across the full spectrum of health care services. Informed by health services research, the framework developed to meet this task had three main components.

The first was a change management task focused on motivating the 65,000 people working in the Partners Healthcare system to change the care they deliver. Health services research “knocked it out of the park” on that part of the framework, he said. The definitive literature on gaps in quality, quality metrics, risk adjustment, practice variation, experience of care, and on safety and errors have been critically important to accomplishing this task. He added one caveat, which is that the extrapolation of studies with wide error bars from a single site or a handful of sites to a national estimate, while perhaps helpful in a political process, can be destructive in trying to make changes within an organization. “Exaggerating the problem can be counterproductive to a change process.”

The second component was to define service delivery changes that presented the greatest opportunities for improvement. In this case, Ferris said health services research was “helpful, but it could have been more helpful.” Mongan et al. (2008) listed the topic areas involved in slowing the growth of health care costs: payment reform effectiveness review for new technology, electronic health record (EHR) research, care coordination for complex illness, transparency, reduction in administrative costs, drug pricing reform, and enhanced prevention. “I would give the history of health services research on these topics a solid B, maybe a B plus,” Ferris said. He also said that two other areas should be added to the list. The

first is variation research using claims data and clinical data, which can produce very different results. The second is artificial intelligence, which enhances our ability to analyze data and perform complex analytic tasks.

For this task, he also listed a caveat: from a manager’s perspective, research is often either too general or too specific in identifying opportunities to improve care. Economists have a tendency to draw “sweeping conclusions from oversimplified models of care delivery using data generated for a different purpose.” At the same time, health services research can be full of clinical detail but lack relevance. Ferris said that he has tried to occupy a space between these two worlds and recommended the paper by Eisenberg et al. (2000) on transforming insurance coverage into quality medical care as “the best articulation of how to think about the problem faced by managers who need to improve service delivery.”

The third component was to design, build, test, and disseminate interventions that clearly demonstrate improved value. In this case, he said, health services research has been “occasionally helpful, sometimes counterproductive, and mostly irrelevant.” The problem boils down to the difference between efficacy and effectiveness research. Programs and practices can be implemented in many different ways, since there are many different solutions to an operational problem. Doing randomized trials of actual changes in care delivery involve such artificial conditions as patient consent, a stable exposure, and relatively short durations, all of which provide a distorted sense of what happens in real life to such an extent that the results of such trials, especially when they are negative, are largely irrelevant, he stated. “Time and again, we have implemented changes in care delivery where there is existing research that says it cannot be done, and yet we have found success. . . . There’s a wide gap between the statement that a particular trial did not work, indicating that the specifics of the trial should be used to learn how not to do something, and saying that the idea behind the trial is itself not possible to achieve. This seems obvious, but like all researchers, health services researchers can push the limits of the generalizability of their results.” At the same time, and “on a more positive note,” he pointed to an extremely important negative trial—the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT) trial funded by RWJF (The SUPPORT Principal Investigators, 1995). “That negative trial has very heavily influenced what we have done to improve end-of-life care.”

He concluded by pointing out how important health services researchers have been to change efforts at Partners Healthcare. “Time after time, our team is asked by various internal stakeholders to justify our expenditures, our focus, our approach. Having a small team of skilled health services researchers has

been essential to our efforts and has led to countless presentations important for sustaining momentum within our organization and more than 60 publications in the past six years to try to disseminate our largely observational trials.”

APPLYING HEALTH SERVICES RESEARCH IN PRACTICE AND POLICY

The presenters’ examination of past advances in health services research led to a discussion of the many ways in which the field could affect policy and practice in the future. As Blumenthal pointed out, health services research can address everything from large health systems reforms and policy issues using traditional research methods to specific implementation issues that apply in particular contexts. With regard to the former set of problems, health services research can be extremely useful in identifying the most pressing health and cost problems and also possibly in designing macro-level policies. When he was working on implementing EHRs, he was guided by the conviction that having such data available would create huge benefits—and now advances such as those in artificial intelligence are proving that conviction correct. But this conviction required “an understanding of the interaction between data and progress in a field that is informed by research.”

Health services research has been less impactful on micro-level implementation decisions, he continued. One reason is that such problems tend to be less suited to traditional academic research approaches. Those who study such problems therefore may need to be rewarded in other ways than through the traditional academic award structure. For example, he pointed out that health services research has rarely supported or guided in a definitive way the decisions he has had to make as a health system manager. The difficulty has been both finding literature that is relevant and applying that literature if and when it is found. For that reason, he has always encouraged the researchers who have worked with him to experiment in the practical world and spend time as implementers or government officials if they can, “because it leads to a completely different set of questions rising to the surface.” He also encouraged researchers to ensure that their work is driven by practical purposes and that the applicability of their results is appropriately described.

He acknowledged, however, that this advice ignores some of the tenets of academic training and the culture of research, where new researchers are told to pick a single area and dig as deeply as they can into that topic. Young researchers may also be discouraged from working in government, serving as a manager, or even doing translational research because they will fall off the promotion

track. Such advice represents an “endemic academic debility that health services research has to contend with,” he said. “Being grounded in academics is both a great strength and also a great limitation.”

Alan Weil, the editor-in-chief of Health Affairs, called attention to the importance of translating results from health services research, commenting that translation of research involves a different skill set than academic research and is not primarily about publishing papers. “Diffusion [of knowledge] is a more complex and multifactorial process than publishing journal papers,” he said. “If we want to maximize value, we need to think about it in the context of diffusion, not just about the creation of health services research.”

Similarly, Ferris also drew a distinction between academic health services research geared toward academic purposes and research geared toward implementation questions. The differences between these types of research have implications for training, hiring, and the dissemination of knowledge that have not always been well recognized, he noted. For example, implementation science can aggregate methods from health services research and business and management science while also using new analytic approaches.

He emphasized that the central issue is the dissemination of useful information, not just the publication of papers or the pursuit of high-impact factor journals. As an example of such research, he pointed to the publications he and his colleagues are producing, which typically are observational studies of problems and of how changes affected those problems one way or the other. However, such studies “do not make the grade for evidence in the way we traditionally think of evidence,” he noted, and they are difficult to publish in prominent journals.

Binder pointed out that health services research cannot and should not attempt to answer all the questions a leader in a health system might have about how to improve, given the difficulties of running a health system. But when leaders are motivated to make changes, they typically are able to do so, even if those changes can only be guided and not dictated by research. “There’s a certain magic to leadership and culture and management that remains yours, but research can inform that.” Research can also compare one system to another, which can help systems understand where they need to go. It can suggest ways of incentivizing improvement while leaving the challenge of change management to health system leaders.

At the same time, Binder pointed to the fundamental value of health services research in providing ways of thinking about value and cost effectiveness in specific and actionable ways. For example, “if there’s any consensus in our health care system, it’s that we have to move away from fee-for-service. We just haven’t

been able to figure out how to do it, and there are lots of reasons for that. But no field has given us more tools to do that than health services research.” As a specific example, she cited the track record of health services research in developing concepts and methods that enable comparisons among providers and health systems, which has been “an extraordinary accomplishment.”

This page intentionally left blank.