2

EXECUTIVE DECISION NEEDS AND DATA PRIORITIES

To set the stage for the breakout discussions, the first meeting’s panel addressed the needs of executive-level decision makers in health systems as they strive to put new knowledge into practice and achieve improved outcomes for patients. Herb Pardes, former CEO and current executive vice chair of New York-Presbyterian Hospital, commented on clinical data as a change tool from the perspective of a health care executive. Nirav Shah, senior vice president and chief operating officer for clinical operations at Kaiser Permanente, introduced the concept of evidence generation from real-time care delivery. As a case example of data transforming practice, Ron Keren, vice president of quality and chief quality officer at the Children’s Hospital of Philadelphia (CHOP), discussed the comparative effectiveness of intravenous versus oral antibiotics for the postdischarge treatment of children with acute osteomyelitis. The presentations were followed by a brief open discussion. Highlights and main points of this session are summarized in Box 2-1.

A HEALTH EXECUTIVE’S PERSPECTIVE

In his opening remarks at the first meeting, Pardes emphasized that the health care landscape is experiencing extraordinary change; health system networks are expanding, and physician practices are consolidating. Payment reform is under way and there is a movement from fee-for-service to value-based care, with health systems becoming increasingly responsible for delivering the highest quality of care, as efficiently and effectively as possible. There is also a growing emphasis on population health management. However, he also stated that medicine in health care should always focus on what is unique to the individual patient.

As a former health system CEO, Pardes noted that executive decision makers want as much information about their patients as possible, to be able to make patient care better. Having accessible data is critically important, especially

when making decisions regarding what to prioritize and where to invest. While health system executives have access to data from their own health system, they struggle to obtain actionable data from outside their system, even within the same geographic region. They have even less ability to access data for comparison purposes from other health care systems, including academic medical systems.

There has been an explosion in research findings. A long-standing estimate suggests that is takes about 17 years for evidence-based care to become routine practice, but that time frame is shrinking with the introduction of electronic systems into routine clinical care. Still, the process of moving new knowledge from the bench to the bedside needs to be more efficient. In this regard, according to Pardes, there is a tremendous need to establish a national clinical data infrastructure to provide actionable data more rapidly and to facilitate more effective and efficient research. He cited the National Patient-Centered Clinical Research Network (PCORnet), with its consortium of Clinical Data Research Networks (CDRNs) and Patient-Powered Research Networks (PPRNs), as a step in this direction. The PCORnet national infrastructure supports patient-centered clinical research, which is critical because the patient voice has long been overlooked, and aims to improve the delivery of health care.

GENERATING EVIDENCE IN HEALTH SYSTEMS

Much of the information used for the improvement of health care over the past 50 years has been repurposed from regulatory, administrative, or payment data. Although substantial useful information has been gleaned from billing data, these data have a very poor signal-to-noise ratio, according to Shah, and cannot be the basis of improvement for the coming decades. Means or averages do not provide information about individual variation, but data on these variations are needed for improvement over the coming years.

As an example, Shah described using practice-based evidence in breast cancer treatment. Because younger women have fewer comorbidities, resulting in cleaner data, the average age of women enrolled in a breast cancer randomized controlled trial is 48 years old. But 48 may be very different from the average age of a particular cohort of women undergoing breast cancer treatment. After analyzing more than 2,000 different protocols for treating cancer patients at Kaiser Permanente, Shah pointed out that the average woman being treated for breast cancer was not 48, but 72 years old. In addition, while the rate of febrile neutropenia in randomized controlled trials was about 2 percent, Kaiser data from treatment protocols indicated that it was closer to 18 percent (i.e., one in five women in treatment was having this unexpected, potentially fatal outcome). As a result of this finding from

practice-based evidence, Kaiser now pretreats all women with breast cancer to avoid febrile neutropenia. To achieve the next level of health care improvement, evidence must be more relevant to real-world circumstances. Leveraging data at scale will be the basis of competitive advantages for health systems.

Top Health System Users

The 40,000 patients in Southern California who constitute the top 1 percent of health system users are seen in a hospital on average by 62 different individuals and cost more than $60,000 each. This population interacts frequently with the health system, and yet its needs are not being met. In partnership with Health Leads (a national organization that connects low-income patients with the basic resources they need to be healthy), Kaiser reached out to people in this group to ask questions about their health, including not only health care use but also food insecurity, transportation, housing insecurity, and other social determinants of health.

Although these patients often had three or four different social workers and case coordinators calling on them (e.g., one for hypertension, one for diabetes, one for asthma, etc.), many basic elements of health were still being missed. One very simple example was the need for a handrail to be installed in a stairwell for an elderly woman to help prevent falls.

The implication is that economic assessments by health care leaders should more broadly define health care and make the business case for investing in the social determinants of health. There is a return on investment for health systems in addressing social determinants of health and relationships with patients are a tactical advantage. Accountable care organizations (ACOs) are evolving, and, Shah noted, while they have the basic “anatomy of integration” by bringing together hospitals, nursing homes, and other health groups, they do not yet have the “physiology of integration,” which is the real-time flow of actionable data to the point of care across teams of providers. He observed that PCORnet embodies the opportunity to “move knowledge, not people.”³

DATA TRANSFORMING PRACTICE: CHILDHOOD OSTEOMYELITIS EXAMPLE

To illustrate the challenges of knowledge generation and translation, Keren described the experience in treatment of acute osteomyelitis in children, specifically, the comparative effectiveness of intravenous versus oral antibiotics

___________________

³ For more information see the IHI New Rules for Radical Redesign in Health Care at http://www.ihi.org/Engage/collaboratives/LeadershipAlliance/Documents/IHILeadershipAlliance_NewRulesRadicalRedesign.pdf (accessed May 31, 2016).

postdischarge. Children who present at a hospital with osteomyelitis are treated with intravenous antibiotics until the fever, swelling, and pain subside, and function returns. Continued antibiotic therapy is needed for several weeks postdischarge which, traditionally, is administered via peripherally inserted central catheter (PICC). There is a trend, however, to continue treatment at home via oral antibiotics.

In 1997, Peltola and colleagues published a study showing that 50 Finnish children with acute staphylococcal osteomyelitis were successfully treated postdischarge with an oral antibiotic (Peltola et al., 1997). Over the years, Ruebner and colleagues observed variation in how osteomyelitis was being treated postdischarge and, in 2004, they undertook a retrospective cohort study of all children admitted to CHOP from 2000 through 2003 with acute hematogenous osteomyelitis (Ruebner et al., 2006). Of the 80 children who met the inclusion criteria, only 5 had been transitioned to oral antibiotics. Of the 75 children who had received at least 2 weeks of antibiotic treatment at home via central venous catheter, 41 percent had a catheter-associated complication (catheter malfunction or displacement, catheter-associated bloodstream infection, fever with negative blood culture, or local skin infection).

Concerned about the high complication rate in children who are treated postdischarge via central venous catheters, Keren and colleagues conducted a retrospective cohort study of prolonged intravenous versus oral antibiotic therapy (Zaoutis et al., 2009). Zaoutis and colleagues searched the Pediatric Health Information System⁴ for children aged 2 months to 17 years who were diagnosed with acute osteomyelitis from 2000 through 2005. At that time there were 29 hospitals submitting data, and 1,969 children were identified who met the inclusion criteria. Approximately half had been treated via central venous catheter (n = 1,021) and half had been treated orally (n = 948). The authors found a wide variation across the 29 hospitals in terms of the percentage of children who were converted to oral therapy, ranging from 10 percent to 97 percent. Propensity score-adjusted rates of treatment failure (rehospitalization within 6 months) were similar (5 percent among children treated via central venous catheter versus 4 percent among those treated with oral therapy). A catheter-associated complication that required hospital admission was identified in 3.4 percent of children who received prolonged intravenous therapy. Keren noted that this number is significantly lower than the

___________________

⁴ The Pediatric Health Information System is a database of the Children’s Hospital Association. It currently compiles data from 44 children’s hospitals and includes 5 million inpatient cases, 30.2 million inpatient days, and 20.2 million emergency department encounters, as well as data from billing systems, patient abstract data, and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses and procedures data.

41 percent observed in the CHOP study because it is based on admissions only and does not include other provider visits for complications due to catheters (e.g., visits to the emergency department or a primary care provider).

Despite these findings, Keren’s review of data from 2009 through 2011 found that there were still hospitals where fewer than half the children were being transitioned to oral therapy. He sought to understand why his earlier findings had not led to a change in practice. Foremost was that there had been no dissemination and implementation plan for the 2009 findings. In addition, colleagues at hospitals that were still using central venous catheters said they had some concerns about the 2009 study. For example, the use of only administrative data led to concerns about the ascertainment of the diagnosis, exposure, and outcome. There were also concerns about residual confounding and new concerns about the rise of community-acquired methicillin-resistant Staphylococcus aureus (MRSA), which was not an issue at the time of the study.

To address this further, Keren and colleagues proposed a clinical effectiveness study to the Patient-Centered Outcomes Research Institute (PCORI) that would include chart reviews to confirm the diagnosis, exposure, and outcomes of the children identified through the Pediatric Health Information System. In addition, more up-to-date methods for dealing with confounding would be included, as would propensity score-based full matching within and across hospitals. Stakeholders would also be engaged to help facilitate dissemination and implementation.

The retrospective cohort study included data from 36 children’s hospitals on children hospitalized from 2009 through 2012 (Keren et al., 2015). The primary outcome was treatment failure, defined as a revisit or rehospitalization for a specified indication (change in antibiotic or dosage, prolongation of antibiotic therapy, conversion from oral to PICC route, bone abscess drainage, debridement of necrotic bone, bone biopsy, drainage of an abscess of the skin or muscle, arthrocentesis, and diagnosis of a pathologic fracture). Of 2,060 children with osteomyelitis, about half (n = 1,055) were treated via PICC lines and about half (n = 1,005) were treated with oral antibiotics. The percentage of children who were treated postdischarge via PICC line varied widely across hospitals, ranging from zero to 100 percent. The treatment failure risk was about 4 percent, similar to the 2009 study, and the risk difference was 0.3 percent, which Keren noted was not statistically significant. Fifteen percent of the children with PICC lines had a catheter-associated complication that required an emergency department visit, rehospitalization, or both.

Keren concluded his presentation on the research findings by focusing on the value of the retrospective cohort study design for this particular research question

and population. According to Keren, randomized controlled trials for children with osteomyelitis would not be feasible given the number of children who would need to be enrolled in order to show a small difference in what is already a relatively low complication rate. By using the retrospective cohort design, researchers were able to confirm the results of the prior study that was done using only administrative data. The results are consistent, even with the rise in MRSA prevalence. Keren suggested that the findings from the retrospective cohort design study are likely the strongest evidence available to answer the question of oral versus intravenous postdischarge antibiotic therapy. As a result of this effort, evidence-based knowledge was ultimately applied to impact practice transformation.

Elements of Success

Keren highlighted several elements that helped to enable knowledge generation and translation for the recent clinical effectiveness study. First, it was essential to have a funding source that was interested in supporting comparative effectiveness research, access to the necessary data (in this case, the Pediatric Health Information System), and engaged clinicians to conduct the chart reviews. The Pediatric Research in Inpatient Settings research network helped to identify the site leads at each hospital who could facilitate the chart review.

For dissemination, Keren partnered with PCORI to hold a continuing medical education seminar on the findings, and the publisher of the article, JAMA Pediatrics, sponsored a Twitter Journal Club. In addition, the Children’s Hospital Association sponsored a webinar, and there was coverage of the findings in dozens of pediatric and lay media. With regard to implementation, Keren is now working with the Children’s Hospital Association to produce quarterly reports that will be disseminated to key stakeholders and each of the children’s hospitals. Administrative codes have been validated and now have high sensitivity and specificity for case, exposure, and outcome ascertainment so that reports can be produced in an automated fashion. Audits and feedback reports will be shared with chief medical officers, chief quality officers, and chief safety officers. A change package is also being developed that will include education, guidelines, and treatment recommendations for dissemination to sites that are still using PICC lines for postdischarge antibiotic treatment.

In closing, Keren posed three questions for consideration relative to clinical data as a change tool: (1) Why did it take almost 20 years to move from the first treatment innovation in Finland in 1997, to the comparative effectiveness research, to the actual implementation work? (2) How many children were treated unnecessarily with central venous catheters? (3) How can we make better use of data to accelerate knowledge generation and translation?

DISCUSSION

In the open discussion that followed the presentations, participants emphasized the need for better communication and feedback between the research community and the practice community, in order to accelerate the application of findings. Danielle Lloyd of Premier, Inc., highlighted the importance of feedback, connecting back to the research, and applying lessons learned to retesting or the creation of new guidelines. She raised the question of how to get community hospitals more involved with the implementation aspect. Keren agreed that the separation between the academic and operational sides is a problem for a variety of reasons, including the absence of expectation of a dissemination/implementation plan for research findings. Researchers have historically relied on publication as the primary method of getting the word out. He observed, however, that there is a movement toward engagement and collaboration between the organizations that are generating the evidence and those that are putting it into practice. He reiterated his example of CHOP working with the Children’s Hospital Association to translate the results of research findings into practice.

Pardes observed that disparate groups are coming together, and that there is more inclusiveness of different constituents in health care than ever before. PCORI is the essence of this. There are challenges, he acknowledged, but the mood is becoming one of collaboration geared toward restoring the health care system. Participants stressed the importance of establishing a partnership between academia and clinical practice from the beginning and involving those who will be implementing the findings in the design of the studies.

Also referenced as a possibility for slowing progress was the existence of a contradiction in the tenets of medical professionalism. Health professionals are taught to “first, do no harm,” which leads to a healthy skepticism and conservatism about new ways of doing things. The other side of professionalism, however, is constant learning and looking at where the evidence points. The evidence may be there (e.g., for the transition to oral antibiotics for osteomyelitis postdischarge), but doubts about administrative data, and reluctance to test further and find a better way, can result in providers taking the fallback position and continuing with the “tried and true ways.” Moderator Larson expressed optimism that the medical profession is moving in a direction where people are questioning the standard assumptions.