6

FOLLOW-ON, THEMES, AND STRATEGIC EXPANSION

In a relatively short time, important progress and advances have been made since the two 2014 Institute of Medicine (IOM) workshops on health system leaders working toward high-value care, Larson noted. The Clinical Data Research Networks (CDRNs) have been hard at work, and have made significant progress in creating the basis for a national data infrastructure that will help to improve quality and outcomes in a patient-centered way. Larson observed that the 2014 IOM workshop report was titled Integrating Research and Practice, and attendees included CDRN leaders and staff, people from their delivery systems, and a few experts (IOM, 2015a). It was meant to be a dialogue session but it was more presentations than interaction. In contrast, the present meeting is focused on accelerating clinical knowledge generation and the use of clinical knowledge in the context of PCORnet. Larson lauded the quality of interactive dialogue at the workshops and felt that it is evidence that advances are being made in the area of accelerating knowledge into practice. Finally, Larson highlighted the value of the National Academy of Medicine (NAM) as a convening body for this effort. The NAM can bring together system leaders, the CDRNs, and other experts, as well as new partners, and keep this dialogue going into the future.

Selby concurred, and added that the past 18 months have seen a growth in appreciation of the potential and the importance of linking research with performance and system improvement on a rapid-cycle, quick-turnaround, performance-driven basis. He emphasized that there will be upcoming opportunities for continued dialogue and interaction. There is work to be done in the short term on preparing and funding the demonstration projects, and the health systems research group within PCORnet will continue its work. Additionally, Selby observed that there was clear and broad interest in linking delivery system and health plan data to better facilitate research. The importance of sociodemographic and socioeconomic determinants of health was also emphasized, and PCORnet would keep that in mind in building out the Common Data Model.

The importance of patient-reported outcomes was also discussed, and Selby noted the need to ensure that delivery systems and patients are not overly burdened with collecting patient-reported outcomes that are intended only for research and that do not support clinical care or health status improvement.

STRATEGIC EXPANSION

In order to build upon the momentum of the first meeting, the second meeting of the series was an opportunity to explore how the Patient-Centered Outcomes Research Institute (PCORI) had incorporated the feedback gathered at the January meeting to develop the PCORnet Health Demonstration Project, a funding opportunity for researchers and health systems. In the time between the two meetings, PCORI developed a request for proposals, solicited applications, and awarded four PCORnet Health Demonstration Project awardees. In the second meeting of the series, participants heard reports on the progress of the demonstrations projects, with particular attention to the impact and lessons learned from conducting these projects in health systems, with the involvement of clinicians, systems leaders, and patients.

Awardees of the PCORnet Health Demonstration Project discussed their research priorities, research and executive collaborations, and projects plans. Kaushal described her project, Identifying and Predicting Patients with Preventable High Utilization, a collaboration of three CDRNs. As she put it, “what we are trying to do is to integrate and quality assure multiple data sets and then characterize and prevent high utilizers.” The project was developed in response to—and in partnership with—health executives.

Kathleen Walsh of Cincinnati Children’s Hospital Medical Center provided an overview of her project, Automating Quality and Safety Benchmarking for Children: Meeting the Needs of Health Systems and Patients. The project, which will incorporate leadership from both patients and families and health executives, is focused on implementing and evaluating electronic measures of pediatric safety and quality for three high-priority conditions (transcranial Doppler for children with sickle cell disease, appropriate antibiotics for ear infections, and metabolic screening for antipsychotics) and ultimately determining the value of electronic measurement benchmarking to health system leaders.

Maureen Smith of the University of Wisconsin-Madison next presented on the Variation in Case Management Programs and Their Effectiveness in Managing High-Risk Patients for Medicare ACOs project, a partnership of two CDRNs to characterize case management programs across the country. Through a series of interviews with patients, case managers, and chief executive officers, the

programs aim to understand how various stakeholders conceptualize the key elements of case management programs, collect concrete program elements, develop a scheme across programs, and examine three case management programs in depth for effectiveness.

Finally, Jennifer DeVoe of Oregon Health & Science University began her presentation by recounting her own experience in practice with dashboards, benchmarking, and the impact of social determinants on patients’ health. She next described a project called The Impact of Patient Complexity on Healthcare Performance, which, through the work of two CDRNs, will integrate multiple social determinant data, such as transportation and environmental hazards, along with clinical and claims data to explore their impact on health care service utilization and health. Through the course of the project, the team will also address a multitude of issues associated with data integration, such as privacy and security concerns.

When asked what being part of PCORnet brings to their individual projects, presenters underscored that the PCORnet data, as well as the Common Data Model, allowed them the opportunity to pull data from multiple sites; that PCORnet provides an opportunity to engage collaboratively with many researchers and move across key networks. During the general discussion, a number of the health executives raised the point that the topics of these projects (e.g., benchmarking, defining metrics, and understanding case management and high utilizers) resonated with their systems’ priorities and could potentially provide valuable insights for the larger health community.

In addition to hearing about the progress of the PCORnet Health Demonstration Project, the meeting series also provided participants an opportunity to look ahead at the continued engagement and active involvement of health executives in evidence generation. To sustain the partnerships that first began with the initial meetings in 2014, the NAM has developed the Executive Leadership Network (ELN) for a Continuously Learning Health System. The ELN operates under the auspices of the NAM Leadership Consortium for Value & Science-Driven Health Care to support ongoing communication, collaboration, and synergy among executive-level stakeholders with a common interest in developing the capacity, infrastructure, and culture necessary to drive continuous learning and improvement within and across health care delivery systems. The ELN is a virtual network, and ELN participants serve as a standing group of ad hoc advisors to the NAM on the issues, strategies, and returns from continuous learning capacities that simultaneously support operational decision making, performance improvement efforts, and the generation of better evidence. In addition to the set of activities with PCORI, ELN members are involved in a number of NAM projects including the development of a

discussion paper on the measurement burden that provided the perspectives of large health systems related to the burden of measurement and reporting programs, its consequences, and actions being taken to increase the benefits of metric reporting while minimizing the burden to systems and providers; and NAM’s initiative to explore a series of strategic activities designed to improve synergy among health systems as they engage their common interests in broad interoperability of health data.

McGinnis next asked the meeting’s participants, “as you look at the sets of challenges related to the evidence generation enterprise, what would you like to see to make your lives easier?” Key comments are highlighted below:

• Amy Abernethy of Flatiron identified the importance of focusing on getting the data “right” and addressing the difficult questions related to data quality.
• A number of meeting participants answered that we need to take into account social determinants and championed stronger connections with social and environmental organizations (a key theme from the first meeting of the series). To this point John Kastanis of University Hospital mentioned that “it would be helpful for you to have better information on the linkages within other agencies, payment models that incorporate social factors, [and] common data sets that made available standardized information for health.”
• John Warner of the University of Texas, Southwestern University Hospitals, reflected on the need for the development of a business case for investment in evidence generation and increased synergy between efforts around science and business.
• Steve Allen of Nationwide Children’s Hospital stated that our problems are not due to a lack of data but the difficulty in turning that data “into action.” And a number of meeting participants also highlighted the need for implementation science—guidance on how to implement new learning activities while addressing work flow and staff satisfaction issues.
• Meeting participants also called for a unification of ongoing regulatory and measurement efforts so that the work of PCORnet might inform the scanning and pilots to test alternative payment models under the Centers for Medicare & Medicaid Services’ Center for Medicare & Medicaid Innovation and vice versa. Collaboration among the various large initiatives could lead to less fragmentation.
• Rachael Fleurence of PCORI noted that her vision is that the work of PCORnet will simplify the administrative process and streamline contracting and Institutional Review Board approval, noting that contracts can take over 6 months to get executed while patients are waiting for answers.

OPEN DISCUSSION OF NEEDS, OPPORTUNITIES, AND STRATEGIES

Participants considered next steps for executive and research collaboration in continuously learning health systems. Key foci included the scale and spread of partnership strategies, sustainability objectives and strategies for PCORnet, how health executives seek to further engage in evidence-generation initiatives, and action items for efforts by the NAM and PCORI to address outstanding barriers facing health delivery systems while enhancing and spreading best practices and promising strategies.

Newton framed the discussion by reflecting on the first meeting of health executives and researchers and how perceptions have evolved, as demonstrated by the increased degree of engagement by those in leadership positions. She also noted that this group of meeting participants has become “the choir” and challenged the participants to think about how to engage health executives that have yet to interact with PCORnet or other research networks.

Participants discussed the current health ecosystem in which health systems are taking on increasing risk and how it is important for the work of PCORnet to align with that transformation. In order to increase partnerships between researchers and health executives, there is a role for PCORI and PCORnet to impact the data needs, to continue to focus on key topics of interest to delivery systems such as the metric burden, to streamline key data elements, to begin to address data quality issues, to develop learning across the PCORnet system that has broader implications across health systems, to explore granularity in data that are currently missing, to contribute toward the streamlining of benchmarking metrics, and to move forward the capability of capturing social determinant data.

One issue that was reiterated by the meeting discussants was the importance of articulating the value proposition and business case of research networks. As stated by Smith, when speaking of her own experience of sharing information about this work: “when we have gone out and talked about this project—there’s a lot of interest but there is concern about costs to become involved and the burden on staff already changing the wings while flying. The more we could articulate the costs and how PCORnet could reduce the costs of new projects, the better.” Amy Abernethy also added that the value of PCORnet could best be offered through the presentation of use cases that demonstrate the success, processes, and costs of these projects within their organizations. Others agreed and also discussed the important role of using visualization of the data to engage stakeholders, such as health executives and clinicians. Finally, the meeting closed with a conversation on training programs for house staff that incorporate the

competencies (e.g. clinical informatics, quality improvement) of knowledge generation activities and would help clinical staff have the infrastructure, tools, and support they need to engage in a learning health system.

SUMMARY AND NEXT STEPS

The meeting series themes and opportunities for action, in Box 6-1,¹⁰ were drawn from the individual presentations, breakout sessions, and open discussions and highlight areas for future efforts by the NAM and PCORI to accelerate progress.

One of the key elements of the meeting series’ discussion was that even with access to multiple data, delivery systems face the challenges of making the data actionable; implementing findings into their specific institution with their distinctive populations; encouraging learning environments while also addressing workforce burnout; balancing investments in science with necessary investments in business; seeking innovation to make care more affordable and efficient; and demonstrating returns of research findings to patients and to the care providers. To demonstrate better implementation strategies, the participants articulated that delivery systems need toolkits and guidance documents that provide use cases and implementation strategies. In addition, they need data presented in formats (e.g. visualizations) that facilitate decision-making by health executives and clinicians and information on how other organizations are addressing the barriers described above. Participants also articulated that PCORnet could have an important role in meeting these needs. There are some findings that will be translational across all institutions, regardless of populations (e.g. the training strategies for house staff focused on the importance of evidence generation, which incorporates clinical informatics, clinical effectiveness research, and quality improvement).

Other projects, such as the FDA Evidence Generation Initiative, and corresponding paper titled Transforming Evidence Generation to Support Health and Health Care Decisions (Califf, 2016) are focusing on the potential of developing high-quality evidence through the integration of health care and research data systems and its implications for health and health care decision making. Likewise, on the topic of data and analytics, the Accelerating Clinical Knowledge Generation and Use meeting series participants noted that, in an ideal scenario, there would not be parallel universes where care is delivered and where the delivery of care is studied. Strategies for bridging these activities include expanding a common

___________________

¹⁰ This section is the rapporteur’s summary of main topics and recurring themes from the presentations, discussions, and summary remarks by the breakout session facilitators. Items on this list should not be construed as reflecting any consensus of the meetings’ participants or any endorsement by the NAM or the consortium.

data model to serve multiple stakeholders and encouraging more opportunities for embedded research. They also articulated that to effectively use data for decision making, knowledge generation, and improved care within delivery systems, there is need for analytics layered on top of multiple data; the use of the same data for clinical care, measurement, and reporting (e.g., duplication of data merging as opposed to a comprehensive strategy); the use of data for benchmarking so that systems can understand how they compare; the return of research results back to the delivery systems; access to data from multiple sites/sources that provides a more comprehensive view of their patients’ care (outside of their particular institutions); and a focus on data quality.

Additionally, one of the most highlighted discussion points throughout the day was that patient complexity impacts health care delivery and performance and it is therefore necessary for electronic health records to record core social determinants and key behavior patterns. Because social determinants are being addressed by institutions outside of health care, participants considered how delivery systems could better connect with other agencies, obtain better information about how other agencies collect and analyze data, and consider whether there are common data models across different sectors.

The meeting participants emphasized the need for networked evidence systems, such as PCORnet, and pointed to the increasing number of health executives who find value in such evidence systems. According to Kaushal, within the Weill Medical College of Cornell University NY-CDRN, there is a sense of ownership from the health executives involved in the CDRN, and several additional hospitals have asked to join. Drawing from the benefits of the PCORnet experience offering unparalleled research readiness for health executives, as well as access to new partners (i.e., PCORnet’s new involvement of health plans), the importance of expanding the opportunities for research partnerships, access to data, tools for collaboration, and continuous learning to other executives and systems across the country is clear.