4

Use of Learning Health Care Systems to Improve Care for Traumatic Brain Injury

___________________

¹ This list reflects the rapporteurs’ summary of points made by the identified speakers, and the statements have not been endorsed or verified by the National Academies of Sciences, Engineering, and Medicine. They are not intended to reflect a consensus among workshop participants.

The third session of the workshop featured stakeholder perspectives on how learning health systems (LHSs) can address unmet priorities that apply to traumatic brain injury (TBI). The session discussed the Centers for Medicare and Medicaid Services (CMS) Innovation Center’s role in testing, learning, and scaling approaches for more effective care delivery and payment; the digital and analytic infrastructure needed to develop and maintain LHSs, examples of operationalizing LHS principles at New York University (NYU) Langone Health, and how the Department of Veterans Affairs models the concept of an LHS for veterans with TBI. Participants explored the use of such systems to identify gaps, barriers, and strategies to effectively address institutional needs, including how their organizations use data and innovation to improve health care delivery. David Goldstein, senior advisor at the Department of Health and Human Services, moderated the session.

PERSPECTIVES PANEL

Center for Medicare and Medicaid Innovation

Kathryn Davidson, director of the learning and diffusion group at the Center for Medicare and Medicaid Innovation (CMMI), described the development of payment models to support patient outcomes. A licensed clinical social worker, she began her career overseeing randomized clinical trials in community settings. At this intersection of behavioral health, social services, and primary care, she witnessed the profound effect of loss of grant funding on outcomes. Her work at CMMI enables her to direct her implementation science focus on the development of payment models that foster positive

patient outcomes and feature flexibilities that engage patients and caregivers. Born from the 2010 Affordable Care Act, CMMI tests new payment models that aim to reduce health care costs and increase quality. The organization works to improve patient outcomes by narrowing quality measures and by identifying incentives and flexibilities that promote innovation at the point of care delivery. Noting the high threshold of simultaneously reducing cost and improving quality, she reported that CMMI has tested approximately 50 models, but only 4 have been scaled to date. Innovation in care delivery is not limited to tested models, and CMMI’s learning and diffusion group and evaluation teams work to examine, understand, and enable innovative practices that benefit patients. As innovators develop practices that increase the patient experience of affordability, coordination, and access, CMMI considers the payment models needed to encourage these practices.

Agency for Healthcare Research and Quality

Edwin Lomotan, senior advisor for clinical informatics in the Center for Evidence and Practice Improvement at the Agency for Healthcare Research and Quality (AHRQ), discussed his organization’s role in providing tools, resources, and funding to support the development of LHSs. The agency’s mission is to produce evidence to improve the safety, quality, accessibility, equity, and affordability of health care, as well as ensuring the use of evidence in the field. For example, the AHRQ Evidence-Based Practice Center program produced a series of evidence reports on LHSs featuring topics such as engaging patients, families, and caregivers and addressing diagnostic errors. The agency’s activities to support patient-centered outcomes research include training LHS researchers. The program that Lomotan leads focuses on advancing patient-centered clinical decision support, which is relevant to LHSs and incorporating perspectives from patients, families, and caregivers. In addition, he is currently among a group convened by the University of Michigan and AcademyHealth that is developing an LHS maturity model that identifies the stages of a health care system transitioning to becoming an LHS. Sociotechnical infrastructure provides the personnel, processes, and technologies needed to create a system in which each unit of a health system—such as a TBI care center—functions as a small-scale LHS. These units can then collectively operate as an LHS at the organizational level, which incorporates and expands learnings across the organization and creates a whole greater than the sums of its parts.

New York University Langone Health

Leora Horwitz, director of the Center for Healthcare Innovation and Delivery Science at NYU Langone Health, explored strategies to shift health

systems toward becoming LHSs. The complexity of health care systems poses challenges to understanding how all parts operate. Feedback loops are a mechanism for developing knowledge and situational awareness about the health system. An effort to promote flu shots exemplifies this dynamic. Each September, NYU Langone Health activates an electronic alert instructing nurses to administer flu shots to patients. Several years ago, Horwitz and colleagues realized that the flu shot alert was initiated 22 times per patient per day, and was ignored 99.5 percent of the time, even though it resulted in 90 percent of eligible patients receiving the vaccination prior to discharge. After soliciting feedback from nurses, the team made several changes to the alert, but this effort only decreased the alerts by one per patient per day. A link added to the alert enabled recipients to provide comments, resulting in large quantities of varied feedback. This feedback loop revealed that the alert was issued within operating rooms (ORs), postanesthesia care units, endoscopy suites, and other locations where flu shots are not administered. The team then recognized that the information technology (IT) department set the alert to initiate each time a health professional updated the patient flow sheet. Although the flow sheet is updated about four to six times a day in many parts of the hospital, update rates within the OR are sometimes as frequent as once per minute. Despite the long-standing frustration that incessant flu shot alerts caused OR nurses, a mechanism for them to communicate this to IT was lacking. Once Horwitz’s team identified the issue, flu vaccine alerts were limited to only appropriate units within the hospital, resulting in a rate of five alerts per patient per day. She highlighted that establishing mechanisms to solicit patient feedback is a primary strategy in creating an LHS. For example, feedback loops could inform staff in medical offices that the television volume and lighting brightness are causing discomfort for their TBI patients.

Generation of higher-quality data is a secondary strategy to employ in shifting systems toward becoming LHSs, said Horwitz. Although data are plentiful in health systems, the ability to collect and convert meaningful data into knowledge is often limited. Health systems typically gather data points rather than use a structured approach that captures synthesis, judgment, experience, trajectory, and thought processes. She offered the example of a project several years ago aimed at optimizing testing in the emergency department (ED) for pulmonary emboli (PE) blood clots in the lungs. These clots can be catastrophic, yet they are easy to miss. Testing involves a computed tomography (CT) scan with radiation and contrast; therefore, it is not feasible or advisable to administer to all patients in the ED. Horwitz’s team built an automatic PE risk score calculation into the software system. When a physician ordered a CT scan for a person with a low PE risk score, the system issued an alert that suggested a preliminary D-dimer test before proceeding with the CT scan. Recognizing that these alerts were often ignored, the team solicited feedback from ED doctors

about the PE risk scoring calculation. Several physicians noted that because of NYU’s proximity to John F. Kennedy International Airport, the hospital frequently treats patients who have traveled on long overseas flights and have blood clots in their legs; however, the scoring system does not account for the PE risk that travel poses. Some doctors cited patients with prominent family histories of blood clots. Others explained they have honed their intuition over decades of experience.

To gather more data, Horwitz’s team added a system prompt that asked doctors to input their reason for overriding the low PE risk alert, such as recent travel, a hypercoagulable state, or high clinical suspicion. After several months of data collection, analysis revealed that CT scans ordered for patients because of recent travel rarely found blood clots. However, when doctors noted clinical suspicion of PE for patients with a low automatic risk calculation, the rate of PE was much higher. The center used this knowledge in reeducating doctors regarding their intuition, encouraging them to trust their judgment in some cases and advising them against overweighing certain factors that did not significantly increase the risk for PE. Horwitz stated that a TBI LHS could solicit feedback from providers about whether they suspect a patient has a TBI, the predicted trajectory, and treatments they predict as likely to be effective. A feedback loop could collect and analyze such data to inform TBI care.

The ability to generate readily and rigorously evaluable data is not reliant on receiving large grants, said Horwitz, noting that her team at NYU Langone Health has conducted approximately 30 randomized trials in the absence of funding from the National Institutes of Health. One such trial arose in response to the substantial decline in pediatric vaccination rates during the COVID-19 pandemic. Her team solicited feedback from parents of pediatric patients in order to target text message reminders. Parents offered reasons for delayed vaccination that included fear of contracting COVID-19 while in the clinic, the time commitment required to attend appointments, and a lack of understanding the rationale for vaccination. The team created several reminder messages with varied content and randomized delivery of the text messages to parents. None of the messages resulted in significant vaccination increases compared to the control group not receiving reminders.

Again reaching out for feedback, the team called parents to ask whether they had received the text messages and why they had not acted on them. Most of the parents indicated the lack of follow through was attributable to receiving the reminders while they were at work or when the clinic was closed, rendering them unable to call and make the appointment. The team conducted an additional randomized trial in which one group of parents received an initial text message in the evening that included a link to online appointment scheduling and a second message sent 36 hours later at lunch time. The vaccination rate among children with parents in this group

tripled in comparison to the control group not receiving a reminder. After randomized trials revealed the role of staggered text reminders in increasing vaccination, the practice was adopted systemwide. The LHS strategies of building feedback loops, capturing higher-quality data, and conducting randomized trials help illustrate opportunities applicable to advancing learning systems for TBI care.

Veterans Affairs Washington, DC, Health Care System

Joel Scholten, executive director of Physical Medicine & Rehabilitation at the Department of Veterans Affairs (VA), described efforts to incorporate integrated interdisciplinary TBI teams into a developing LHS. He noted that although VA features over 100 TBI teams, the VA health care system has over 1,000 points of care delivery; thus, only 10 percent of VA locations include onsite specialized, interdisciplinary TBI care. Drawing from their experience working with both TBI teams and individual veterans, case managers provide highly valued input. Using the electronic health record (EHR), VA created a plan of care templated note that is easily searchable and can be updated as a patient progresses through a treatment plan and transitions to a wellness plan. VA’s mission of helping veterans access lifelong care and wellness underlies a shift from episodic care to long-term, quality care that includes inpatient TBI units accredited by the Commission on Accreditation of Rehabilitation Facilities. Extensive VA research efforts—such as long-term implementation studies, partnerships with academic institutions, and collaboration with 16 TBI model systems nationwide—have generated a longitudinal database. Outcome comparisons from these data foster an understanding of the unique health care needs veterans experience following TBI.

Scholten noted a VA caregiver program that provides stipends to reimburse some of the time caregivers dedicate to care. Other VA benefits include home modification, vehicle adaptation, and housing support for homeless veterans. Implementing a whole health model of care, VA is working to shift from a focus on what is the matter with a veteran to what matters to the veteran. This priority seeks to offer integrated, individualized care delivery that maximizes patient engagement and treatment outcomes for veterans.

DISCUSSION

Data Strategies to Develop Learning Health Systems

Goldstein asked how organizations can move an LHS forward strategically and innovatively while integrating considerations such as program evaluation and stakeholder engagement. Davidson noted CMMI efforts to foster patient-centered and value-based care in the context of the large

Centers for Medicare and Medicaid Services (CMS) system, and that these strategies align well with LHS core principles. For example, efforts often seek to create flexibility, use data-driven decision making, center patients, and make use of cohesive payment models to defragment care delivery and improve outcomes. Lessons from such models can inform other organizations’ LHS efforts.

Davidson noted challenges in making vast quantities of CMS data usable in order to identify trends in health care. For example, the Medicare skilled nursing facility 3-day inpatient hospital stay waiver was coded with different names in various models. The learning and diffusion group has been systematically identifying such data entry issues to generate consistency and simplify data collection and analysis. Using data from claims, quality measures, learning systems, and evaluations, the group draws conclusions that inform future model development.

Partnering with external stakeholders, CMS capitalizes on its ability to use payment models for point-of-care innovation while learning from care delivery experts about practices that improve patient outcomes. To that end, CMMI is building feedback loops with providers, patients, and patient organizations to better understand what patients need and, in turn, establish incentives to shift practice toward meeting those needs. Thereby, CMMI creates an LHS internally while reinforcing LHS processes externally through partnerships with model participants.

Goldstein asked about data infrastructure innovations that a care system can adopt early in the process of becoming an LHS. Lomotan noted the relevance of a maturity model in considering an organization’s starting point from which to compare and track improvement, and stated that data infrastructure strategies will vary between care systems. Data should capture unmet patient needs and practices that are working, he said. Furthermore, organizations should strive to collect data that are findable, accessible, interoperable, reusable, and computable. Various mechanisms for collecting such data include decision logs, use spreadsheets, and systems based on artificial intelligence (AI). With a focus on applying what has been learned across settings and contexts, AHRQ works to analyze data from multiple agencies and has recently developed an application programming interface that indexes repositories and creates interoperable data points that can be used repeatedly for a variety of purposes.

Data and Continuous Improvement

Regarding the creation of a learning system that incorporates up-to-date evidence and establishes a culture of continuous quality improvement, Horwitz emphasized the importance of collaboration with departments and disciplines within a health system when determining areas of evidence-based

care in need of support. Rather than dictating changes, she and her team ask providers about pain points patients are experiencing, practices that seem to work but are lacking data, and facets of the program that may not be working well. Next, they collaborate in devising a mechanism to test the practice. She noted that the majority of the time, data on untested practices do not reflect the benefit that providers ascribe to them. These cases are viewed as opportunities for iteration and improvement. For example, the team spent 6 years tweaking the flu vaccine computer alert to achieve a rate of less than one alert per patient per day. The learning process requires time and experimentation, she emphasized, and once a better method is identified, NYU Langone Health generalizes it to routine practice across appropriate departments.

Competing priorities are inherent in a system as large as VA, and data can serve as leverage for assets and resources, said Scholten. In an effort to prioritize TBI needs within the entire VA health system, he works to integrate TBI with other agency-wide priorities. Narratives of individual veterans living with TBI highlight the role of quality care in improving outcomes. TBI teams communicate these stories to the community clinics where many veterans receive treatment. Given the 1,000 points of care delivery within the VA system, medical records can span numerous care locations. VA will soon introduce a new EHR system, and integration of all existing medical records could require a 10-year time frame. Meanwhile, investment in data and programming requests for the existing EHR will be limited. Furthermore, 75 percent of post-9/11 veterans with TBI also have post-traumatic stress disorder and receive care from mental health partners. This holistic model of care increases the complexity of integrating all medical records.

Comprehensive data enable dashboard metrics that aid in identifying frequency of quality-of-care outliers and can be used in improving the practice of individual providers. Over time, the use of data shifts the culture toward evidence-based care, he said. Proactive VA case management uses data to annually identify TBI patients with chronic disability; case managers then follow up with veterans and update wellness plans accordingly. After 2 years of establishing this process, the percentage of veterans with TBI that receive annual, in-person visits with health providers has increased from 20 percent to 50 percent. Goldstein remarked on the interplay of care delivery, organizational structure, data, and technology. As a result, he reflected, consideration should be given to the timing of engaging stakeholders in public and private partnerships to promote transparency while avoiding overburdening the process.

Supporting LHS Efforts Within a Care System

Joseph Giacino, director of rehabilitation neuropsychology at Spaulding Rehabilitation Hospital and professor of physical medicine and reha-

bilitation at Harvard Medical School, asked about the operational structure of NYU Langone Health’s evidence-to-practice feedback cycle. Horwitz replied that the Center for Healthcare Innovation and Delivery Science employs a project manager, two research associates, and two data analysts. She and a senior-level statistician dedicate part of their time to the center. One data analyst is focused on moving data into the EHR, which uses Epic software. Epic features a core function that enables randomization of electronic health alert recipients; thus, other health systems using Epic could conduct experiments similar to the work carried out at NYU with assistance from their IT departments. She emphasized the value of NYU Langone Health’s IT collaboration in enabling an iterative process of trial and error. This collaboration is ensured by the chief information officer’s commitment to creating an information system that meets the needs of both patients and providers. While grants enable some center activities, NYU Langone Health has funded the core operations of the center for 10 years. Horwitz meets with the executive team on a quarterly basis to ensure that the center’s activities are aligned with institutional priorities, further building organizational alignment and support for LHS efforts.

Engaging Health Systems in LHS Efforts

Corinne Peek-Asa, vice chancellor for research at the University of California, San Diego, asked about mechanisms such as incentives to translate knowledge gained across systems, including those in rural areas, and support the ability of less well-resourced systems to implement LHS practices. Scholten replied that VA uses virtual care in helping address inequity in the geographical availability of TBI care centers and that the federal nature of VA enables nationwide use of virtual care, regardless of state boundaries. Davidson noted that CMMI recently analyzed penetration of value-based care to identify types of providers and geographical areas that these models have not yet reached. To address identified gaps and bolster primary care, CMMI is testing models that feature upfront infrastructure, streamlined models and simplified incentives to support more widespread participation in value-based care in alignment with LHS goals. She noted the Making Care Primary model and States Advancing All-Payer Health Equity Approaches and Development (AHEAD) model, aimed at fostering coordinated, high-quality, and cost efficient care across primary and specialty disciples to improve health outcomes.

Lomotan echoed the importance of addressing health inequities in the design stage of an LHS, which needs to incorporate feedback from patients and families in order to build patient-engaged systems that address barriers to care. He emphasized that too often, patient input is not solicited until the end stages of system design.

Horwitz commented that although advanced EHR randomization and AI models can be used to analyze and learn from system data, NYU also uses simple strategies to align practice with LHS principles, and that these strategies can be employed by many types of care systems. For example, a project conducted with a call center alternated between two scripts on a weekly basis and filtered call lists according to medical record numbers being odd or even. These simple methods are not formal randomized trials but can be used to test different options in a wide range of settings with limited research budgets, including rural care settings. She added that NYU Langone Health has a toolkit available on its website to assist with this process.²

___________________

² See https://med.nyu.edu/centers-programs/healthcare-innovation-delivery-science/sites/default/files/chids-toolkit.pdf (accessed January 3, 2024).