Proceedings of a Workshop

WORKSHOP OVERVIEW¹

Phillip E. Rodgers, The George A. Dean, M.D. Professor and Chair, Department of Family Medicine, University of Michigan Medical School, observed that “we are at a critical time in the field” of serious illness care, and “as we look to the future to meet the challenges of an aging society” with increasingly complex health care needs, a “high-quality evidence base for the work we do is essential to meet the moment.” Translating evolving research and evidence into improved care is critical, as Jean Kutner, distinguished professor of medicine and associate dean for clinical affairs at the University of Colorado School of Medicine, emphasized: “We have made tremendous advances in the field, and we owe it to those we care for with serious illness and their families to ensure we are providing the best evidence-based care.”

To examine the current and future state of serious illness care research, the National Academies of Sciences, Engineering, and Medicine’s Roundtable

___________________

¹ The planning committee’s role was limited to planning the workshop, and the Proceedings of a Workshop has been prepared by the workshop rapporteurs as a factual summary of what occurred at the workshop. Statements, recommendations, and opinions expressed are those of individual presenters and participants and are not necessarily endorsed or verified by the National Academies of Sciences, Engineering, and Medicine, and they should not be construed as reflecting any group consensus.

on Quality Care for People with Serious Illness hosted a public workshop, Serious Illness Care Research: Exploring Current Knowledge, Emerging Evidence, and Future Directions, on November 2–3, 2023. Workshop speakers representing a diverse range of disciplines, including nursing, medicine, health economics, biostatistics, social work, and implementation science, explored gaps in serious illness care research, suggested best approaches for addressing those gaps, and identified high-priority areas for future research; the ultimate goal of such research is improving care for people of all ages and all stages of serious illness, their families, and care partners. The 1.5-day workshop had eight sessions. It opened with presentations that provided the critical perspective of individuals living with serious illness, their families, and caregivers. This discussion on lived experience was followed by an overview of the past, present, and future states of serious illness care research. The second and third sessions examined approaches for improving evidence generation to address gaps in the research in methodology and study design (Session 2) and outcome measures and data capture (Session 3). The fourth and final session of the first day featured presentations on barriers and facilitators to integrating health equity in serious illness care research.

The second day opened with a session on the role of implementation science in translating serious illness care research into practice. The final session wove together the key themes raised throughout the workshop and discussed the future of serious illness care research to guide the field.

The sessions included a mix of presentations, panel discussions, and Q&A periods with participants. This Proceedings of a Workshop summarizes the presentations and discussions. The speakers, panelists, and participants presented a broad range of views and ideas. Box 1 summarizes suggestions on ways to expand the evidence base for serious illness care research provided by individual workshop participants. Appendixes A and B contain the workshop Statement of Task and workshop agenda, respectively. The speakers’ presentations (as PDF and audio files) have been archived online.²

___________________

² For additional information, see https://www.nationalacademies.org/event/40554_11-2023_serious-illness-care-research-exploring-current-knowledge-emerging-evidence-and-future-directions-a-workshop (accessed March 25, 2024).

INTEGRATING LIVED EXPERIENCE INTO SERIOUS ILLNESS CARE RESEARCH

Benzi Kluger, the Julius, Helen, and Robert Fine Distinguished Professor of Neurology and Medicine and director of the Palliative Care Research Center at the University of Rochester Medical Center, opened the first session by recounting what he referred to as his “accidental journey” into palliative care. Once he began seeing patients after completing his training as a clinician and researcher, he began to hear heartbreaking stories from them, leaving him feeling helpless and hopeless. “I was not prepared for that in my training,” explained Kluger. “I think the rule in medicine is that you want to have a professional distance,” Kluger noted. He shared, however, that he was lucky to have a mentor—workshop Co-Chair Jean Kutner—who encouraged him to follow his heart (Kluger, 2018). Over the next 2 years,

Kluger listened to and communicated in a new way with patients, and that experience was almost like a third fellowship for him. “Patients and their families taught me what really matters and taught me a lot about how I could be a better doctor for them,” explained Kluger. He shared three key lessons from this experience:

• Clinical researchers need encouragement, community, and practice to follow their hearts.
• Relationships with people living with serious illness in and out of the clinic are crucial to inform research priorities and questions and teach researchers how to do their jobs better.
• Clinical innovation and quality improvement can drive high-quality research.

Motivated by a vote of patients and families at the 2010 World Parkinson’s Congress, Kluger organized the first international working group on palliative care and Parkinson’s disease (Kluger et al., 2017). The working group’s first meeting, funded by a grant from the Parkinson’s Foundation, identified gaps in the serious illness care field and developed an ambitious road map that the field is still following today. Kluger credited the patients and families who participated in the meeting as critical to its success (Hall et al., 2017). Kluger specified that “it was not just respect for people living with serious illness, it was parity of esteem that made their input possible. They were very much the experts and, in fact, were more of an expert in a lot of ways than I was.”

A Patient’s Perspective

In 2013, Kirk Hall, an individual living with Parkinson’s disease, advocate, author, and speaker, was working on a book about his experience with Parkinson’s-related cognition issues (Hall, 2013). In the chapter on palliative care, he questioned whether it is possible for doctors to properly advocate for patients whom they do not know well. Hall urged clinicians to have a frank discussion with their patients to learn what is important to them and their family and explain care options. Kluger and Hall had ongoing discussions about integrating the patient experience into research. Their collaboration led to the development of “a model that included patient and care partner issues at all stages of the disease starting at diagnosis,” explained Hall.

Hall suggested that the Patient-Centered Outcomes Research Institute (PCORI) fund research on palliative care for Parkinson’s disease. Hall and Kluger wrote a grant that funded a 3-year multisite study demonstrating the value of such care (Kluger et al., 2020). Since then, there have been two international conferences on palliative care and Parkinson’s disease, a presentation Hall made at the 2016 World Parkinson’s Congress, and a follow-up study on training and implementing palliative care for Parkinson’s disease at the Parkinson’s Foundation Centers of Excellence. A new Community Outreach and Patient Empowerment program on Parkinson’s disease is under development to increase awareness and availability of palliative care.

Hall offered suggestions for moving the field forward:

• support efforts to build awareness and facilitate expanding palliative care for Parkinson’s disease and neuropalliative care,
• proactively identify individuals for advocacy training,

• engage current advocates to develop and implement training, and
• brainstorm new ideas.

Hall also noted the importance of removing barriers to including music and vocal therapy groups in palliative care programs.

Kluger highlighted the lessons he took away from Hall’s presentation:

• No one knows a serious illness better than the person living it.
• People living with an illness know the real urgency and importance of improving care.
• Patients improve research when treated as experts.

Kluger pointed out that care partners are the least supported and most important members of a care team (Prizer et al., 2020). His team surveyed care partners about how much time they spend providing care, and one respondent said she was working up to 100 hours a week providing care for her loved one. “Having a care partner improves quality of life and improves outcomes for people with Parkinson’s and other serious illnesses,” observed Kluger. Given their key role, Kluger reiterated the importance of involving care partners in research.

A Care Partner’s Perspective

Malenna Sumrall, a patient care partner for her husband and caregiving advocate based at the University of Colorado Anschutz Medical Campus, recounted that when she met Kluger late in her husband’s Parkinson’s disease journey, he was the first doctor in 14 years who asked how she was doing. “It was toward the end of my husband’s life, and the palliative care was just so valuable to us in so many ways. It made me a passionate advocate for palliative care,” she shared. Having served on two PCORI-sponsored advisory councils and with a Ph.D. in educational research, Sumrall explained that she found it interesting to be involved in research as a care partner rather than as a researcher. She was impressed that the projects Kluger and his colleagues were conducting included both qualitative and quantitative components. “It made me realize that the researchers wanted the data to be as rich and as informative as possible,” she said.

Sumrall was also impressed by how highly the researchers valued input from advisory council members. At one council meeting, an individual with Parkinson’s disease mentioned how he always tried to do his best physically at medical appointments; as Sumrall explained, “that led to the concept of ‘holding back,’” which then became an area of investigation in

the research project. As part of that same project, the research team heard from patients and care partners how much they needed to talk to others who understood what they were going through. That led to creating a peer navigator program as part of Kluger’s clinic at the University of Colorado Anschutz Medical Center. Sumrall also started a Zoom support group for care partners. “I realized they do not have the time necessarily to go someplace for an hour or two,” explained Sumrall. “This gave them the flexibility to join in as much as they could.”

Sumrall said that working with care partners has helped her contribute more than just her own experiences and perspective. “I have the voices of others when I participate,” she said. Her volunteering, she added, is her way of contributing to help others. “It is also the best way I can honor my husband. He was a social work professor whose passion in life was helping others, and that is what I hope I am doing now.”

Kluger summarized lessons learned from Sumrall’s presentation:

• Care partners need to be recognized and included as part of the care team in both clinical care and research.
• Care partners have unique perspectives, expertise, and unbelievable amounts of lived experience.
• Care partners need attention and support as individuals and second-order patients in addition to care partners.

Kluger underscored that “PCORI does an amazing job in terms of including people with lived experience and other partners in every aspect of the research journey, from finding out what we should be researching, the agenda, and the evaluation process.” He credited PCORI’s rubric with helping him design an effective engagement plan and changing how many investigators approach research to think about things more pragmatically. Kluger said that he establishes patient and advisory councils for all his projects and advises his mentees to do the same.

Kluger credited PCORI, Hall, Sumrall, and the Parkinson’s Foundation for supporting a randomized comparative effectiveness trial showing that palliative care improved patient and family outcomes more than standard care (Kluger et al., 2020). The research grew directly from his clinic work, and he pointed out that it was successful because of the role the project’s partners played in developing the consent form, engaging participants, and choosing outcomes. They “helped us make better decisions, which helped it be a positive trial as opposed to what we see a lot of, which are negative trials of good palliative care interventions,” emphasized Kluger.

Building on the trial’s results, Kluger and his collaborators worked to integrate palliative care into the Parkinson’s Foundation Centers of Excellence network. Kluger noted that as a result of the key dissemination role played by the foundation, 33 of 34 academic sites he approached have joined the implementation project.

The Parkinson’s Foundation Perspective

Nicole Yarab, vice president for clinical affairs and information and resources at the Parkinson’s Foundation, has worked as a neurology nurse and cared for patients in a multidisciplinary amyotrophic lateral sclerosis clinic. Based on her clinical experience, she advocated for implementing palliative care across the foundation’s Centers of Excellence network. She explained that this work aligns with the foundation’s ethos of putting people with Parkinson’s disease and their care partners first through true patient engagement.

Yarab pointed out that when involving patients and care partners in research, it is important to identify a project’s champions. The power of partnering with a nonprofit, according to Yarab, lies in its connections and resources. The foundation has the trust of the Parkinson’s community and many ways to reach people on a broad basis through its global care network. Yarab has used her position in the organization to champion the implementation effort and alert people about its importance in delivering quality care and improving people’s lives.

In addition to identifying champions, another important aspect is regular meetings with its People with Parkinson’s and Care Partner Advisory Councils and clinician advisors. Collaboration, in Yarab’s view, is part of the “magic sauce.” “This is not something the foundation would have been able to do by itself,” she said, “and I do not think the community can do this work by itself, but if we put all our powers together, we are able to make a real impact.”

Reflecting on Yarab’s presentation, Kluger underscored the importance of working with people who are passionate and know how to get things done. He summarized the lessons learned from working with the Parkinson’s Foundation, noting that nonprofits

• provide key resources to their communities, yet researchers often overlook them as partners;
• have outstanding reach and can accelerate implementation and dissemination efforts; and
• possess unique perspectives and expertise on systems, policy, communication, and community.

Expanding Access to Palliative Care

Kluger distinguished between implementing programs in academic settings and bringing palliative care to the broader population. To take on the challenge of broader dissemination, Kluger secured grants from the National Institute of Nursing Research (NINR) for a project to train community neurologists on the basics of palliative care, which they would otherwise not learn. It also relies on telemedicine to provide chaplains, social workers, pharmacists, and other resources to neurologists in rural areas (Kluger et al., 2024). Noting high rates of burnout among neurologists, Kluger pointed out that the project is also trying to improve the lives of community health providers in addition to patients and care partners (Brashear and Vickrey, 2018).

Kluger highlighted that a key lesson has been fully appreciating how much health care providers are interested in improving care. “They are very busy people and do not have time to be part of clinical trials, but they were honored to be part of this one because they wanted outcomes to be better, they wanted to learn palliative care, and they wanted to get social work, counseling, and other services for the people they serve,” said Kluger. One neurologist said that he valued being part of the study because he felt less alone in caring. “It is frustrating that we have to work so hard against the system to care for patients, and it can feel very isolating,” observed Kluger.

Another lesson, noted Kluger, is that researchers can create alignment by co-learning and collaborating on identifying systems’ issues and solutions. He added that the research team is working with Hispanic communities in Rochester, NY, to learn about their lived experiences so they can translate this model effectively for those communities.

Kluger offered several key reflections and suggestions:

• Research questions, priorities, and outcomes must come from thoughtful collaboration with people with lived experience, including patients, family carers, and other key partners.
• Principles of parity and respect should be part of all aspects of collaborative research with lived experience experts.
• For research to be applicable to underserved communities, additional efforts are needed to capture their lived experience.
• Early career support and community is needed for researchers interested in serious illness care and palliative care.
• Training programs are needed to increase the skills and confidence of persons with lived experience to approach and work with researchers.

• A complementary need exists for training programs to improve the skills and openness of researchers to foster optimal collaborations.
• Persons with lived experience need to have ways to raise research questions and identify collaborators.
• Dissemination plans must consider real-world impact and involve influential community partners, including nonprofits.

Audience Q&A

Kathleen Unroe, associate professor of medicine at the Indiana University School of Medicine and research scientist at the Indiana University Center for Aging Research, asked panelists for ideas on how to include the perspective of and recruit residents in nursing homes and their care partners into research efforts. Sumrall noted that in all settings including nursing homes, it is important to include people who are willing to express their opinions, make them feel comfortable doing so, and have as much participant diversity as possible. Hall said that based on his experience, individuals who are engaged and have meaningful questions and input at support group meetings generally are the type of people researchers would want to include. He added that it is helpful to know the other activities potential advocates are involved in because that provides clues as to what they might bring to the table.

Rebecca Aslakson, professor and chair of the Department of Anesthesiology at the University of Vermont Larner College of Medicine, said that the hardest thing about palliative care research is not finishing the journey one takes with an individual because of that person’s disease. Noting that she feels that research is a patient’s legacy, she asked panelists how to innovatively involve people who may be able to start but not finish it. Hall replied that this is difficult, particularly when people are not getting the help they need. Sumrall said that it is important to realize that involving people in research provides them with some meaning. “You are giving them the opportunity to make lives better for other people, and that is really powerful,” she said.

Kathryn Pollak, professor of population health sciences at Duke University School of Medicine and member of the Duke Cancer Center, asked panelists for their thoughts on how to make research participants feel comfortable when they come from such different life experiences. Kluger added another layer to that question by noting that Parkinson’s disease affects people with different levels of cognitive ability to accommodate and

people from diverse populations. For people with cognitive difficulties, he sends out a meeting agenda and questions in advance, so they have time to consider the questions. Given the importance of parity of esteem, it is key to elevate the ideas of every participant; find what was wise, courageous, and compassionate in what they said; and incorporate their ideas into research. “If you are able to do that consistently, you create a culture and a family where everybody feels safe and welcome,” said Kluger.

Deborah Swiderski, attending physician and associate professor of medicine and family and social medicine at Montefiore Einstein, asked Kluger to elaborate on his ideas regarding training. Kluger replied that the Parkinson’s Advocates in Research program conducts training programs for patients and care partners who want to be involved in research. More remains to do, however, to engage people from underserved communities who are not at academic medical centers to enroll in the program. Kluger pointed to his work with Community Health Ministries to develop a train-the-trainer program. He emphasized that the goal is to get people to see themselves as experts and understand they have a great deal to contribute. Yarab added that the Parkinson’s Foundation works with community partners and its Centers of Excellence to engage with people who are not normally involved in research. The key is to take advantage of the connections and relationships its partners have built and have experts and representatives of those communities help design a program that is culturally relevant and meaningful.

Charlotta Lindvall, assistant professor of medicine and Director of Clinical Informatics, Dana Farber Cancer Institute, noted that the oncology community is developing centralized websites where researchers can reach out to diverse populations to enroll in clinical trials and wondered whether there were similar websites to enable patients and their care partners to engage in research. Kluger responded that one of the wonderful aspects of the Parkinson’s community is that care partners are recognized and embraced. He noted that the Michael J. Fox Trial Finder³ provides that capability, and the Parkinson’s Foundation was part of a study his group conducted that enrolled patients and care partners. Yarab added that the foundation offers other opportunities to reach care partners through its annual Care Partner Summit, helpline that provides connections to a variety of services and programs, and weekly offering through the Parkinson’s Disease Health at Home program.

___________________

³ Additional information is available at https://www.michaeljfox.org/trial-finder (accessed March 16, 2024).

EXPLORING THE STATE OF THE SCIENCE OF SERIOUS ILLNESS CARE RESEARCH:
PAST, CURRENT, AND FUTURE

Sean Morrison, Ellen and Howard C. Katz Professor and Chair of the Brookdale Department of Geriatrics and Palliative Medicine and director of the National Palliative Care Research Center (NPCRC) at the Icahn School of Medicine at Mount Sinai, reminded participants that the goal of developing the evidence base is to appropriately care for, relieve suffering, and improve the quality of life for people living with serious illness and their loved ones. He shared the definition of serious illness as “a health condition that carries a high risk of mortality and either negatively impacts a person’s daily function or quality of life or excessively strains their care partners” (Kelley and Bollens-Lund, 2018).

Morrison pointed out that people often conflate palliative care and serious illness. “Serious illness is the population who we are attempting to care for. Palliative care is a delivery system by which we care for that population, and palliative care is beneficial at any stage of a serious illness,” he explained. Morrison referred to the definition of palliative care developed by the National Consensus Project for Quality Palliative Care (2018)⁴:

Morrison highlighted three key events in the United States in the mid-1990s that focused attention on the inadequate care provided to people with serious illness. The first was the AIDS epidemic; on average, a person living with AIDS experienced 17 symptoms daily, about twice as many as someone with metastatic cancer, and many were marginalized, with no effective treatments for the disease or to manage symptoms.

The second event was the emergence of the assisted suicide movement led by Dr. Jack Kevorkian. Morrison pointed out that its growth underscored the poor quality of care provided by the U.S. health care system, as individuals preferred to end their lives rather than experience the care provided in hospitals at the time.

The third event was the publication of the results of the Study to Understand Prognoses and Preferences for Outcomes and Risks of

___________________

⁴ For more information, see https://www.nationalcoalitionhpc.org/ncp/ (accessed January 5, 2024).

Treatments trial (SUPPORT Principal Investigators, 1995). The trial enrolled 9,000 patients at five of the top-rated U.S. teaching hospitals; 50 percent of them experienced moderate to severe pain over half the time before they died, and 40 percent spent the last 10 days of their lives in an intensive care unit (ICU) on a ventilator or in a coma. “That was the state of care for people with serious illness in the mid-1990s,” observed Morrison.

Turning to the topic of funding research to improve the quality of life for people with serious illness, Morrison pointed out that the National Institutes of Health (NIH) awarded 64 grants in 1995 (24 were educational rather than research), totaling slightly more than $9 million, which represented 0.069 percent of total NIH funding. Morrison referenced Amy Berman of the John A. Hartford Foundation, who frequently pointed out that “0.069 rounds to zero.”

Morrison explained that as of the mid-1990s, only 719 U.S. hospitals—15 percent of all U.S. hospitals—reported that they offered “end-of-life care services.” Five palliative medicine fellowship training programs, each graduating approximately two fellows a year, existed in 1997. That year, Morrison explained, the Institute of Medicine⁵ released the first of six reports on end-of-life care, which concluded that most people with serious illness experience inadequately treated symptoms, fragmented care, poor communication with clinicians, and strains on care partners and support systems (IOM, 1997). The report also concluded that

• legal, organizational, and economic obstacles obstruct reliably excellent palliative care;
• education and training of health care professionals fail to provide the necessary knowledge and skills required to care for the seriously ill; and
• the knowledge base is inadequate to support evidence-based palliative care clinical practice.

Tracking the growth in palliative care throughout the ensuing two decades, Morrison pointed out that 100 percent of U.S. teaching hospitals had a palliative care team in 2015, and by 2019, more than 70 percent of all U.S. hospitals had palliative care teams (see Figure 1). Morrison added that by 2020, nearly 85 percent of hospitals had palliative care teams. As of 2022, there were more than 180 palliative care fellowship slots available in the United States (see Figure 2) and almost 8,000 board-certified palliative medicine physicians (see Figure 3).

___________________

⁵ As of March 2016, the Health and Medicine Division of the National Academies of Sciences, Engineering, and Medicine (National Academies) continues the consensus studies and convening activities previously carried out by the Institute of Medicine (IOM). The IOM name is used to refer to reports issued prior to July 2015.

___________________

⁶ The National Palliative Care Registry, a joint project of the Center to Advance Palliative Care (CAPC) and the National Palliative Care Research Center (NPCRC), was active from 2008-2020, and its goal was to build a profile of palliative care teams, operations, and service delivery. In fall 2020, a national registry for the collection of palliative care data launched. The National Palliative Care Registry™, the Palliative Care Quality Network (PCQN), and the Global Palliative Care Quality Alliance (GPCQA) united into one national system called the Palliative Carer Quality Collaborative (PCQC) available at https://www.palliativequality.org (accessed July 1, 2024).

⁷ See https://aahpm.org/fellowships/match and https://www.nrmp.org/match-data-analytics/fellowship-data-reports/ (accessed July 1, 2024).

The growth of palliative care did not happen by accident, emphasized Morrison, but was a result of a sustained strategy. First, unlike any other field of medicine, philanthropy built this field. Morrison pointed to early funders, such as the Robert Wood Johnson Foundation and Open Society Foundation, along with many others, that spent over $250 million to build the field and develop the evidence base that palliative care improves care for people living with serious illness.

Morrison highlighted the key contributions of George Soros, who funded the Project on Death in America (PDIA)⁹ in 1995, an initiative conceived to create the PDIA Faculty Scholars Program. The program developed leaders in U.S. academic medical centers to build the field by leveraging a small amount of money to install pillars in these centers because that is where the nation trains its health care professionals and researchers conduct the studies needed to advance the field. “When we reflect back, we owe a huge debt of gratitude to private sector philanthropy in this country, for this would not have happened without both commitment and their vision,” Morrison said.

Morrison explained that Diane Meier originally developed a strategy to advance the field of palliative care focused on three key areas: improving education and training, becoming a recognized specialty through board certification, and building the evidence base. The strategy featured both a top-down and bottom-up component, each reinforcing the other.

___________________

⁸ See https://www.abms.org/ (accessed July 1, 2024).

⁹ For more information, see https://www.opensocietyfoundations.org/publications/transforming-culture-dying-project-death-america-1994-2003 (accessed January 18, 2024).

The top-down strategy called for creating a supportive environment using the media, advocacy, public awareness, accreditation, regulation, and payment. The bottom-up strategy involved developing an evidence base, enhancing the workforce, and increasing the number and quality of palliative care programs.

Morrison described how the research effort proceeded according to a set of guiding principles, starting with creating a unified scientific community that brings researchers together to work collaboratively, with a shared vision, to maximize the use of existing resources. The second principle was to align research with the prevailing scientific culture and infrastructure. “The field of science is very conservative, and you do not get very far if you are way outside,” explained Morrison. “You have to belong to the culture.” As an example, he described framing his first NIH grant proposal around hip fractures and not palliative care and pain relief because at the time, the National Institute on Aging (NIA) had a focus on hip fractures. Thus, he was able to conduct the research he wanted because it aligned with NIA’s funding interests.

The third principle was to focus on what is scalable, has low unit costs, and reduces opportunity costs without reinventing the wheel. “We were too small a field at that time for everybody to be doing the same thing over and over and over again,” said Morrison. The final principle was to create long-term sustainability because, despite the incredible investment philanthropy had made to jump-start the field, philanthropic dollars are scarce, and funders can change their priorities.

Morrison pointed out the common misperception that NIH is the primary funder of biomedical research. In reality, industry sponsors two-thirds of it, most of that allocated to developing drugs and devices (see Figure 4) (Teconomy Partners, 2022). Morrison noted that only 10 percent of industry spending goes to health services research, whereas 76 percent goes to biopharmaceuticals.

NIH accounts for nearly 80 percent of federal funding on biomedical research. “If you work through all those percentages, we compete for about 9.5 percent of all industry funding,” explained Morrison. “If you look at philanthropy, we are competing for about 1.2 percent of total funding, and if you look at NIH, we compete for about 20 percent of NIH funding.”

Morrison pointed out that a key milestone was the 1997 workshop on end-of-life care convened by Patricia Grady, the director of NINR. This led to the NIH director establishing the NINR Office of End-of-Life and Palliative Care Research, which served as a home for research on people with serious illness for more than two decades.

To develop a road map and strategy for research, NINR’s Office of End-of-Life and Palliative Care convened a state of the science conference focused on the needs of people with serious illness. The recommendations developed during this meeting included the following:

• Create a network of investigators and well-defined cohorts of patients to facilitate coordinated interdisciplinary, multisite studies.
• Explore public–private partnerships related to end-of-life research support.
• Enhance training of a new generation of interdisciplinary scientists.

A confluence of personal and foundation interest led to establishing NPCRC at Mount Sinai, where the Center to Advance Palliative Care (CAPC) was located. Morrison said that NPCRC’s mission is to create a new generation of scientific leaders to address workforce needs and close knowledge gaps in serious illness care through 2-year career development awards for early-stage investigators and pilot award funding for senior investigators. In addition, NPCRC has been providing technical assistance for investigators to better compete for federal funding to build the evidence base, create consensus around priorities for research across the lifespan, and establish and nourish a diverse national interdisciplinary community of serious illness scientists.

Morrison explained that with funding from NPCRC, Jean Kutner and Amy Abernethy organized a meeting to develop a strategy for creating a palliative care research cooperative (Abernethy et al., 2010). “This was important because not only did they develop this idea, but they also socialized it and sold it,” said Morrison. As a result, NINR decided in 2013 to fund the

Palliative Care Research Cooperative (PCRC), which has focused on clinical trials over the subsequent decade. PCRC’s strategy has been to provide technical assistance, training, and pilot awards for scientists at all levels of experience in conducting clinical studies. It has also been coordinating and resourcing nationally representative, multi-institutional studies that include diverse populations; providing a research infrastructure; and establishing a cooperative group-like structure to foster recruitment for clinical trials.

Noting NPCRC’s significant contributions to the field, Morrison explained that in 1997, only 18 institutions had a critical mass of three or more NIH R01-funded palliative care researchers. By 2022, that number was more than 70. Between 2006 and 2020, Morrison noted, over half of NIH serious illness research grants went to an NPCRC grantee or PCRC member, and nearly 60 percent of early-stage investigator awards went to NPCRC grantees.

NIH funding for serious illness research increased from $60 million in 2001 to more than $304 million in 2022, Morrison noted, and NIA is now the largest funder (see Figure 5) (Brown et al., 2018; Buehler et al., 2022; Gelfman et al., 2013). “I think all of us in the field owe a huge debt of gratitude to the people at NIA for recognizing that serious illness needed a home, and they were supportive of that,” emphasized Morrison. From 2016 to 2020, NIA contributed approximately $140 million into research focused on older adults with serious illness.

In terms of the top-down strategy, Morrison said that NINR’s support has been critical, but NINR is a small institute with limited funds to support research. NIA was a good candidate to host more serious illness research, particularly since the National Alzheimer’s Project Act (NAPA)¹⁰ has provided millions of dollars to NIA to focus on Alzheimer’s disease. NAPA’s lack of a focus on people with advanced dementia prompted Morrison and his team to put together a list of evidence-based priorities to identify gaps in future directions in research for people with advanced dementia (Mitchell et al., 2012). With those priorities added to NAPA, NIA had a road map for supporting research on how to care for people with advanced dementia. To further inform NIA, Morrison explained, NPCRC partnered with NIA to convene a conference to establish research priorities in geriatric palliative care. The result was a series of papers published in the Journal of Palliative Medicine (Allcroft et al., 2013; Allen et al., 2013; Ersek et al., 2013; Hwang et al., 2013; Kelley, 2013; Kerr et al., 2013; Liu et al., 2013; McNamara et al., 2013; Morrison, 2013; Stanford et al., 2013; Xhixha et al., 2013), which sparked an NIA program focusing research on the needs of older adults with serious illness.

Morrison noted that a series of fortunate coincidences led CAPC and NPCRC to connect with Rebecca Kirch and colleagues of the American Cancer Society’s Cancer Action Network,¹¹ through which they learned about the importance of advocacy. Morrison recounted that the American Cancer Society understood from its members that the needs of people living with advanced cancer were not being met and partnered with NPCRC and CAPC to insert language about enhancing research for people living with serious illness into Senate appropriations. Morrison explained that the Senate Labor Health and Human Services committee strongly urged in its fiscal year (FY) 2011 appropriations report that NIH should develop a trans-institute strategy for increasing funded research in palliative care for persons living with chronic and advanced illness.¹²

NPCRC, CAPC, and the American Cancer Society, along with the American Academy of Hospice and Palliative Medicine, formed the Patient

___________________

¹⁰ For more information, see https://aspe.hhs.gov/collaborations-committees-advisory-groups/napa (accessed January 18, 2024).

¹¹ For more information, see https://www.fightcancer.org/ (accessed January 19, 2024).

¹² For more information, see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3214714/ (accessed January 19, 2024).

Quality of Life Coalition (PQLC)¹³ in 2013, which included more than 40 advocacy organizations committed to improving care for people with serious illness. Morrison explained that the coalition serves as a unified voice to advocate on Capitol Hill for the needs of patients and their families. PQLC’s advocacy led to the Senate Labor Health and Human Services committee calling on NIH to increase support for palliative care research in its FY2019 appropriations report, noting that “research funding for palliative care, including pain and symptom management, comprises less than 0.1 percent of the NIH annual budget.” Morrison noted that the Senate committee strongly urged NIH to increase its support for palliative care research.

Morrison explained that after additional discussions with several Senate leaders in 2023, the FY2023 Congressional Consolidated Appropriations Act included the following language:¹⁴

Morrison added that NIH provided the requested update, stating in part that it would lead efforts to convene subject matter experts from across the NIH institutes (including the National Cancer Institute, National Institute of Child Health and Human Development, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, and NINR) to expand and strengthen the strategic coordination of palliative care research while working to identify future areas for research.

NIA issued a comprehensive response to Congress and claimed the lead role in convening subject matter experts from across the NIH institutes and centers to expand and intensify the strategic coordination of palliative care research efforts and identify future research topics. “What that meant was that there was now alignment at NIH for what [the field] wanted to do,” explained Morrison. The Appropriations Committee, with overwhelming bipartisan support, was then comfortable designating $12.5 million for NIA to implement a trans-institute, multi-disease strategy to focus, expand, and intensify national research programs in palliative care. In October 2023, NIA announced it would dedicate $66 million over 5 years to create a

___________________

¹³ For more information, see http://patientqualityoflife.org/ (accessed January 19, 2024).

¹⁴ See https://officeofbudget.od.nih.gov/pdfs/FY24/br/NIH%20FY%202024%20CJ%20Significant%20Items%20Volume%20final.pdf (pp. 94-96; accessed January 19, 2024).

consortium and trans-institute strategy to promote palliative care research.¹⁵ Morrison emphasized that although NIA has been designated as the lead institute, the focus is on serious illness care research across the lifespan.

Morrison closed his remarks with a quote from Sir Winston Churchill: “Now, this is not the end. It is not even the beginning to the end. But it is, perhaps, the end of the beginning.” Morrison shared that it is his hope that the workshop’s discussions would set the stage for the next phase of serious illness research in this country.

Audience Q&A

Session moderator Aslakson opened the Q&A session by asking Morrison what is next for the field of serious illness care given the potential for increased NIH funding. Morrison responded that some general principles are that research should include the entire lifespan, include multiple diseases—investigators will have to leave their silos—and be interdisciplinary in its approach to accounting for the factors—such as who pays for care—that can influence and improve care. As funding for palliative care research is still only about 1 percent of the NIH budget, it will be important to be strategic over the next 5 years regarding how the field can most effectively leverage the resources it has spent 20 years trying to secure. “We have to have a shared agenda,” Morrison emphasized. “We have to come together the way we did in terms of planning the field.” In Morrison’s view, the available funds need to go to scientific research that will make a difference tomorrow, not in 20 years.

Erik Fromme, faculty member in Ariadne Labs’ serious illness care program, asked Morrison to elaborate on his point about researchers getting out of their silos. Morrison replied that NIA wants a transdisciplinary and trans-institute approach, and although NIH has struggled with how to promote such efforts, he believes things might be different for palliative care. First, funding is allocated to this effort, and second, it has leadership. “There is a group of people at both the top and mid-levels of NIH who are committed

___________________

¹⁵ For more information, see https://www.nia.nih.gov/about/budget/fiscal-year-2024-budget (accessed January 19, 2024). See also https://grants.nih.gov/grants/guide/notice-files/NOT-AG-23-050.html (accessed January 19, 2024). In March 2024, Congress passed an FY2024 appropriations package that will provide $12.5 million in funding for NIA to implement a trans-institute, multi-disease strategy to focus, expand, and intensify national research programs in palliative care across the lifespan. See https://www.appropriations.senate.gov/imo/media/doc/fy24_lhhs_bill_summary.pdf (accessed May 9, 2024).

to this, both at a personal and professional level, and we have not seen that before,” said Morrison. He noted that he is not naive enough to expect every silo to disappear but is optimistic because the proposed consortium includes the large NIH institutes, including the National Cancer Institute.¹⁶

Rebecca Kirch, executive vice president of policy and programs at the National Patient Advocate Foundation,¹⁷ said that one of the many things palliative care has done well has been its engagement with patients and families. The next step, she said, is to integrate the voices of those who are underrepresented. Her organization’s infrastructure is designed to help link researchers to underrepresented, lower-income, and rural populations and people of color who do not trust the system or clinical trials. The opportunity, she said, is to bring patients from those populations into advisory groups as experts and not as tokens.

IMPROVING EVIDENCE GENERATION TO ADDRESS GAPS IN SERIOUS ILLNESS CARE RESEARCH: RESEARCH METHODOLOGY/STUDY DESIGN

Dio Kavalieratos, associate professor and director of research and quality in the Division of Palliative Medicine at Emory University’s Rollins School of Public Health, reflected on the growth and progress of scientific inquiry and increased rigor in serious illness research over the past two decades. He cited a 2008 systematic review of clinical trials for specialized palliative care intervention that found scant evidence that it was effective (Zimmermann et al., 2008) and noted that the evidence base was hampered by methodological limitations. Twelve years later, however, another systematic review found multiplicative growth in the number of studies of specialized palliative care (Ernecoff et al., 2020) and highlighted the diversification of the breadth and depth of the methods used in serious illness care research. “Those of us in serious illness research know all too well that the populations, the settings, and the context within which we work make our research different,” noted Kavalieratos. “Standard methodologies, like a traditional randomized clinical trial, often do not perform as well when they are taken off the shelf and applied in serious illness contexts.” Rather, he said, such trials require remarkable scientific expertise and agility to understand when and how to adapt those methods to fit the purpose of the study and the reality of the population being studied.

___________________

¹⁶ An updated overview is available at https://grants.nih.gov/grants/guide/rfa-files/RFA-AG-25-002.html (accessed March 22, 2024).

¹⁷ For more information, see https://www.npaf.org/resources/patient-insight-institute-2/ (accessed January 5, 2024).

Mixed Methods Research and Evaluating Complex Interventions

Douglas White, vice chair and professor of critical care medicine, and director of the Program on Ethics and Decision Making in Critical Illness at the University of Pittsburgh School of Medicine, characterized qualitative and mixed methods research as “crucial tools to improve serious illness care.” He defined qualitative research as “a type of research that aims to gather and analyze non-numerical data in order to gain an understanding of individuals’ social reality, including understanding their attitudes, beliefs, and motivations” (Tashakkori and Teddlie, 2010). Mixed methods research, White said, “includes both qualitative and quantitative data collection and analysis in parallel form or in sequential form, in which one type of data provides a basis for collecting another type of data” (Tashakkori and Teddlie, 2010). An example of the latter, he explained, would be doing qualitative research and developing a quantitative survey based on those results.

The purposes of quantitative and qualitative research are quite different, explained White. Quantitative research aims to establish incidence and prevalence, measure risks and frequency of events, and determine treatment efficacy in rigorous trials. The goal of qualitative research, on the other hand, is to describe a phenomenon with rich texture, understand attitudes and behaviors in a deeper way than can be obtained from a quantitative survey, and describe why interventions tested in large clinical trials do or do not work. Both methods are part of the crucial work needed at the formative stage of research if the goal is to improve serious illness care, as White noted: “We need to understand the nature of the problem or problems if we are to intervene successfully to address them.” He added that it is also important to understand where dominant paradigms and extant policies do not fit either method because of insufficient consideration of diverse views or insufficient inclusion of groups that have historically been marginalized.

White gave two examples of qualitative research studies in serious illness care to illustrate those points. In one study, investigators conducted focused ethnography to understand Navajo perspectives regarding disclosing negative information and talking about negative health status in the future (Carrese and Rhodes, 1995). This study found that the dominant Western paradigm of needing to talk about what an individual wants in the context of becoming seriously ill is in tension with cultural views in the Navajo community around not talking about negative future events. “That was an important study drawing out concerns about whether our

Western bioethical perspective fully fits with the cultures that we serve,” said White.

The second study characterized key intrapersonal tensions experienced by surrogate decision makers in ICUs and explored associated coping strategies (Schenker et al., 2012). In this case, the problems are not solely informational, as deep existential and psychological challenges arise in making end-of-life decisions for loved ones.

White shared an example of a mixed methods study he and his collaborators conducted, in which quantitative data suggested that surrogate decision makers for critical illness often have overly optimistic expectations about their loved one’s prognosis; the team wanted to understand why this might be so (White et al., 2016). The assumption most people make is that decision makers are misunderstanding the information they receive from clinicians. To investigate, White’s team interviewed surrogate decision makers and physicians during patients’ stays in the ICU and asked them what they believed the chances of survival were. The researchers found that 43 percent of surrogates held overly optimistic expectations compared to the physician. This was more common for Black or strongly religious surrogates. White’s team also asked the surrogates about the doctor’s perspective on survival. The answers revealed that optimistic expectations arose from both miscomprehension and differences in belief.

As a follow-up, the team interviewed the surrogates who did not believe the physicians and asked them to explain why that was the case (White et al., 2019). White explained that the qualitative portion of the project provided information that was illuminating and had significant implications for the design of future interventions. In general, the answers fell in three categories: a belief that they know the patient’s fortitude better than the physician, optimism grounded in religious beliefs, and defensive processing of prognosis. “I would just point out that each of these reasons is not really amenable to a decision aid or an information-focused intervention, and this tells us that we need to go looking in different places for the kind of interventions that may work,” explained White.

White cited two papers on reporting and designing qualitative research (O’Brien et al., 2014; Tong et al., 2007) and pointed out that when evaluating complex interventions to improve serious illness care, rigorous strategies are needed to monitor and maintain fidelity. An intervention might be complex because of its own properties, such as the number of components; range of behaviors targeted; or expertise and skills required (Skivington et al., 2021). Although many factors can make an intervention complex, the important point, said White, is that almost everything in serious illness

care involves complex interventions. “We have not quite conceptualized it in that way, to our detriment,” observed White.

White emphasized that it is critical to monitor and maintain intervention fidelity—the degree to which it is delivered as planned—“because that affects how to interpret both positive and negative trials.” For example, if a trial result is negative, it is important to know whether the intervention was delivered per protocol or did not work despite sufficiently high fidelity. The latter provides useful evidence to guide interventions. White highlighted three keys to fidelity:

• specify the core components of the intervention when designing it,
• train the interventionists and ensure they demonstrate proficiency to deliver it, and
• measure the extent to which it is being delivered (Bellg et al., 2004; Resnick et al., 2005).

Also important, said White, is studying the extent to which participants enacted the intervention and acquired the necessary treatment-related behavioral skills or cognitive strategies.

White shared an example of a complex randomized trial his team conducted in which a family support specialist was added to an ICU team to deliver four kinds of support to surrogates throughout the ICU stay (Seaman et al., 2018). None of the supports improved family-centered outcomes, such as depression, anxiety, or posttraumatic stress disorder; surrogates’ perceptions of the patient-centeredness of care or other patient outcomes; or health care use. He emphasized that although the results were negative, they were nonetheless informative because the intervention was delivered with very high fidelity.

White highlighted an interesting finding from the qualitative and quantitative interviews he and his collaborators conducted: the surrogates perceived that the interventionist gave them very high levels of emotional support and that the quality of the communication was very high. Unfortunately, they showed no improvement in clarity about patient values or misunderstanding the prognosis. “That tells us that this particular approach was not effective in intervening in what we think are probably important targets on the causal pathway,” explained White.

White concluded his remarks by reinforcing two key points: “Qualitative and mixed methods research are crucial tools to improve serious illness care, and when evaluating complex interventions to improve serious illness care, we need strong plans for monitoring and maintaining intervention fidelity.”

Identifying Effective Components of Serious Illness Care

J. Nicholas Odom, the Doreen C. Harper Endowed Professor of Nursing, University of Alabama at Birmingham (UAB) School of Nursing, and director of caregiver and bereavement support services, UAB Center for Palliative and Supportive Care, spoke to the challenge of rigorously evaluating the different components of a palliative care intervention to determine which are effective. He argued that one way to address that challenge is the multiphase optimization strategy (MOST), an approach to intervention development and testing capable of identifying the effective components of serious illness care interventions (Collins, 2018; Collins and Kugler, 2018).

Odom cited a 2016 systematic review and meta-analysis of palliative care interventions, which noted that “future research should aim to identify the efficacious component(s) of palliative care” (Kavalieratos et al., 2016). “Having been part of this systematic review, I can tell you it is very difficult to systematically identify what is a palliative care intervention,” said Odom.

Typically, investigators take what Odom called the “classic bundled package” approach to intervention development and testing (see Figure 6). It combines an intervention’s components into one package and then tests the “bundle” in a randomized controlled trial. However, even if the trial is positive, the results cannot identify which components were effective and which were not, whether one component had an effect on another, if a component’s contribution is enough to offset its costs, and if all the components are necessary. Ultimately, said Odom, the results cannot point to the next steps to make the intervention more effective, efficient, and scalable.

An intervention component, explained Odom, is any aspect that can be separated out for evaluation, such as the following:

• Program content, such as disease education or health promotion;
• Amount or exposure, whether the intervention occurs over single or multiple sessions;
• Delivery mode, such as in-person, telehealth, and text messaging;
• Behavior change technique, such as motivational interviewing or cognitive behavioral therapy;
• Adherence strategy, such as reminder phone calls and text messages;
• Fidelity measures, such as enhanced clinician training and education;
• Timing, whether the intervention occurs early or later;
• Interventionist type, such as community health workers, social workers, nurses, and physicians; and
• Drug being tested, such as comparing methylphenidate against dextroamphetamine for fatigue.

Odom explained that within the MOST framework, components are evaluated both together and individually in an optimization trial, allowing investigators to determine the performance of individual components and how they interact with one another (see Figure 7). MOST does not do away with randomized clinical trials, explained Odom: “It is still included in the steps of intervention, development, and testing.” He noted that the next step would be to compare the new optimized intervention against usual care to confirm efficacy.

Odom explained that optimizing an intervention means identifying an intervention that provides the best expected outcome within key constraints, including the following:

• Effectiveness—is the intervention beneficial?
• Affordability—does the intervention offer good value for the expended costs?
• Scalability—can the intervention be implemented widely with fidelity?
• Efficiency—does the intervention minimize resource use and avoid waste?

MOST begins with a preparation stage in which researchers devise and revise their conceptual model, identify a set of candidate components, and identify their optimization criteria, noted Odom. That is followed by an optimization trial, which includes multiple types, such as a factorial experiment, micro-randomized trial, and sequential multiple assignment randomized trial. The results can identify the intervention components that meet all optimization criteria. “If it looks like it’s going to be sufficiently effective and affordable, you can then evaluate it via a traditional randomized controlled trial,” Odom explained.

Factorial trial designs, explained Odom, provide an efficient method to screen intervention components, allowing researchers to identify under-performing ones and determine the magnitude of each component’s effect (Collins, 2018; Dionne-Odom et al., 2022; Gazaway et al., 2023b). The factorial trial designs also allow for examining whether one component augments or reduces the effect of another and enable optimizing the scalability of the intervention package by including costs and other resource use parameters.

Odom said that a key research priority is “to look more at the genetics of our field” and assess active components of serious illness care interventions and services. MOST is a framework for developing and testing multicomponent interventions, and within it, a factorial trial design enables testing the individual components of a multicomponent intervention. “The beauty of these designs is you do not just have to look at effectiveness, you can look at affordability, you can look at scalability, and you can look at efficiency,” explained Odom.

Pragmatic Trials in Serious Illness

Corita Grudzen, division head of Supportive and Acute Care Services and Fern Grayer chair in oncology care and patient experience at Memorial Sloan Kettering, and professor of emergency medicine at Weill Cornell Medical College, explained that she uses the NIH stage model (see Figure 8)

when planning behavioral interventions, which most palliative and serious illness care interventions are. She focused her remarks on stages IV and V—effectiveness and implementation and dissemination.

The aim of a pragmatic trial, explained Grudzen, is not to determine whether an intervention can work under ideal conditions but whether it will work under usual conditions. In a complex intervention, while adaptation is embraced, it should be studied and guided. Complex health interventions have complex causal pathways, and an estimate of average effect size is not predictive but is context sensitive; it might work at a major medical center but not at a community hospital, for example. The main effects in complex interventions are often weak, with outcomes strongly influenced by context, and impacts are often indirect and mediated by other factors and thus can be variable and attenuated. As a result, pragmatic trials often produce weak effects

with high variance compared to those from a typical trial. Pragmatic trials do not produce evidence but insights and guidance, emphasized Grudzen. Simple interventions provide guidance and dichotomous “yes” or “no” type answers. Complex serious illness interventions instead yield insights and guidance for practice. They answer the Ws—why, where, when, and for whom does it work—along with how to enhance their effectiveness and how they work.

Grudzen suggested that another way of thinking about complex interventions and pragmatic trials is to consider form and function rather than components (Perez Jolles et al., 2019). This approach specifies core functions in advance, such as the purpose, intended effect, and link to patient needs. Form refers to the format—telehealth versus face-to-face delivery, for example—activity, and operationalization. “The form can be specified in advance and adapted, but there has to be fidelity to the function in a complex, pragmatic trial,” she explained.

As an example of this approach, Grudzen described the Emergency Departments Leading the Transformation of Alzheimer’s and Dementia Care study she and her collaborators are conducting to examine the effectiveness of three transitional care interventions, alone and in combination. Grudzen explained that the cluster-randomized, multifactorial trial will be embedded in 80 emergency departments, with emergency department revisits, hospitalizations, and healthy days at home following the visit as the outcomes. The three interventions, all based on previous research, include the following:

• emergency department redesign of new and intentional workflows for providers reinforced by digital alerts and structured collaboration between sites, an intervention that Grudzen’s testing showed increases identification of advance care plans and enlisted multidisciplinary support (Gettel et al., 2022);
• nurse-led telephone care program that can increase advance care planning and patient connection with hospice (Liddicoat Yamarik et al., 2023); and
• community paramedic–led structured coaching intervention that can reduce the odds of an emergency department visit within 30 days by 75 percent (Shah et al., 2022).

Grudzen explained that she and her colleagues used the Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) wheel to determine where they wanted their trial to be on the pragmatic/explanatory continuum (based on how it scored according to the PRECIS-2 domains)

(see Figures 9 and 10) (Kirsty et al., 2015). She noted that the trial will be open to anyone who comes to any of the 80 emergency departments and has two diagnostic codes for dementia, with recruitment done as part of usual care. A combination of existing emergency department staff and any new personnel will deliver the intervention. Outcomes include how flexible the intervention is in terms of providers adhering to the trial protocol, how much contact with patients is needed to get the outcomes, whether outcomes can be obtained as part of usual care, and health care utilization. Grudzen specified that given that this is a pragmatic trial, it will use intention-to-treat rather than only looking at the patients who received the intervention.

In closing, Grudzen emphasized that pragmatic trials in serious illness care are inherently complex, multicomponent transitional care interventions. Such trials must specify, track, and maintain core functions with integrity. Moreover, mixed methods approaches are required to provide why, how, when, where, and in whom the interventions work.

An Economic Perspective on Serious Illness Research

Mireille Jacobson, associate professor and assistant dean in the Leonard Davis School of Gerontology and codirector of the Aging and Cognition Program in the Schaeffer Center for Health Policy and Economics at the University of Southern California, pointed out that 60 percent of U.S. adults have at least one chronic disease, and 40 percent have two or more (CDC, 2022). In addition, a significant percentage of those living with serious illness experience distress beyond the physical symptoms associated with their condition, such as feeling anxious, confused, helpless, isolated, or left out and lacking companionship (Schneider et al., 2018). Almost half of all individuals with serious illness have emotional or psychological problems caused by their condition, and 61 percent reported at least one problem while receiving care (Schneider et al., 2018). Jacobson explained that they encounter a range of problems, including difficulty understanding a medical bill or what their health insurance will cover, being sent for duplicate tests or diagnostic procedures, and getting conflicting recommendations from different health professionals. Moreover, a huge source of anxiety is stress related to paying for care.

Considering how health care systems can do better and to make the best use of limited resources, Jacobson noted “a stunning amount of evidence” from over 40 randomized controlled trials that investing in palliative care improves patient and care partner outcomes (Kavalieratos et al., 2016). While acknowledging the negative trials and that there is much to learn about how to best deliver palliative care, Jacobson noted that “we should not lose sight of the really positive findings about improvements in quality of life for patients.”

Jacobson identified two important gaps related to the issue of why adoption of palliative care is not more widespread given the strong evidence supporting its effectiveness. First, questions exist about external validity and whether programs showing positive results can be replicated. She noted that although randomized controlled trials are the gold standard, investigators on older studies conducted them in highly controlled environments, such as academic medical centers, so they do not produce real-world evidence. The field is starting to address this gap by focusing on multisite, nonacademic settings.

Another approach, said Jacobson, is to prioritize quasi-experimental research designs that leverage features of the natural world and environment to mimic randomization and attempt to get closer to causal inference using real-world evidence. One benefit of that approach is that trials and subsequent analysis can cost less, but the challenge is identifying a source of random variation in terms of who has access to palliative care. Nonetheless, Jacobson is confident that someone will use this approach to develop evidence

that will move the needle. Moreover, research quasi-experimental designs exist that are deployable and generate evidence more quickly than a long, randomized controlled trial. “I think it is worth spending some resources both in terms of time and money on this type of design,” Jacobson noted.

The second gap relates to payment policy, which Jacobson suspects is a major barrier to health system adoption of palliative care outside of the hospital setting. “Clearly, we do not have strong incentives to provide [palliative care], but we should get real evidence of whether this is a fundamental barrier,” she said. She suggested that one way to address the lack of real evidence is to use descriptive evidence from an international context. Taiwan, for example, has a robust palliative care system, and lessons can be learned from its experience. The U.S. Veterans Administration could also illuminate whether payment policy is a core issue. Jacobson also suggested that a pragmatic trial could examine different incentives to see how they affect adoption with an eye toward gaining insights to inform replication efforts.

Jacobson emphasized that whether palliative care saves the health system money is the wrong question. The better question, in her view, is whether it brings value––that is, whether the benefits outweigh the costs. As an example of a high-cost, high-value intervention, Jacobson noted that hepatitis C treatments, although expensive, pass any cost-effectiveness analysis because they extend life. The challenge for palliative care is attributing value to quality-of-life benefits for patients and their care partners. “How do we measure that?” she asked. “People are working on [it], but we need more to assess what we are getting from palliative care.”

Jacobson noted that the aging of the U.S. population will increase the demand for serious illness care. Given the data she discussed at the beginning regarding how patients and their care partners feel about the care they are receiving, it will be critically important to identify and generate evidence to improve the quality of care for individuals with serious illness.

Statistical Methods, Study Design, and Workforce Training

Kathryn Colborn, associate professor, director of the Adult and Child Center for Outcomes Research and Delivery Science Biostatistics and Analytics Core, and secondary faculty member in the Department of Biostatistics and Informatics at the University of Colorado School of Medicine, noted that the current state of study design includes innovative trial designs and cluster randomization techniques, with all stages of maturity of research represented in the palliative care literature. She explained that with support from PCORI, researchers have been engaging patients and other partners

to learn what outcomes matter to them, and PCRC has both improved the ability to conduct multisite randomized and cluster-randomized trials and trained researchers on how to develop fully powered trials.

What remains challenging, explained Colborn, is establishing research cooperatives or collaboratives for facilitating multisite trials and reaching all relevant populations, including those that are underserved, rural, diverse demographic groups. Another challenge is that pragmatic trials are often unblinded, with significant dropout rates or deaths in the serious illness population. Colborn pointed out several new opportunities to address those challenges. For example, the proposed NIH-funded U54 Consortium for Palliative Care Research Across the Lifespan,¹⁸ discussed by Morrison early in the workshop, will hopefully lead to the creation of “more multisite trials and facilitate collaboration across different institutions,” observed Colborn. Efforts to improve trial designs to mitigate missing data and target outcomes important to partners are bearing fruit, as are initiatives to use publicly available data or data captured as part of routine care. “These are data used ubiquitously in other medical settings, but in serious illness care, we tend to rely on primary data collection,” explained Colborn.

Colborn noted that despite exciting methodological and statistical advances for handling missing data, statistical methods can accommodate only so much missing data. Researchers have begun using census data to incorporate social determinants of health in trial design and have—to a limited extent—been incorporating electronic health record (EHR) data in trials and using natural language processing (NLP) to extract useful information from the EHR. Colborn pointed out that researchers have also begun using causal mediation methods to evaluate mechanisms important for NIH. One exciting area, in Colborn’s view, involves jointly analyzing dyadic data from both care partners and patients.

Many cluster-controlled randomized studies are underpowered, especially when focusing on non-inferiority as an outcome, explained Colborn. She suggested that one improvement would be to incorporate Bayesian statistical methods¹⁹ that can provide inference without having to worry

___________________

¹⁸ For more information, see https://grants.nih.gov/grants/guide/notice-files/NOT-AG-23-050.html (accessed January 18, 2024).

¹⁹ “Bayesian analysis is a statistical analysis that answers research questions about unknown parameters of statistical models by using probability statements.” See https://www.stata.com/features/overview/bayesian-intro/ (accessed March 21, 2024).

about p-values. She noted that contemporary mediation techniques, such as coefficient methods, potential outcomes, and counterfactual frameworks, are underused in serious illness care research, although they are ubiquitous in other areas of medical research. In Colborn’s view, it will be important to rethink which outcomes to target on the causal pathway to reduce missing data and increase effect size and engage in more cross-disciplinary collaborations with computer scientists and biostatisticians who work regularly with data from EHRs and clinical notes.

Colborn emphasized the very few collaborative biostatisticians, such as herself, and few postdoctoral training programs in the field. The United States has strong biostatistics programs, but not every institution has a group of collaborative biostatisticians. A key problem, Colborn observed, is that universities are losing students to industry, which can promise higher pay, powerful computers, and large datasets. To address the challenge of developing more collaborative biostatisticians, the National Heart, Lung, and Blood Institute is funding summer institutes in biostatistics.²⁰ Colborn added that PCORI and NIH are funding data coordinating centers, but more such centers are needed, as are more opportunities to bring together larger groups of collaborative biostatisticians who can support large pragmatic trials. “There is value in having statisticians training and mentoring along with the people doing serious illness research,” she emphasized.

Colborn noted the importance of clearly defining what matters most and when and posed three key questions:

• Is it possible to reduce missing data by looking at outcomes earlier in the causal pathway?
• What important outcomes can an intervention affect before patients decline?
• How can investigators get connected to the biostatistics support they need?

She explained that researchers do primary data collection by asking patients to fill out surveys. Although it is important to not ask patients to complete surveys too many times, not surveying patients creates gaps in information.

___________________

²⁰ Additional information is available at https://www.nhlbi.nih.gov/grants-and-training/summer-institute-biostatistics (accessed March 4, 2024).

“We are blind to their outcomes that might be impacted by the intervention,” explained Colborn. She suggested that researchers could look at alternative outcomes, such as hospital-free days, location of death, and number of visits to the hospital. NLP, she explained, could mine unstructured EHR data (i.e., clinical notes) to determine if goals of care were collected from patients.

In closing, Colborn noted that pragmatic trials require many sites and that many hypotheses require large studies to have sufficient power. Other approaches, such as Bayesian methods, can yield meaningful results and inference. It is important, she stressed, to design studies to mitigate missing data and incorporate data collected through routine care. She recommended engaging interdisciplinary research teams to better integrate large, observational datasets and clinical notes and also stressed the importance of increasing opportunities for biostatisticians to train alongside serious illness scientists to become effective collaborators and mentors.

Panel Discussion

Session moderator Kavalieratos opened the discussion by asking Colborn for her thoughts on how to train the next generation of palliative care and serious illness scientists to account for the variety of available methodologies. He also asked about any historically underrepresented disciplines that need to be at the table to improve trials and analytical methods. Colborn noted an opportunity for not only statisticians but all researchers to train together, with clinical trial intensives being a good example. An opportunity might also arise to look at more pragmatic trial intensives and bring in psychometricians who work in psychological research settings. “It is valuable to get a group of people together in a room all from different disciplines and discuss how they would think and work through that problem,” said Colborn.

To build a pipeline of investigators, White suggested encouraging clinically oriented trainees to pursue advanced degrees and learn advanced methods. “If we are going to be doing rigorous science, people who have had 7 years of medical training and then go into research with zero methodologic expertise are not as well positioned as people who have advanced degrees and have put sustained attention toward the topic,” noted White. He also proposed ensuring that academic center programs cover the breadth of methodological expertise discussed at this workshop, including qualitative and Bayesian methods and pragmatic trials. He pointed out that NIH’s Clinical and Translational Science Awards do a good job at this.

White noted that many medical centers offer advanced degree programs in clinical research that would benefit palliative care researchers.

Odom pointed out that one discipline missing from clinical trials work is community-engaged research, which involves community members, patients, and families in conceptualizing and designing a study and interpreting and disseminating the results. “Having those partners with you the whole time adds an invaluable contribution,” said Odom. Jacobson recommended approaching the Abdul Latif Jameel Poverty Action Lab²¹ to find an economist for a project.

Kavalieratos asked panelists to comment on the opportunities and challenges to bringing nonacademic partners, such as for-profit home health agencies or hospice agencies, into a research team conducting high-quality research. Grudzen noted that it is important to bring those voices into the design and conduct of a pragmatic trial. “If we want to have a real impact, we have to work with those partners and look at things from their shoes,” she said. Jacobson said that the key to success is finding a person within an organization who will champion a project and has a commitment to understand the evidence. She recommended getting a partner organization’s up-front commitment to a project and clearly stating which data will be shared and what the expectations are.

White said that when bringing in people with other perspectives, it is important to consider their economic and value incentives when trying to convince them about the merits of a study or quality improvement project. NPCRC, for example, has done a good job of thinking about the value proposition for palliative care from various perspectives.

Kavalieratos asked panelists to identify one methodological priority that U54 grants should include. Grudzen replied that she recently asked junior faculty the same question, and they were interested in using serious illness care as a framework and layering on different additional components. For example, a psychiatrist might want to look at “additional ways in which psychiatry can be involved, and I think that makes a big tent,” she explained. Grudzen added that bringing biostatisticians, economists, and project managers into research projects and “in general, getting junior people excited about the field is the most important thing to do with U54.” White pointed to the importance of having a balanced portfolio of research across all areas, from NIH Stage 0 to real-world effectiveness testing and pragmatic trials.

___________________

²¹ Additional information is available at https://www.povertyactionlab.org/ (accessed January 29, 2024).

Odom pointed to the need to have a strong investigator development core that trains early-career investigators to conduct studies requiring a variety of different expertise that no one person could learn: “You cannot be one’s own trialist, methodologist, content expert, and economist all at once. This is team-based science, and it needs a community that grows young investigators, allows them to have collaborators, and provides them with the skills needed to conduct and manage complex trials at multiple trial sites.” Colborn commented on the need for more data coordinating centers to help engage rural communities and hospitals that do not have sufficient in-house biostatistics capacity.

Audience Q&A

Marie Bakitas, professor and associate dean for research and scholarship in the UAB School of Nursing and director of the UAB Center for Palliative and Supportive Care University-Wide Interdisciplinary Research Center, asked panelists how to get funding for nontraditional and quasi-experimental studies that incorporate learnings from program development and the patient voice. Odom recommended having people with expertise in the area on study sections and for researchers in this small but growing field to serve on study sections and review manuscripts. Colborn noted that what has helped her be a better collaborator was to be present at the beginning of the conversation about a proposed study and tell a succinct and convincing story about each part of it.

Neeraj Arora, associate director at PCORI, explained that PCORI funds real-world pragmatic trials and encourages, if not requires, qualitative data collection as part of the studies. However, investigators often do not integrate quantitative and qualitative data. He asked panelists for ideas about what PCORI can do to address this problem and maximize the impact of taxpayer-funded studies. White agreed that a schism exists between the quantitative and qualitative in studies, diminishing their overall quality. He suggested that one strategy to overcome this is to integrate the qualitative methodologists and ethnographers doing direct observation and have them go to sites that appear to not be deploying an intervention or where it appears to be ineffective or only partially effective. “That is a place that they could then target to try to understand positive and negative deviants,” said White. Grudzen added that many researchers, such as herself, do that integration but then cannot get their studies published. “We need a journal of mixed methods,” she said.

Sheria Robinson-Lane, assistant professor, Department of Systems, Populations and Leadership, University of Michigan School of Nursing, said that mixed methods are a critical mechanism for improving equity in research outcomes. An age cutoff can also lead to studies that miss important comparisons between populations with similar outcomes. Similarly, leaving out diverse populations, such as rural Indigenous people, creates inequitable research outcomes. However, even including minoritized populations does not guarantee equitable outcomes, because they will be swamped by the number of White participants. She asked panelists about approaches that do not split up participants and can provide an understanding of the issues and challenges of including minoritized groups in studies.

Jacobson noted that this is where she would turn to quasi-experimental research designs and leverage Medicare data. “When you have 40 to 60 million people, all of a sudden, the American Indian population is not that small,” said Jacobson. “You have to be creative about inferring causality there and what to learn from that, but I do think that is the virtue of secondary data analysis.” Grudzen suggested recruiting from emergency departments, which generates a more diverse population than one would get from a major academic center clinic. “Setting is important if we are doing primary data collection or enrolling patients,” said Grudzen. White added that NIA recently issued a commitment to inclusion with teeth to it by “prioritizing funding requests with proposed planned enrollment that are representative of the population affected by the disease, condition, or health experience and, number two, appropriately inclusive of racial and ethnic groups as well as those across the lifespan.”

Colborn encouraged people to think about looking at differences in proportions and consider clinically meaningful differences instead of p-values. “That will move us away from trying to collapse into groups that were powered to detect a difference,” she said. Bayesian approaches are one way of doing this, she added.

Rebecca Price, senior policy researcher at RAND Corporation, asked panelists to comment on how observational data can provide an understanding of what works best in real time and what some of the methodological considerations might be. Jacobson cited innovation on matching techniques. Colborn said that as a statistician, she gets excited when people talk about observational data because those datasets are huge, presenting complicated statistical problems. This is where it is important to have team science and people with experience working with these datasets.

However, such datasets have inherent underlying bias. “You have to be careful when working in observational data that you are actually able to answer the hypotheses that you are testing and that it will be generalizable to the population of interest,” Colborn cautioned.

Aslakson asked panelists how they get their qualitative studies published in high-impact journals. White said that it is not reasonable to expect top journals to publish a study unless the findings are truly remarkable, as would be true for a quantitative study. One thing he does to increase the odds of acceptance is to cite in the introduction that his paper reports on a Stage 0 study in the context of the NIH stage model. “There is some validation that comes when you cite the NIH, and NIH saying this is a core part of the scientific enterprise,” said White. A second step is to follow the correct framework for qualitative study reporting and cite that at the beginning of the methods section. This tells the reviewers that the study used an appropriate method. He noted, too, that some NIH institutes have done a good job issuing a request for application calling for Stage 0 and Stage 1 research. Odom added that noting a study falls into Stage 0 or 1 implies a Stage 3 and 4 that will move toward action and a solution.

Karen Bullock, the Louise McMahon Ahearn Endowed Professor in the Boston College School of Social Work, asked panelists how their robust, well-designed studies are not perpetuating frameworks of White supremacy. In particular, she worries about including older Black Americans because they lived when racially segregated health and hospital settings were the rule until 1965. “We have older Black Americans who have lived through a racially, systematically different health care system and who tend to be or have the highest rates of morbidity and mortality, and we continue to design our studies in the absence of attending to the need to close this gap,” said Bullock. She noted that when she looks systematically at the literature, she finds that clinical trial enrollment is consistently 80–85 percent White.

Grudzen replied that one thing about pragmatic trials is that they enroll many patients, and her team prespecifies exploratory outcomes looking at race and ethnicity to see if an intervention was effective in those groups, with deep, qualitative interviews as an important tool for doing so. For example, her research found that uptake of a nurse telephonic intervention was higher in minoritized populations. She noted that the population for that study was 30 percent Black, with recruitment conducted in emergency departments.

Bullock replied that “it is great we can say what is happening in emergency departments, but not all Black people are being seen in the emergency department.” Grudzen countered that almost everyone visits the emergency department in their lifetime, with 120 million visits a year, which in her view provides a broader population to draw from compared to a clinic where the patients need a certain type of insurance to get in.

Odom and White both noted that the key to developing an intervention that decentralizes Whiteness is to start by centering equity and using human-centered design that brings in people who are the worst off to help design the study. The assumption is that if an intervention can help those who are most in need, it should benefit everyone else. Bakitas added that it is time for researchers to stop making excuses regarding why they cannot recruit a more representative population and instead start engaging underrepresented communities.

Rodgers noted that much of this work is in the community at the bedside, and those investigators do not always have familiarity with and fluency in research design. They may, however, be using a quality improvement framework, and he wondered how the quasi-experimental, pragmatic study design integrates with the frameworks that many organizations have. Grudzen responded that moving palliative care or other serious illness research into such a framework can provide feedback on pragmatic trials or interventions that goes into the dashboards that doctors and nurses use routinely. She said that framing a study in terms of quality improvement rather than research exempts it from institutional review board (IRB) approval unless it generates identifiable data. Jacobson added that quality improvement projects at large institutions are well suited to randomization. As a final comment, Colborn pointed out that studies in the Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) and PRISM frameworks²² “do not always have to be about patient outcomes.” An outcome could be reaching the target population. “Maybe that is a pilot study, but we have to learn where we are going wrong with trying to recruit certain populations,” said Colborn. “That is a study in itself, so be creative in that area.”

___________________

²² For more information, see https://re-aim.org/learn/prism/ (accessed March 20, 2024).

IMPROVING EVIDENCE GENERATION FOR SERIOUS ILLNESS CARE RESEARCH: OUTCOME MEASURES AND DATA CAPTURE

Moderator Anna Gosline, senior director of strategic initiatives in the executive office of Blue Cross Blue Shield of Massachusetts and executive director of the Massachusetts Coalition for Serious Illness Care, explained in her opening remarks to the session focusing on outcome measures and data capture for evidence generation that choosing the health outcomes for an intervention is not just an academic exercise. “I know that what we choose to focus on in research will translate into what gets funded, what gets scaled, what gets adopted by a health plan, and what gets translated into quality measurements and incentives,” said Gosline. As a result, who gets to decide what outcomes matter and what outcomes to focus on is critically important.

Gaps and Opportunities for Patient- and Care-Partner-Reported Outcomes

Antonia Bennett, professor in the Department of Health Policy and Management and faculty director of the Patient-Reported Outcomes Core at the University of North Carolina at Chapel Hill, explained that although she would focus primarily on patient-reported outcomes, thinking about person- and care-partner-relevant outcomes is also possible. She stressed that although valid and reliable measurement tools are crucial for productive science, they are often lacking. Just as a trial does not generate useful information without fidelity to an intervention, not having quality measurement tools means not knowing whether it had any effect.

Bennett explained that many of the instruments available were developed in a prior era when the standards for patient-reported outcomes were not as rigorous and the measurement properties often poor or unknown. Through her work with the PCRC Group,²³ she had support to assemble an instrument library, and in cataloging even the high-quality instruments developed from systematic reviews, she found many gaps in their measurement properties. One characteristic often missing is responsiveness to change, which will provide information on whether an outcome measure

___________________

²³ For more information, see https://palliativecareresearch.org/ (accessed January 24, 2024).

can detect an actual effect of an intervention. Although this is one of the most important characteristics, investigators often do not validate it because that typically requires long-term follow-up at the end of a cross-sectional validation study, with no funding.

Another major shortcoming in the measurement realm, noted Bennett, is that many of the older instruments were developed in predominantly White, middle-class samples, so the content may not be relevant to many other groups in the United States. “The issue is that key content in the instruments may be lacking, and study participants may pick up on the fact that what they are being asked about does not resonate with their experience,” said Bennett.

These gaps occur because, too often, time or funding are inadequate to develop instruments fully, explained Bennett. Her impression is that research investment in instrument development over the past 10–15 years has been limited, following NIH’s substantial investment in Patient-Reported Outcomes Measurement Information System (PROMIS) instruments.²⁴ “Continuing to invest in targeted instrument development and thinking strategically about what particular fields would really benefit would be a good way to advance serious illness care research,” she observed.

Bennett noted that in her view, one of the interesting aspects of serious illness care research, especially at the end of life, is the number of domains and concepts beyond symptoms and physical function that become relevant. For example, spiritual concerns, issues with the quality of hospice care, and the degree of care partner distress arise. Bennett pointed out that these domains require conceptually specific and nuanced instruments, and she has been pleased to see the complexity and thought investigators have put into capturing interesting topics in their instruments.

Another area of concern, explained Bennett, is the financial hardship for most patients facing serious illness and their families. Understanding and addressing this concern is a rapidly growing area of research and policy interests across disease conditions, particularly for cancer and Alzheimer’s disease. She noted the reviews identifying gaps in the existing instruments for measuring financial hardship (Nanda and Banerjee, 2021; Patel et al., 2021; Salazar et al., 2023) and finds it interesting to see the recognition of and efforts to measure the impact that health care costs and the costs associated with caregiving have on both patients and families.

___________________

²⁴ For more information, see https://commonfund.nih.gov/promis/index (accessed January 18, 2024).

Bennett emphasized that data collection options are gradually improving. Despite significant challenges for collecting patient-reported outcome data, she sees two things converging to address them. The first is the increasingly improving technology for capturing information, such as EHR patient portals, text messaging, apps, and smartphones. The second is the increasing ability and willingness of people to provide data for clinical care and research. Bennett noted that U.S. adults ages 65+ who own smartphones have grown from 13 to 61 percent over the past 10 years (Faverio, 2022). That is a huge change, but smartphone use is highly correlated with socioeconomic characteristics, making it important to be mindful about who is able to participate in studies that depend on it. Although automated telephone surveys, an older method, work well when relatively short, even 83 percent of U.S. adults ages 50–64 now own a smartphone, suggesting that a very large majority of older adults will do so in another 10–15 years.

In summary, Bennett emphasized that informative research studies require high-quality, valid, and reliable outcome measures that can capture the unique experiences of all U.S. cultural and socioeconomic groups. “I see this as being a key barrier in conducting research that is inclusive of or that is specific to groups that have not been heard from in research historically,” explained Bennett. However, serious illness care research includes many topics relevant to end-of-life care for which such outcome measures do not exist, including financial hardship. Such measures are necessary for developing program and policy interventions tailored to the specific challenges people are facing. She added that the evolving landscape for data collection can provide new opportunities for collecting patient- and care-partner-reported data.

Gaps in Patient-Reported Outcomes in Serious Illness Care Research

Jennifer Mack, associate professor of pediatrics at Harvard Medical School and associate chief of the Division of Population Sciences and faculty vice president for the Office for Faculty Development, Professionalism, and Inclusion at Dana-Farber Cancer Institute, described the rapid growth in the availability and use of patient-reported outcomes to inform clinical care. “There is increasing recognition that we should allow patients to use their own voices to talk about their needs and experiences,” noted Mack. “That helps us make care better.”

Investigators have also been increasingly using patient reports to inform their research agendas and assess the efficacy of health care interventions. The state of the science, however, includes important gaps, such as that available patient-reported outcomes only cover select domains and are appropriate for research on select populations. Robust, well-validated patient-reported outcomes are available in several important domains, especially for symptoms, physical function, and quality of life.

One example of a relevant measurement system, explained Mack, is the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)²⁵ developed by the National Cancer Institute to evaluate symptomatic toxicities by self-reports by patients participating in cancer clinical trials (Kluetz et al., 2016). This measure covers discrete symptoms, such as frequency, severity, and interference, to facilitate symptom management and monitor toxicity. The PROMIS set of person-centered measures monitors symptoms and broader domains, such as function and quality of life. Mack noted that both PRO-CTCAE and PROMIS are available in versions suitable for use across the lifespan and include proxy reports for the youngest patients and pediatric self-reports at around age 7–8.

Mack said that although these measures are a good start, questions in the serious illness domain need answers:

• How do patients with serious illness come to understand and accept their conditions?
• How should we talk about these issues with patients?
• What are patient priorities for care and treatment?
• What do patients want to experience and achieve in life?
• How do priorities translate into decisions?
• How well do decisions reflect personal priorities, goals, and values?

“These are big questions that we have in the field, and we have very few tools to measure these things,” observed Mack.

Mack pointed out that the measures for patient preferences focus heavily on treatment. As a result, they do not cover the broader range of preferences, such as whom patients want to be with and where and how they want

___________________

²⁵ For more information, see https://healthcaredelivery.cancer.gov/pro-ctcae/ (accessed January 18, 2024).

to spend their time. Mack noted the lack of robust measures for patient goals, decision making, and concordance of care, and those measures that do exist are often not adapted culturally or available in multiple languages suitable for use with a wide range of populations. Moreover, they are often not tested for responsiveness to change, important information for assessing health care interventions, and not suitable for use across the lifespan. “That is a lot of holes for things that are important to the field,” observed Mack.

Addressing these gaps is important because a lack of robust, validated measures useful for understanding patient goals and values is a significant limitation to progress, said Mack, and creates challenges for developing interventions, attaining funding, testing intervention efficacy, and comparing across studies. It also limits how well the patient voice is heard in palliative care research, which, Mack noted, is “contrary to the reason most of us are in the field.”

It is also important to ensure research considers all voices, including children and adolescents; those with limited literacy or English proficiency; and those who require proxy reports because of their age, cognitive disability, or critical illness. Mack noted that proxies are not very good at accurately reporting what a patient is experiencing, illustrating the need for methods to optimize proxy reports.

If funding were not limited, Mack would have the field identify domains of key importance for developing measures and convene working groups to review and assess available measures in each domain. If appropriate measures were available, she would dedicate funds to refine and test them. If not, she would prioritize developing them as a fundamental need. Mack referred to Bennett’s presentation and underscored the importance of funding psychometric research and the need to incentivize including “quieter voices,” such as children, those with cognitive impairment, and people from a range of underrepresented cultural backgrounds.

Mack noted in closing that the field needs to ensure that patients and care partners are involved in every step of developing new measures, including prioritizing needed domains. The goal, emphasized Mack, is to have a set of robust measures that are recommended for use.

Spirituality in Serious Illness: Current Evidence and Future Priorities

Karen Steinhauser, professor in the Departments of Population Health Sciences and Medicine at Duke University and health scientist at the Durham Veterans Administration Health System, said that a question she heard early in her research was “are you at peace?” She explained that patients and families rank the construct of peace as equal in importance to freedom from pain.

Steinhauser described a multistep psychospiritual intervention she and her collaborators conducted in which they asked people to what extent they felt at peace and heard a range of answers. One person, for example, said that they were 100 percent at peace and not worried, noting that they had had a growth experience during their illness. Another person said that they were not at peace with their life because their functional status fundamentally changed their inner experience. A third person said that they left a light on at night because they were terrified in the dark. Steinhauser said that the answers illustrate several underlying issues that need addressing in serious illness care and offer some chances for guiding questions, such as how to assess and meet the varied spiritual needs of different populations and across the lifespan.

What is known, said Steinhauser, is that patients and families in acute health care situations have spiritual and religious needs, so it is important to identify them, the resources available to meet them, and ways of helping them mobilize those resources. It is important, too, to understand how those needs evolve over time and over the life course. These are fascinating issues, she said, but data are insufficient to examine them and develop an intervention, decide who should perform it, and determine the best way to implement it in serious illness care.

Steinhauser and her collaborators have been collecting data and now have good descriptive data that provide insights into how integral spirituality is to patients and families in the setting of serious illness. Even people and families who identify as neither spiritual nor religious have spiritual needs, making it necessary to think broadly about beliefs and practices that are central to coping. Negative spiritual coping is associated with poor outcomes, such as poor quality of life and depression, and positive measures of spirituality are associated with a higher quality of life. Beliefs influence decision making and treatment choices, and the costs of care are lower when spiritual care is addressed. Interventions addressing spirituality have tended to be associated with improved quality of life in seriously ill patients.

Steinhauser explained that people with serious illness frequently desire spiritual care as part of medical care, with estimates of 50–96 percent of patients wanting it. However, medical care for patients with serious illness addresses spiritual needs infrequently, with patient reports indicating that spiritual care from medical teams is 9–51 percent (Balboni et al., 2022).

These findings have come largely from descriptive work in Western settings with a largely Protestant, middle-class, White population. One of the key tasks for the field, emphasized Steinhauser, is to think about embracing population and cultural diversity and the growing population that considers themselves neither spiritual or religious across the range of questions that get

to the roles, beliefs, and practices around spirituality. She noted that spiritual beliefs are deeply embedded in U.S. culture and individuals’ lived experiences.

Steinhauser also noted the need to improve specificity of the questions. Many of the existing measures on spirituality and religion are a single item that do not provide much nuance about a highly nuanced domain. “I will say part of that is because we have not had studies funded that are predominantly about spirituality,” she said. Rather, studies have used measures of quality of life as a proxy for spirituality. Questions she would like to see addressed include the following:

• What is our conceptualization and definition of spirituality?
• What components of spirituality are of greatest interest?
• How are those components related to one another?
• How are they related to outcomes of interest?
• What are suitable outcomes?
• How might we intervene?
• Who should intervene?

Steinhauser shared a consensus definition of spirituality developed by an international group of palliative care clinicians: “a dynamic and intrinsic aspect of humanity through which persons seek ultimate meaning, purpose, and transcendence, and experience relationship to self, family, others, community, society, nature, and the significant or sacred” (Puchalski et al., 2009). That is a good definition for a clinical setting when wanting to set up a big tent, said Steinhauser, but it creates challenges when deciding how to operationalize it in research, making it important that investigators provide their working definition when beginning a study.

Steinhauser argued that a common taxonomy of spirituality is needed (see Figure 11). She noted that some good measures for positive and negative coping exist, along with specific measures related to values and beliefs and how they influence decision making and that can identify important spiritual practices. She said that work remains to refine the design of studies and measurements, particularly regarding the need to specify the domains of spirituality a study will examine and choose a measure that taps into them. More work is also needed on identifying a common core set of measures and creating a working group to establish it. She pointed out that the few available interventions do not do a good job of specifying particular components. Steinhauser noted an effort to develop a common taxonomy on what chaplains do, for example, and which meaning-centered interventions, such as in psychotherapy and dignity therapy, are the best specified in the literature. The field also needs well-specified trials that test various delivery modes and comparative effectiveness trials.

The benefits of focusing on meaning rather than religion and spirituality, explained Steinhauser, include being inclusive, because everyone has a meaning system and participates in meaning making, which contributes to who people are, whether or not they define themselves as spiritual or religious. Other benefits are that it demystifies beliefs, is value based, empowers patients, addresses spirituality as fundamentally relational, and provides a model linking to physical symptoms. “There is a lot of opportunity to look at the mechanisms within spiritual practice and beliefs, and how those are linked in making physiologic changes within our patients,” explained Steinhauser.

She noted the need to understand the timing of an intervention and conduct longitudinal studies of spirituality. Other important questions include the following:

• How do spiritual needs evolve over the course of serious illness and the life course?
• Is spirituality the same in different illness trajectories?
• When are specific interventions best delivered?

Although many studies include all comers to a clinic, it is necessary to be more refined and screen for spiritual distress and needs. Just as every patient with cancer does not get morphine, the field needs to be more specific in determining who will receive an intervention and when to deliver it. It is also important, said Steinhauser, to assess the strengths of each patient. She also believes that some ways to deliver an intervention may add value by saving time. For example, a measure she and her colleagues created has shown that high needs correlate with symptomatic levels of depression and anxiety, which the medical team found useful for contextualizing goals-of-care conversations.

In closing, Steinhauser listed a set of guiding questions for future investigations:

• How do we best identify needs and resources in a manner that is specific and inclusive?
• How do those needs and beliefs impact care decisions?
• How do we intervene?
• When is the best time to intervene?
• Who intervenes?
• How do we best implement intervention in the stream of care?

Quantifying Language for Serious Illness Care Research with NLP

Charlotta Lindvall, assistant professor of medicine and director of clinical informatics at Dana-Farber Cancer Institute’s Department of Informatics and Analytics, pointed out that so much of serious illness care happens in conversations with patients and their families. The challenge, she explained, is that data are not collected at the point of care, so conversations about goals of care, symptom assessment, and care partner distress are often not captured by the clinician until hours or even days later. The clinician then must decide what to document, interpret, and process about that conversation in the EHR.

Having different types of data in the EHR creates another challenge, explained Lindvall. In cardiology, for example, discrete data are captured at the point of care, such as blood pressure and heart rate. In serious illness care, the data on clarifying goals of care and assessing care partner distress are in unstructured clinical notes. Lindvall explained that at Dana-Farber, physicians are supposed to document these conversations in structured tabs in the EHR, but it has been difficult to change clinician behavior. “If we only analyze the structured tab in our EHR, we miss a lot of documentation around goals of care and symptoms,” cautioned Lindvall.

What is exciting about NLP, noted Lindvall, is that it combines computational linguistics with rule-based human language modeling, statistics, and machine learning to allow computers to process and manipulate either spoken or written natural language. It enables capturing text-based data from dozens to thousands of charts a day. Although fully automated NLP software can analyze upward of 5,000 charts a day compared to 50 with an NLP system that keeps a human in the loop, the latter is more pragmatic, has better security, and has human confirmation of the analysis. Lindvall noted that the hybrid system she and her colleagues developed, Clinical Regex, can analyze very large datasets and enable subgroup analysis to see if an intervention works or did not work in different subgroups.

Lindvall and her collaborators conducted a study using Clinical Regex²⁶ to analyze millions of EHR notes from 42,019 older adult patients in 22 clinics and document goals-of-care conversations (Volandes et al., 2022). It completed this analysis in 1 year, said Lindvall. “We were able to see that the specific intervention worked better among Black patients and Hispanic patients compared to White patients,” she said. Lindvall foresees a huge opportunity to combine EHR notes with ambient audio recordings of clinic visits and capture outcomes in real time.

Lindvall is particularly interested in using NLP to capture symptoms from patient-recorded outcomes, and she and her colleagues have conducted a pilot study that initially used a dataset of 578 transcribed excerpts from 50 synthetic clinical conversations in which medical students pretended to be patients (students were used to ameliorate security concerns that have since been addressed) (Fareez et al., 2022). To drive the analysis, the investigators prompted two well-established NLP models—Open AI’s GPT-3.5-turbo and GPT-4²⁷—with the question, “Are any medical symptoms mentioned in the transcript?”

The gold standard results for comparison were those for which clinicians manually coded each excerpt for symptoms as measured using the Patient-Reported Outcomes Version of the Common Technology Criteria for Adverse Events (PRO-CTCAE²⁸ framework. Lindvall explained that GPT-4 identified 442 symptoms with 0.98 sensitivity and 0.71 specificity, performing better than the earlier GPT-3.5-turbo model. She pointed

___________________

²⁶ For more information, see https://lindvalllab.dana-farber.org/software.html (accessed February 22, 2024).

²⁷ For more information, see https://openai.com/ (accessed February 21, 2024).

²⁸ For more information, see https://healthcaredelivery.cancer.gov/pro-ctcae/ (accessed February 21, 2024).

out the importance of the specific way in which she and her collaborators wrote the prompt, which took some time. Methods exist to use prompt engineering to constrain the models and reduce the risk of their creating false information. Lindvall’s team is using NLP to identify text-based communication outcomes in five clinical trials (Eneanya et al., 2022; Lakin et al., 2020; Volandes et al., 2022).

Lindvall would like to disseminate free or low-cost NLP tools to the wider research community. With artificial intelligence (AI), she added, it is important to establish secure computing platforms and understand the challenges with models making up data. She would also like to develop the expertise to provide a service to people in the serious illness care research community so it can leverage these models for outcomes such as goals-of-care conversations and then follow up on patient-reported outcomes. Her vision is that if NLP models can find symptoms not reported through the latter, clinicians could prompt the patient or family for targeted outcomes.

A Pragmatic Trials Perspective of Outcome Measures for Persons with Serious Illness

Vincent Mor, the Florence Pirce Grant University professor of health services, policy, and practice at the Brown University School of Public Health, said that a pragmatic trial often requires working with existing data, particularly for persons living with dementia and cognitive impairments who may not be able to respond to psychometrically appropriate measures. He noted that care partners can be the object of an intervention and not just a potential source of proxy information about the patient’s experiences.

Mor explained that although researchers maintain control in Phase 3 efficacy studies, pragmatic trials seeking to replicate these results are embedded in functioning health care systems, where system staff implement the interventions. Phase 3 studies collect primary patient outcomes using research staff; pragmatic trials rely on medical and administrative data to the extent possible to ascertain outcomes.

Collecting patient-reported or patient-relevant outcomes is complex and costly in pragmatic trials, said Mor. It is often in conflict with the pragmatic orientation of a trial that a health care system needs to be willing to adopt without interfering with the daily provision of care. Collecting primary data directly from patients or care partners requires consent, adding an additional impediment that makes a trial less pragmatic and not as readily implemented within the context of a functioning health care system.

Research data collectors are often required in the clinic unless the trial is run in an academic medical center that has data collectors in place as part of cancer care or a large Phase 3 clinical trial infrastructure. “Unless they are already there, it is going to be difficult for these people because they get in the way of the routine provision of care,” said Mor. Because pragmatic trials are embedded in routine care experiences, patients and care partners may not be able to respond to questions at the scheduled follow-up, leaving holes in the data. Recruiting control subjects is also a challenge because in a pragmatic trial, providers often do not know who is participating as a member of the control group, and they and their patients do not even know they are involved in a study.

Mor pointed out that health system leaders may host an intervention study only if it might produce relevant outcomes that solve problems relating to meeting patient needs. Even then, they may choose not to do so if it will disrupt the existing workflow. Mor noted that looking at the issue of patient-reported outcomes from the perspective of a health care system’s leader, interventions should address standard quality performance measures that insurers, payers, or the Centers for Medicare & Medicaid Services (CMS) use to rate the system or that might increase health system leaders’ bonuses. Health care systems may be willing to increase sampling of patients in existing satisfaction surveys, but incorporating an entirely new set of surveys to achieve that goal will be more complicated and costly.

As a result of these concerns, compromise is essential, noted Mor. One solution is a short, standard self-survey, but that often excludes cognitively impaired or visually impaired patients or those who are too sick to respond. Even with satisfaction surveys, which are standard in most health care systems, response rates tend to be low and biased in a manner that leaves out the sickest patients who are non-responsive. Another challenge is incorporating and dealing statistically with the problem of getting a proxy measure under those circumstances.

Researchers can conduct posttest interviews, explained Mor, but that can rule out a cluster-randomized design because it would require interviews with passively recruited controls who may have no idea they are part of a study. Providing control participants with an explanation would introduce a twist that the experimental subjects do not experience. One solution is to use Internet logins to obtain narrative content or conduct short-form surveys. Mor explained that although this approach seems to perform well, it produces lower and more biased response rates that exclude digitally naive patients and care partners and those who are cognitively or visually impaired, if the survey is not set up properly.

Mor described his NIA grant to fund pilots and demonstrations of nonpharmacologic interventions focused on patients living with dementia and their care partners. The key focus of these pilots and demonstrations is on pragmatically ascertaining outcomes, including how to use the EHR to identify appropriate subjects using EHR or claims data as an outcome measure. They have funded more than 20 pilots and six demonstration projects with interventions being implemented in primary care emergency departments, hospitals, nursing homes, assisted living facilities, and home care. These pilots and demonstrations are using pragmatic outcomes, some of which have tried to address the issue of how to get a patient-relevant and patient-reported outcome from care partners or patients in the context of an embedded pragmatic trial. Mor explained that the pilots and trials are providing important lessons on how to replicate efficacious interventions in a functioning health care system.

The experience to date, explained Mor, is that it is hard to pragmatically identify care partners, particularly using the EHR. Complex IRB and data safety monitoring board issues are associated with sending materials to patients yet expecting care partners to respond and determining the proper subject of the study. For less pragmatic Phase 4 studies, he and his collaborators have identified common and usable patient-reported outcomes, and some using administrative data on patient satisfaction, for example.

Mor emphasized the challenges involved in pragmatic trials, including the difficulty of getting measured outcomes that are most relevant to patients and their care partners. Noting work underway exploring EHR use in different settings, he stressed that using the EHR as a data source is particularly challenging for people with dementia and underscored the importance of ensuring that the quality and completeness of the EHR in facilities that are exposed to the study’s intervention need to be similar to those in facilities that are not.

Panel Discussion

Gosline asked panelists for their thoughts on prioritizing the measure domains that are most important to patients and families, particularly to those who have been most marginalized and excluded and are likely having the most challenging experiences. Steinhauser replied that the tools exist to develop measures for priority domains, but it has been difficult to get measurement studies funded. Bennett suggested thinking broadly about how to conceptualize the value of serious illness care and the things that are most important to key partners, patients, families, and health care systems. From there, it would be possible to identify those domains for

which interventions are available and any gaps in measurement. The next step would be to decide on the specific type of study; for example, will it be a pragmatic trial and leverage all the information in EHRs using NLP to gather data or an intervention study with a smaller sample?

Gosline asked panelists about the structural processes that could be put in place to ensure that patients and families are at the center of research efforts. Mack said to start with qualitative work to understand what the fundamental domains are for different groups. For example, she and her colleagues tried to develop quality measures for adolescents and young adults with advanced cancer. One finding from patient and family interviews was that people held different views about what the best care looked like. “We have to have room for these more preference-based measures or value-driven measures,” said Mack. These measures should reflect the patient’s or family’s satisfaction with the care they wanted to receive, not whether they were satisfied with the health system’s conception of quality care. Conceptualizing measures in this way may lend itself to account for different preferences among different populations with different cultural backgrounds.

Bennett remarked that the NIA Imbedded Pragmatic Alzheimer’s Disease and Alzheimer’s Disease-Related Dementias Clinical Trials (IMPACT) Collaboratory²⁹ convened a standing lived experience panel with which investigators can meet and get advice on subjects such as whether different types of consent are acceptable in pragmatic trials. She and her colleagues have had conversations with this panel to get the members’ perspectives on the value of reducing hospitalizations as an outcome. Establishing opportunities such as this to have a standing mechanism for getting input is valuable, she emphasized.

Mor agreed, adding that conversations with real people reveal nuances that are not always clear when collecting data from EHRs or claims. For pragmatic trials, this raises the fundamental question of whether the outcomes that a health care system might be interested in maximizing or minimizing, such as rehospitalization, are relevant to the individuals who have avoided hospitalization and want to know what an appropriate hospitalization is for them. He noted that the Hospital Consumer Assessment of Healthcare Providers and Systems³⁰ (HCAHPS) survey questions do get at individuals’ experiences, though Gosline wondered if it would be possible to add more questions about relational and communication issues. Mack added that HCAHPS results are challenging for health care systems because

___________________

²⁹ For more information, see https://impactcollaboratory.org/ (accessed January 18, 2024).

³⁰ For more information, see https://www.hcahpsonline.org/ (accessed January 18, 2024).

the results are not nuanced enough to identify the actionable items that would improve care for populations based on HCAHPS.

Mor shared that HCAHPS measures frustrate him because they underrepresent the sickest patients that most interest him. As a result, some of the actions that CMS takes are biased toward the dominant, less sick population. “It is critical for us to realize that what is necessarily going to help in making a health system more responsive to the average patient is not necessarily the best solution for a patient with advanced cancer, dementia, or end-stage congestive heart failure,” emphasized Mor. Gosline added that these measures are being asked to speak to many different purposes, so it may not be possible to harmonize them across the arc of basic research to implementation and practice or use them to determine financial incentives.

Gosline asked whether NLP can discern and identify the different ways that patients and families use language compared to clinicians and health care systems. Lindvall noted that a challenge with mining EHR data is that the clinician decides what to document from an encounter, which is why collecting audio recordings is important for providing a better understanding of the language used in a patient encounter. For example, it is useful to know whether the clinician or patient mentioned a particular topic, which audio recordings can provide. “You can do a different level of analysis when it comes to capturing audio recordings,” explained Lindvall. She pointed out that using NLP to mine the EHR works well for process-based quality measures.

Audience Q&A

Scott Halpern, the John M. Eisenberg Professor of Medicine, Epidemiology, and Medical Ethics and Health Policy and founding director of the Palliative and Advanced Illness Research Center at the University of Pennsylvania Perelman School of Medicine, said that in his view, researchers want patient-reported outcomes to serve many purposes and be completely, practically, and equitably measured. For Halpern, this rules out NLP, at least until new methods are developed. Halpern asked about tradeoffs in terms of which attributes patient-reported outcomes are expected to address, and whether the field should continue prioritizing patient-reported outcomes. Bennett replied that when she gives investigators advice on whether they should use patient-reported outcomes in their studies and how to design data collection, she first asks about the project budget and whether it can support multiple modes of data collection. Mack said that this may be a matter of prioritizing which domains are the most important, which might

require patient reports because they are about the patient’s experience. With the priority domains set, the next step is to have a robust measure that is valid in many populations and responds to change. For domains that are less important, she would look for other measures. Mor noted the different sets of tradeoffs for a pragmatic trial regarding the desired outcomes at the systems level versus the patient level.

Lindvall noted the importance of understanding what data are missing, particularly regarding patient-reported symptoms. The concern is that an intervention will improve care for the population that is able to report symptoms and exacerbate disparities for those who do not or cannot. Her approach is to use NLP to track patients who do not report their symptoms to see if that is because they do not have any. If the EHR does mention symptoms, she proactively prompts those patients to report their symptoms.

An online participant asked the panel to comment on the challenges and strategies for improving evidence generation from older datasets to avoid perpetuating implicit biases, incomplete information, and insufficient priorities. Bennett replied that it is essential to be cognizant of the strengths and limitations of a dataset, particularly when doing secondary data analysis. “You need to look carefully at who is eligible for the study, who participated in the study, what were the questions, and were they relevant to these priority populations that you have in mind,” she said.

Joseph Rotella, chief medical officer for the American Academy of Hospice and Palliative Medicine, reflected upon Steinhauser’s presentation and shared his realization that the physical and spiritual domains are no longer separate. For example, he might want to diagnose a spiritual problem arising from a patient asking for medical aid in dying or palliative sedation and test an intervention that uses psilocybin with medical supervision. Although this is a medical treatment, the patient might describe their experience in spiritual terms. He also noted that not everyone wants peace—some people may place a higher value on raging against dying, for example. To him, these examples point to how difficult it is to measure spiritual outcomes with respect to what matters to the patient.

Robinson-Lane said that one significant challenge regarding equity in pragmatic trials is that the nation’s institutions are still very much segregated in terms of available resources, and institutions in underserved and under resourced communities need to collect pragmatic data. She explained that the systems in place to ensure good science automatically exclude particular groups. She asked panelists about ways to address this challenge that allow for resource sharing or combining quality improvement efforts with a pragmatic trial. Mor replied that one of his colleagues has put together advice on how to address health equity issues when

designing a pragmatic trial by carefully considering site selection, patient selection, and measurement. “When you properly design a study and think about issues of equity and balance from the outset, you can do cluster-randomized trials where you reach out and make sure you have adequate representation of facilities that have a high concentration of minority populations,” explained Mor. “You are then looking at hundreds or dozens and dozens of facilities per arm as opposed to just one or two, which is critical for those kinds of studies.” Mor noted that studies often exclude those facilities that are least capable of implementing an intervention, which creates a problem when trying to understand how an intervention can be disseminated and how its influence might be different as a function of the inherent resource capacity of those facilities.

Paul Galchutt, research chaplain at the University of Minnesota Medical Center, asked about a role for the growing body of palliative chaplain researchers in designing studies and collecting data. Steinhauser said that role exists because an important question to answer for clinical health chaplaincy regards how beliefs affect decision making.

Anne Walling, director of palliative care research in the Department of Medicine at the University of California, Los Angeles, and the Veterans Administration Greater Los Angeles Healthcare System, asked if it was possible to establish a central hub for sharing innovations, including code, in developing validated EHR phenotypes and clinical quality dashboards that have both process measures and patient-reported outcomes needed for quality improvement and enable multisite pragmatic clinical trials. Lindvall replied that such a hub would be amazing.

INTEGRATING HEALTH EQUITY INTO SERIOUS ILLNESS CARE RESEARCH

Session co-moderator Carine Davila, palliative care physician and equity director of the Division of Palliative Care and Geriatric Medicine at Massachusetts General Hospital and clinical instructor at Harvard Medical School, opened the next session by explaining that the planning committee was very intentional about ensuring adequate attention was paid to the importance of integrating health equity into serious illness care research. Not only were speakers in other sessions asked to weave health equity throughout their presentations, but an entire session was developed to focus on integration. She noted that speakers would focus on several minoritized populations that represent only a small subset of the many perspectives that need to be included in research.

Social Work Competencies for Advancing Health Equity Research

Karen Bullock, Louise McMahon Ahearn Endowed Professor in the Boston College School of Social Work, emphasized the importance of using an equity-focused approach to research in the design, implementation, and evaluation phases. “Otherwise, we are at risk of perpetuating disparities,” she cautioned. She said that it is critical to look at serious illness care research and “consider the degree to which studies have been inclusive of populations that have been historically excluded and marginalized.” Bullock noted that the health equity lens she uses in her research focuses primarily on those individuals who did not have a legal right to access health care in health care systems and hospitals, notably Black people who were not allowed to receive care in the same settings as White people. “We need to be thinking about effective ways that we can intervene and close some of the gaps that are still experienced by older Black Americans,” emphasized Bullock.

Bullock cited a systematic review of a decade of studying the drivers of disparities in end-of-life care for Black Americans, which concluded that the sociocultural environment, physical and structural environment, and behavioral and biological domains remain understudied areas of potential causal mechanisms for racial disparities in care (Chuang et al., 2022). Bullock noted that the review concluded that studies focusing on individual factors should be better screened to ensure they are of high quality and avoid stigmatizing Black communities.

Bullock cited another systematic review that evaluated the racial and ethnic diversity of study participants and how hospice and palliative medicine studies reported race and ethnicity data (Rhodes et al., 2022). The review identified 78 unique racial and ethnic group labels in the literature, with over 85 percent of the studies including at least one nonstandard label compared to Office of Management and Budget racial and ethnic designations. She emphasized that one-quarter of the studies did not explain how the investigators collected race and ethnicity data, and 83 percent lacked a rationale for their collection methods. Over half did not include race or ethnicity in their analyses. For the most part, Bullock explained, 80+ percent of the populations sampled in these studies was White. “I think it is very important for us to say out loud that theses samples are not diverse,” said Bullock.

A scoping review on the experience of Black patients with serious illness concluded that “more articles focused on establishing evidence of disparities between Black and White patients than on understanding their root causes. Further investigation is warranted to understand how factors at the patient, provider, health system, and society levels interact to remediate disparities” (Heitner et al., 2023). Bullock asked participants to think about what it

means to them as researchers and how they see their own work as following the review’s recommendation.

One way to engage in research to better understand how to address disparities is to think about incorporating social workers into research teams, said Bullock. She explained that she felt comfortable making that recommendation because social workers play an important role in clinical care and have a skill set that is transferable to research. On research teams, social workers can contribute in ways that advance health equity as part of implementing an intervention. As a discipline, social work focuses on awareness, skills, and knowledge rooted in the profession’s code of ethics, which states explicitly that social workers promote diversity, equity, inclusion, and antiracist practices. Bullock explained that being antiracist means being a change agent and acting to dismantle structural and systemic racism rather than merely acknowledging that racism or structural barriers exist.

In terms of resources, Bullock called attention to the Social Work Hospice and Palliative Care Network,³¹ explaining that it is focused specifically on providing the best and latest information, professional resources, policy updates, and new and emerging education and research best practices. She added that the Journal of Social Work in End-of-Life and Palliative Care³² publishes original research and conceptual articles exploring issues crucial to individuals with serious, life-threatening, and life-limiting illness and their families across the lifespan. The journal’s focus includes interprofessional practice and research, practice and policy innovations, practice evaluation, end-of-life communication and decision making, pain management and palliative care, and grief and bereavement.

Bullock pointed out that the Advanced Palliative and Hospice Social Worker Certification (APHSW-C) is the first competency/test-based certification for social workers in hospice and palliative care. APHSW-C recognizes undergraduate- and graduate-level social workers with experience, specialized skills, and competency in hospice and palliative social work, with recertification required every 4 years. She emphasized that social workers are credentialed, licensed, and equipped with a skill set that can contribute to serious illness and palliative care. She noted that the field’s practice standards focus on cultural competence, empowerment, and advocacy, which are essential in serious illness care. Social workers are trained and educated to foster relationships between individuals and broader systems of care and with communities at large, recognize racism as a social determinant of health, and aim to eliminate structural and systemic barriers to equitable serious illness care.

___________________

³¹ See https://www.swhpn.org/ (accessed January 5, 2024).

³² See https://www.tandfonline.com/toc/wswe20/current (accessed January 5, 2024).

Bullock explained that when a research team is interested in engaging populations they might find difficult to reach, specific strategies align with the social work code of ethics regarding engagement and addressing structural and systemic racism (Brown et al., 2023; Fischer et al., 2023). “We look at it from a practice perspective but also from a data collection and dissemination perspective,” said Bullock, who noted the number of studies that have incorporated social workers (Barrett et al., 2022; Bullock et al., 2022; Bullock and Makaroun, 2022).

Bullock concluded her comments by quoting Kimberly S. Johnson, director of the Duke Center for Research to Advance Healthcare (REACH) Equity:³³ “What is most exciting about REACH Equity is its goal to move from describing racial and ethnic disparities in health to developing actionable ways for health care providers and health systems to improve patient care and reduce these disparities.” Social workers, said Bullock, can help achieve those goals.

Unmuting Rural Voices:
Toward Health Equity in Palliative Care

Marie Bakitas, professor and associate dean for research and scholarship in the UAB School of Nursing and director of the UAB Center for Palliative and Supportive Care University-Wide Interdisciplinary Research Center, said she has learned two things in her 40 years of caring for people in rural areas: (1) palliative care has a limited philosophical or pragmatic penetration in rural areas, rural areas have a limited health care and palliative care workforce, and rural areas have a limited number of health systems supportive of palliative care; and (2) every rural community is different. “Hence, [there is a] need to unmute the voices of people in these rural communities,” said Bakitas. She noted that 14–20 percent of the U.S. population is rural, and nearly half of the rural population lives in the South. Referring to an infographic from the National Institute for Health Care Management Foundation,³⁴ she noted that rural America has an older, sicker, and poorer population. Associations exist among rurality, persistent poverty, and poor health outcomes, as well as medical mistrust resulting from social injustice, racism, experimentation, broken treaties, “hit-and-run” research, and one-size-fits-all interventions (Bakitas et al., 2015, 2020; Marshall et al., 2023).

___________________

³³ See https://reachequitycenter.duke.edu/ (accessed January 5, 2024).

³⁴ For more information, see https://nihcm.org/publications/rural-health-addressing-barriers-to-care (accessed March 24, 2024).

These factors create what Bakitas called a “perfect storm” that has led to inequities in rural palliative care.

Bakitas explained that 96 percent of the land mass of Alabama is rural, and more than 50 percent of hospitals are in rural areas. The state has significant areas of persistent poverty, which she explained as having poverty rates of 20 percent or greater for 30 years, where the likelihood of change without intervention is zero. Bakitas pointed out that in Alabama, rurality and persistent poverty overlap with areas of poor health outcomes. She added that health care resources and specialty resources are limited in the rural areas, with nine rural hospitals closing since 2000 and nine counties having no hospitals. Bakitas recounted that a rural hospital chief executive officer (CEO) observed during an interview conducted by her doctoral student that more than half of all rural hospitals in Alabama were at risk of closure in the coming year (Beasley, 2023).

Citing state-level data compiled by CAPC, Bakitas noted that 50 percent of public hospitals in the state have palliative care programs. She pointed out that although 62 percent of not-for-profit hospitals in Alabama have such a program, only 4 percent of for-profit hospitals in the state do.³⁵ Referencing a conversation with the CEO of a for-profit hospital, Bakitas noted that “cost avoidance, which is the often-quoted savings that palliative care has for programs,” is not valued by for-profit hospitals. “This has major implications for how we design and market palliative care services and programs,” she emphasized.

When developing palliative care programs in rural areas, Bakitas noted, it is important to keep in mind that one of the definitions of palliative care is to improve the quality of life for the patient and family with care that is provided by a specially trained team of doctors, nurses, and other specialists who work together with a patient’s other doctors to provide an extra layer of support.³⁶ However, this raises the question of how to provide palliative care in rural areas with no foundational layer of primary health care. “This is one of the most critical issues that we face because we generally were not designed to be a primary care service and, therefore, we are not really good at it,” explained Bakitas.

___________________

³⁵ For more information, see https://reportcard.capc.org/state/alabama/ (accessed February 23, 2024).

³⁶ See https://www.capc.org/about/palliative-care (accessed February 23, 2024).

Bakitas summarized several lessons learned from the research conducted in rural areas since the late 1990s (Bakitas et al., 2009; Dionne-Odom et al., 2014; Zubkoff et al., 2021):

• “If you build it, they will come,” except for any number of reasons; as people in rural areas may not know what palliative care is, they may not identify any reason to drive a long distance or do anything to go out of their way to come to a palliative care clinic.
• Limited connectivity makes accessing services, such as telehealth, that require broadband or cellular capabilities difficult.
• Rural areas have limited or no palliative care workforce, which requires specialty-trained nurse telephone coaches, and more recently lay advisors, to provide teleconsultations.

In rural areas, Bakitas explained, a hub-and-spoke model makes the most sense, as it enables centralized expertise to be disseminated while simultaneously embedding expertise in rural communities. She emphasized that merely providing broadband access to rural areas will not magically enable everyone to receive palliative care (Bakitas et al., 2020). The reality, she said, is that there is often a culture clash between palliative care and local culture, which she addresses by developing community partnerships. Another barrier to providing palliative care in rural areas is the inattention to religion and spirituality, which can be addressed by partnering with local spiritual leaders for guidance on how to leverage religion and spirituality to make palliative care culturally responsive. A third barrier is the powerful influence that local “healers” have on advanced illness care, so the solution is to partner with them to promote local palliative care resources rather than transferring patients to distant centers.

The research Bakitas and her colleagues conducted in rural palliative care has focused largely on technology-enabled solutions and community–academic partnerships (Huff, 2019). Technology-enabled solutions include telehealth, eHealth, mobile health, video consultations, and remote monitoring. Community–academic partnerships provide home visits, train community health workers, teach primary palliative care skills through the End-of-Life Nursing Education Consortium, and form palliative care networks and community advisory groups (CAGs).

Bakitas discussed a study she and her colleagues conducted in which they tailored palliative care to rural Southern Black and White communities (Elk et al., 2020). They used the results of that feasibility study to conduct a

randomized controlled trial of a culturally based palliative care tele-consult at three rural hospitals in Mississippi, Alabama, and South Carolina that did not have palliative care services available (Watts et al., 2020). A critical piece of this study was the help the project’s CAG provided in tailoring informational materials and recruitment procedures (Gazaway et al., 2023a). Bakitas shared a brief video developed by CAG members who, due to pandemic-related concerns, could not accompany research coordinators to introduce the study. In the video, a racially concordant CAG member introduced the study to potential study participants. Bakitas asked, “Who wouldn’t sign up for a study that was introduced by your community member?”

Bakitas emphasized that palliative care inequities in rural areas result from a “perfect storm” of older, sicker, and lower-income populations combined with medical mistrust and limited health care and palliative care resources. She suggested that one way to improve equity is research that engages with and hears the unique voices of rural community members, recognizes the limited or absent health care foundations, and is creative and innovative in developing the palliative care workforce and resources at a community level.

Integrating Health Equity into Serious Illness Care Research for the Lesbian, Gay, Bisexual, Transgender, and Questioning/Queer (LGBTQ) Community

Carey Candrian, associate professor in the Division of General Internal Medicine at the University of Colorado School of Medicine, pointed out that dealing with serious illness is difficult, but it is even harder for LGBTQ adults because they have three major disadvantages. The first is financial, with one-third of them living at or below the poverty line. The second is social support, with many of them having thin networks. They are less likely to be married or have children and more likely to be estranged from their biological family, so they have less help with support, care, and advocacy.³⁷ The latter, she noted, is critically important in serious illness.

___________________

³⁷ See https://www.sageusa.org/wp-content/uploads/2021/03/sage-lgbt-aging-facts-final.pdf (accessed March 27, 2024); https://www.hrc.org/resources/understanding-poverty-in-the-lgbtq-community (accessed March 27, 2024); https://www.kff.org/report-section/lgbt-peoples-health-and-experiences-accessing-care-report/ (accessed March 27, 2024).

Underlying these two is the third disadvantage, a lifetime of stigma and discrimination, which can reduce life expectancy by 12 years (Miller, 2020). Stigma and discrimination have concrete health effects, said Candrian, with research showing that LGBTQ people have higher rates of anxiety and depression, substance abuse, cardiovascular disease, lung and breast cancer, and suicide (Caceres et al., 2017; Kaniuka et al., 2019; Moagi et al., 2021; Quinn et al., 2015). As context for why stigma and discrimination have such an impact, Candrian pointed out that older LGBTQ adults grew up when being LGBTQ was dangerous and even illegal in most states. She noted that the American Psychiatric Association listed homosexuality as a mental illness until 1973, and the World Health Organization kept that classification until 1990.

“Just imagine growing up thinking that there is something this wrong with you, that you are literally sick because you are LGBTQ, and how vulnerable you would be and how much you would internalize this stigma,” said Candrian. Even today, although the laws have changed, and homosexuality is no longer considered a mental illness, the stigma has remained and continues to affect the way LGBTQ people move through the world. This is why many have honed what she calls a “habit of silence” about a core part of themselves. Silence is a critical defense strategy that exists in all aspects of their lives but most importantly in medicine. Candrian referenced an AARP survey³⁸ in which nearly half of LGBTQ adults 45 and older reported that the doctors and those who care for them do not know they are LGBTQ because they are afraid that would impact the quality of the care they receive. “They are not crazy for worrying,” she added.

Candrian noted that according to a statement by the American Heart Association,³⁹ 56 percent of LGBTQ adults surveyed reported experiencing some form of discrimination from a health care provider, which jumps to 70 percent for those who are gender diverse. Candrian explained that in 2020, she interviewed a woman who had just lost her wife to cancer. This woman said when hospital staff realized they were a couple, the quality of her wife’s care declined. When Candrian asked her what the most difficult part of the experience was, she responded that her wife was dying, and because they were hiding their relationship, she could not say that they

___________________

³⁸ See https://www.aarp.org/research/topics/life/info-2018/maintaining-dignity-lgbt.html (accessed February 15, 2024).

³⁹ See https://newsroom.heart.org/news/discrimination-contributes-to-poorer-heart-health-for-lgbtq-adults (accessed February 15, 2024).

were married; as a result, she could not talk with the care team about next steps, diagnosis, or treatment or advocate for her wife on any level.

What this story reveals, according to Candrian, is how communication affected outcomes: the quality of care worsened, and the patient did not have the person she needed in the room to advocate on her behalf because she was excluded as nonfamily. The woman Candrian interviewed said that she did not receive any grief support because the health care system treated her as if she were just the emergency contact or a good friend and not a grieving spouse who had just lost the love of her life for the past 33 years. She was so traumatized that she avoided the medical system for an entire year even though she noticed a suspicious growth on her neck.

Candrian noted the documented discrimination and disparities in outcomes, but many people are worried about sharing that information. Compounding the problem is a health care system that does not routinely ask about sexual orientation and gender identity. “It is like an unofficial ‘don’t ask, don’t tell’ policy,” said Candrian. “We do not ask because it is currently not built into our routine forms, our intake questions, or the ‘get to know you’ ways of interacting with patients. It can feel uncomfortable for both the asker and the teller,” she added.

Candrian noted that the effect of this unofficial policy in medicine is to unintentionally limit how people share critical information about themselves and who matters to them, which means clinicians have incomplete data about their patients. This will continue unless the health care system makes it clear that it is safe to share such information, emphasized Candrian. Legislation would provide a first step toward a solution by giving protections to those who need it, but it is hard to legislate how people think and feel. “What we can do is change the way we communicate, the way we talk, and the way we listen, and one big way to do that is what I call ‘breaking the script,’” said Candrian. She explained that interactions with patients are often scripted and formulaic. Breaking these scripts opens communication channels to give people space to answer in a way that fits them and their life. Breaking the script takes the burden off of the patients so they do not have to be the ones to come out repeatedly to a battery of new people when they are sick, scared, and overwhelmed.

An example of breaking the script would be instead of asking if someone is married, ask who the biggest support in their life is and if they live with anyone. Candrian said that another script used often in medicine is around the family meeting and palliative care, but like marriage, the word “family” can have boundaries around it and box people in or out. Instead of asking someone about their family, ask them who needs to be in the room when discussing care options and who needs support so they can support the patient.

Candrian explained that the habit of silence also affects research at the macro level because it results in incomplete and inaccurate data about the LGBTQ community and leaves them invisible in much of the medical literature, including palliative care. When dealing with incomplete data, it is hard to analyze disparities and outcomes and develop interventions targeted to address them. “We need to consistently start asking these questions around sexual orientation and gender identity, and we need to do it in a way that people feel safe,” said Candrian. “I realize that this might feel uncomfortable and awkward, but the more we start asking these questions like we do other demographics, the more habitual and routine and hopefully comfortable they will feel.”

To illustrate how this can happen, Candrian shared examples the Census Bureau tested based on recommendations from the National Academies of Sciences, Engineering, and Medicine (2020) and Office of Management and Budget.⁴⁰ Gender identity has two questions: What sex were you assigned at birth? What is your current gender? The response categories are male, female, transgender, nonbinary, and “this person uses a different term” with space to write in a response. Sexual orientation has one question: Which of the following best represents how you think about yourself? The response categories are gay or lesbian, straight or not gay or lesbian, bisexual, and “this person uses a different term” with space to write in a response.

Candrian closed her remarks with two key suggestions:

• break the script routinely, and
• consistently ask questions about sexual and gender identity and include the data in research.

She underscored that she hoped everyone would remember that the habit of silence has existed for decades, and it persists and affects the way LGBTQ people move through the world as well as what information they share and with whom. “We have to be proactive here at every level at every interaction and think about what you really need to know,” she said. “Does the way you frame your question give space for lots of different types of answers? Does it say, ‘I am okay with who you are and who you love, and you are safe here’?” She noted that the LGBTQ people she has met in her research are not asking for much, just to be able to talk openly and be heard without prejudice. “All we have to do is break these scripts just a little, show them they are safe, ask, and listen,” emphasized Candrian.

___________________

⁴⁰ See https://www.federalregister.gov/documents/2023/09/19/2023-20256/agency-information-collection-activities-submission-to-the-office-of-management-and-budget-omb-for (accessed February 15, 2024).

Diversity in Clinical Trials

Robert Winn, director of the Massey Cancer Center and senior associate dean for cancer innovation in the Virginia Commonwealth University School of Medicine, began his presentation by paraphrasing the late Toni Morrison, who wrote that changing the status quo occurs not by just doing something different but by doing a different thing. “This issue around getting different folks on to clinical trials and increasing the enrollment of at-risk populations, where it is rural, African American, or what have you, I saw as an opportunity to do a different thing,” said Winn.

Applauding the work of clinical trialists and their ability to create designs for the most highly impactful clinical trials, Winn noted his disappointment that the enrolled populations in these trials were not diverse. Despite a 40+ year body of work on the science of outreach and engagement, the design of a trial is often set before engaging with outreach and engagement experts, which Winn said is too little, too late. Winn established the Robert A. Winn Diversity in Clinical Trials Award Program (Winn Awards), sponsored by the Bristol Myers Squibb Foundation, Gilead, and Amgen,⁴¹ to address this challenge. The goal of the 2-year community-oriented program is to provide the skill sets needed to both design and implement clinical trials and interact with the community to advance trial diversity for 650–700 early-stage investigators and medical students.

Administered by the Virginia Commonwealth University Massey Cancer Center, Winn Awards are available for early-stage investigators who are within 10 years of finishing their terminal degrees. Winn said that the program has now trained more than 300 community-oriented clinical investigators, providing ongoing career support and leadership development as they go through the program. Awardees, who receive a salary of approximately $120,000 a year, learn how to use a community asset map at the beginning of the trial design process. They also establish an individual professional development plan and engage in tiered mentoring to support their ability to do something different. The therapeutic areas of focus are cancer, cardiovascular disease, and immunologic disorders.

The individual professional development plan, said Winn, focuses on the scholar’s training and career development road map informed by community-oriented clinical trial skills and developed in collaboration

___________________

⁴¹ Additional information is available at https://diversityinclinicaltrials.org/ (accessed January 24, 2024).

with the grantee’s mentors. It is reviewed, updated, and revised during the program and includes self-assessment of one’s competencies. In addition to helping the grantees develop clinical research skills, such as biostatistics and statistical analysis, the program also provides training on various career and professional development skills, such as grant writing, public speaking, project management, and technical report writing.

The program’s core principle, explained Winn, is sensitizing the awardees to the difficulties of getting members of diverse communities, who may be struggling with other aspects of their lives, enrolled in clinical trials. He noted that program leaders have been getting the sense that this is a different and needed approach to training. The proof will be once the first cohort of grantees begins to put the skills they learn into practice, however.

Experts in outreach and engagement are frustrated that the people who design trials are unaware of recruitment challenges or that the science of engagement exists, said Winn. At the same time, those same experts doing outreach and engagement in the community have no idea how difficult it is to develop a candidate molecule that becomes a medicine that gets into a clinical trial. The Winn Awards’ hypothesis is that exposing a group of people to both at the same time with the same emphasis and value would produce different results.

To inform the next generation of researchers, the Winn Awards launched a parallel 6-week program for medical students who come from underrepresented and minoritized populations and those who have an interest in diversity and disparities work. The early-career investigators mentor the students.

Winn said that the early-career grantees practice trauma-informed community engagement and partnership building in local communities. The goal is to deepen their knowledge of how to work with focus groups, conduct key informant interviews, establish listening sessions, participate in community meetings, and use a community asset map before designing a clinical trial. He noted the literature stating that African Americans and members of other minoritized communities do not want to enroll in clinical trials. In fact, he said, research shows that going into the community and simply asking people to enroll will generate a good response.

However, asking is not enough, because how one asks is what matters most. “Not only do we commit the first sin of not asking, but when we do ask, we are not talking to and trying to connect with the community,” said Winn. “We are simply talking at and trying to get someone into our clinical trials, and they know that.” After 2 years in the program, however, and learning the necessary communication skills, the results have been radically different. The results from the first two cohorts of awardees show that they

have been able to not only recruit diverse patients into clinical trials but also increase enrollment overall. One of the key messages they learn to deliver is that a clinical trial is both an experiment and an extension of health care and improves the standard of care.

In closing, Winn emphasized that the problem is not that diverse communities do not want to be on trials. Rather, it is that the clinical trials community has been fragmented and disorganized and is ignoring the importance of being able to communicate the power of trials with a unified voice. “I hope this program gives us at least an attempt to move things forward,” he said. He noted that based on the early success of the career development awards, the Winn program has added a 3-year clinical investigator leadership award powered by Conquer Cancer’s EveryGrant, which is open to graduates of the early-investigator program and designed to provide more advanced skills in clinical trials design, leadership, and mentorship.

Panel Discussion

Session co-moderator Robinson-Lane asked panelists about some of the ways that researchers can be more thoughtful about identifying implicit bias in research study design, data collection practices, and interpretations of findings, as well as the use of a community assets-based approach discussed by Dr. Winn. Bullock replied that awareness of implicit bias is a starting point. She has found that when talking about it with many of her colleagues, it is the first time they have considered it as something that creates a barrier. She recounted how a physician asked her why he was not aware of structural racism and implicit bias, and her response was that they did not have to know about it and never learned about in medical school. In Robinson-Lane’s opinion, it is important to continue to engage in discourse and to inform the medical and research communities about commonly used language that is racist and marginalizing that has been normalized. As an example, she said that a colleague told her they needed to have a powwow. “I was offended,” she said. “A powwow is a very sacred spiritual event. How dare you refer to a difficult meeting as a powwow.” Another example is the phrase “low on the totem pole,” which is offensive language to Indigenous people because a totem pole is sacred to them.

Bakitas suggested turning the question around from why people from certain communities do not enroll in hospice, for example, to why people from those same communities choose to enroll in hospice, which was because a trusted clinician recommended it. “I do think it is important for us to think about our language, reframe it, and stop admiring the problem and start doing something about it,” said Bakitas.

Winn added that focusing on biology and race without also considering the role an individual’s zip code—their “ZNA,” as he called it—plays in how individuals interact with the health care system and has an impact in terms of poorer health outcomes for certain populations. “We all need to start wrapping our minds around the fact that our DNA is not the sole driver and that there is intersectionality and interplay with our ZNA. Maybe we can start focusing from there and have better results,” said Winn.

Davila asked Bakitas to comment on what can be learned from rural patient populations to craft interventions that effectively serve them. Bakitas said one has to be willing and able to invest the time it takes to get to know people and then listen to and use their advice. She acknowledged that this can be difficult given funding deadlines, but “you might as well take the time to do it right. If you want to go far, you need a village to go with you.”

Bullock noted the importance of meeting people where they are when designing studies, being flexible, and using icebreakers appropriate for specific communities. For example, when addressing Black Americans in a focus group, she first asks people how they want to be referred to. Older Black people, for example, find it disrespectful for younger people to call them by their first name. She then gives them the option to tell her anything else about themselves, and although some people provide no other information, that is okay. “When we force people to tell us things, that is starting off wrong. Right there, they know you do not really care about them, that you just want to get your data and get out of there,” Bullock observed.

Robinson-Lane asked Candrian to discuss the strategies she uses to build relationships with people who are not naturally extroverted. Candrian replied that the strategy differs with each community, but the key is to engage with humility and involve the community in every stage of research. “You have to let them know that you are really doing this because you want to improve the way they are cared for and you want to improve their lives,” she said.

Audience Q&A

An audience member asked how researchers can maintain an antiracist approach and equity-focused perspective in all aspects and phases of research. Bullock responded that in her view, the first step would be to ask the research team what would be required to take an antiracist approach. She noted that the requirements would vary depending on the research team and the setting and noted that a question worth exploring is “whether or

not the research may be perpetuating structural and systemic racism.” She added that “sometimes asking the question will get you much further than if you never asked.”

Stephen Bartels, the inaugural James J. and Jean H. Mongan Chair in Health Policy and Community Health in the Department of Medicine, professor of medicine at Massachusetts General Hospital and Harvard Medical School, and director of the Mongan Institute, commented that he and his colleagues in implementation research have wondered why interventions designed for complex conditions are shown to be effective, and yet no uptake or impact occurs at the public health level. “Maybe the trial was not designed from the get-go to have uptake, acceptability, reach, and sustainability, and maybe the people designing these interventions live in ivory towers like the one I live in,” he said, referring to the possibility that intervention designers are not using user-centered or community-engaged or participatory design. Research, he said, has to embrace the recipients, but being sufficiently flexible to incorporate community input in a trial is difficult. He asked panelists how to accommodate intersectionality—Black, LGBTQ individuals who have a disability, for example—in recruitment efforts. He also asked if the field should be thinking less about effectiveness and more about reach, uptake, acceptability, sustainability, and impact when designing an intervention.

Bullock replied that no magic wand exists for designing any intervention, research project, or clinical program to be inclusive. The key is to start with focusing inward and look introspectively at what each team member is doing to create barriers. “If we remove barriers, then people can come together as their whole self,” explained Bullock. Doing so would lead to interventions that everyone can use and benefit from. The problem, she said, is that people working within programs and research are perpetuating stereotypes and biases. If individuals took responsibility to ensure they are being antiracist and antihomophobic, for example, it would lead to programs that are inclusive, which would mean that interventions would not have to be designed for specific populations. “If we cannot allow ourselves to see and know that we are creating barriers, then we cannot dismantle them,” she emphasized.

Winn provided an example of a common procedure that was not designed to accommodate differences. Electroencephalograms (EEGs) do not work on African American people with certain hairstyles. “Wouldn’t it have been wonderful to actually have a team up front designing the EEG and saying here are some of the different variations on that, and how could

we actually design something at the beginning that could fit our different populations that we are aware of?” Winn asked, explaining that the Winn Awards are trying to address that challenge.

Davila added that engaging the community and having true community partnerships from the beginning of a project, before beginning to design, is critical. She said that the Massachusetts Coalition for Serious Illness Care is partnering with the Patient Insight Institute to establish the Massachusetts Patients and Community Review Board to provide a resource for implementation researchers to share ideas with about a dozen individuals who have received training and are compensated for providing their lived experience and expert input into a study in its infancy.

IMPLEMENTATION SCIENCE: TRANSLATING RESEARCH INTO PRACTICE

Huong Nguyen, interim director of the Division of Health Services Research and Implementation Science at Kaiser Permanente Southern California and professor in the Department of Health Systems Science at the Kaiser Permanente Bernard J. Tyson School of Medicine, referred to the oft-cited statement that it takes 17 years to scale research evidence into real-world practice. Those in implementation science are working to significantly reduce that time frame, she noted, while maintaining the fidelity and effectiveness of interventions. Nguyen highlighted the significant amount of retrofitting needed to move research interventions into real-world settings given that, historically, interventions have not been designed at the outset to align with the lived experiences of diverse populations. She noted that the session would begin with a broad discussion of implementation science followed by presentations focusing on implementation science in acute care settings and in nursing home settings and ending with a discussion of implementation of Hospital at Home.

Implementation Science and Serious Illness Care Research

Implementation science is important, explained Stephen J. Bartels (James J. and Jean H. Mongan Chair of Health Policy and Community Health, Professor of Medicine, Massachusetts General Hospital [MGH] and Harvard Medical School, and Director, Mongan Institute at MGH), especially in light of the long time frame referenced by Nguyen. That it takes 17 years to implement only 14 percent of research into practice is “a poor

investment of precious research resources,” emphasized Bartels (Balas and Boren, 2000). He shared that health care system leaders often point out that academic research is too incremental, cumbersome, siloed, and bound up in an investigator’s methods to make a difference at a time when health care leaders need help. Potentially contributing to the problem, explained Bartels, is adherence to an incremental model of conducting science one step at a time, as represented by the six-stage NIH stage model (presented by Grudzen earlier in the workshop). He argued that innovations in research methods, including hybrid-effectiveness implementation research designs, and agile, adaptive approaches to research accelerate the process of moving from research findings to implementation in real-world settings.

Bartels pointed out that effective implementation is essential to the impact of an intervention: even if an intervention is 100 percent effective, ultimately its success depends upon how and whether it is adopted, practitioners receiving training to deliver it, trained practitioners choosing to deliver it, eligible populations receiving it, and the sustainability of the intervention over time. Achieving a 50 percent success rate for each of these qualifiers would be considered a “home run,” Bartels explained, but overall, the cumulative benefit would be relatively limited.

Bartels shared the widely promulgated definition of implementation science: “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and hence, to improve the quality and effectiveness of health services and care” (Eccles and Mittman, 2006). Describing a continuum of science, from efficacy trials to hybrid-effectiveness implementation trials (see Figure 12), Bartels explained that implementation trials do not focus on efficacy because it is assumed that an intervention ready for implementation is effective. “Adoption, fidelity, organizational change, spread, and sustainability are our outcomes above and beyond effectiveness, which is really confirmatory,” explained Bartels.

Focusing on the other end of the continuum, hybrid-effectiveness implementation trial designs are accelerating the science, in Bartels’s view (Curran et al., 2012). Basic hybrid “Type I” clinical effectiveness trials measure whether an implementation has occurred and observe any barriers or facilitators to implementation. “I would suggest that all effectiveness trials should observe and collect data on barriers and facilitators to implementation,” he said. “It is a missed opportunity not to use mixed methods or other qualitative measures of factors associated in a randomized effectiveness trial with successful implementation to inform subsequent dispersal,” argued Bartels.

Bartels explained that a hybrid trial that tests implementation strategies while observing and assessing clinical outcomes may use one of many outcome measures and frameworks, such as the RE-AIM framework (Glasgow et al., 1999). Reaching the targeted population, adopting an intervention, or developing organizational support to deliver an intervention and maintain or sustain it in the long term are critical to achieving real impact of an intervention, in Bartels’s view.

Referring to a statement from an earlier session regarding the importance of maintaining fidelity for implementation, Bartels posited that “strict adherence to fidelity may not be what we should be doing.” Instead, he argued, we should be incorporating the science of adaptation into implementation. Adaptations, he explained, are changes to the design, delivery, or form of an evidence-based intervention with the goal of improving uptake, fit, reach, efficiency, effectiveness, equity, and sustainability.

Bartels provided several examples of the importance of adaptation in the context of serious illness care. First, given the lack of access to serious illness communication and care delivery in many areas, adaptations could include delivering evidence-based interventions using community health workers, peers, and digital tools, such as the EHR and AI, for behavioral nudges as part of an intervention. Second, given disparities in serious illness communication and care delivery, it is critical to consider codesigning implementation strategies to advance health equity involving individuals who are representative of end users with differences in race/ethnicity, culture, language, disability, etc. Third, an intervention may need to be tailored for different populations because all serious illness interventions are not always going to work for different disease conditions or situations, such as translating palliative care skills and competencies to other specialties.

Adapting an intervention requires identifying core elements that are empirically or theoretically associated with the desired outcome and cannot be modified (Perez Jolles et al., 2019). “Form is modifiable, but the core components associated with the basic function of the intervention are immutable,” emphasized Bartels. For palliative care, identifying the core components is challenging because they change depending on illness trajectory. For example, advance care planning and treatment decisions become much more important components at the end of the visit cycle, as opposed to the beginning, whereas common issues such as rapport and illness understanding decreased in importance from initial visits to later in the visit cycle (Hoerger et al., 2018).

Bartels highlighted ongoing research efforts focused on implementation contextual issues, factors, and efforts focused on factors and drivers of implementations and adaptations (see Figure 13) (Paladino et al., 2022).

Adaptations could include modifying the illness conversation guide, customizing EHR documentation and workflows with prompts, and tailoring patient identification and data feedback approaches for specific needs. He characterized Paladino and colleagues’ work as “an elegant approach to using mixed methods and observational methods to describe drivers of successful implementation” in serious illness conversation research. Bartels noted that organizational factors such as leadership; incentives; program resources; data infrastructure; and at the local level, teams that can help with culture change also are important for successful implementation.

Bartels and his colleagues also discussed approaches to implement adaptations that drive equity (see Figure 14). The initial steps consist of meeting with partners, including community providers and community health centers, to review and assess the data to see how the implementation of an intervention addresses equity issues, what might be missing from the implementation plan, and how to adapt implementation to fill any gaps.

Subsequent steps include selecting adaptations to advance health equity followed by rapid cycle testing of planned adaptations. “This is an approach to use a community-engaged, data-driven, and stakeholder-engaged approach to adaptation to enhance equity in terms of implementation that could apply to the whole area of serious illness conversation that we are talking about in serious illness interventions,” Bartels explained.

Citing the work of Baumann and Cabassa (2020), Bartels outlined the key steps to achieving the goal of equity-focused implementation of serious illness interventions:

• focusing on reach and equity from the very beginning;
• designing and selecting interventions for complex conditions and needs and low-resource communities with implementation in mind;
• implementing what works and developing implementation strategies that are scalable, sustainable, and able to reduce inequities in care;

• developing the science of adaptations for equity and complex health challenges associated with serious illness; and
• using an equity lens for implementation outcomes.

In closing, Bartels emphasized that it is important to do the following:

• Advance implementation research methods to accelerate uptake, spread, impact, and sustainment of serious illness interventions.
• Identify the essential core components of interventions that cannot be changed with attention to fidelity to function and not to form; embrace community-engaged adaptation to improve uptake, fit, acceptability, feasibility, reach, and sustainability of serious illness interventions; and use these measures as guides.
• Engage patients and families representing a spectrum of identities and health inequities in intervention codesign to improve the likelihood of successful implementation, spread, and impact.
• Focus on engagement, equity, and reach from the beginning.

Implementation Research in Acute Care Settings

Meghan Lane-Fall, executive director of the Penn Implementation Science Center at the Leonard Davis Institute of Health Economics at the University of Pennsylvania Perelman School of Medicine, noted that she prefers Ann Sales’s definition of implementation science as “the study of planned human behavior change under organizational constraints.” ⁴² This definition, she said, highlights that implementation science is the amalgam of many different fields that are interested in these concepts. For example, human behavior change involves psychology, sociology, anthropology, public health, behavioral economics, and even engineering. “Anyone who has anything to say about how to change people’s behavior, what they have learned, and what we have learned from them feeds into implementation science,” observed Lane-Fall. When thinking about how to change behaviors in the context of a specific organization, management theory and organization theory are also important to implementation science.

___________________

⁴² See https://factor.niehs.nih.gov/2024/2/science-highlights/implementation-science-enviro-justice#:~:text=Defining%20implementation%20science&text=Lane%2DFall’s%20favorite%20definition%20comes,behavior%20change%20under%20organizational%20constraints.%E2%80%9D (accessed February 24, 2024).

In acute care settings, trials might be time limited and involve people at risk of death or other poor outcomes, noted Lane-Fall. For example, clinicians may not know whether to mechanically ventilate or start medications to support blood pressure and cardiovascular function, so they might try something for 24–48 hours to see if it works. For interventions such as family-centered rounds, evidence-based practices exist for advance care planning and inpatient palliative care consultations, but the question is whether those practices need to be adapted to the acute care setting for them to be implemented on a sustainable basis.

Human factors engineering, or ergonomics, is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system.⁴³ It applies theory, principles, data, and methods to design as a means of optimizing human well-being and overall system performance. Lane-Fall explained that for the fast-paced acute care setting, where clinicians are doing multiple things simultaneously, human factors engineering offers the opportunity to think through how to decompose the work system in a productive manner to determine how to enact complex behavioral interventions.

Lane-Fall pointed to the Systems Engineering Initiative for Patient Safety model (see Figure 15) (Carayon et al., 2006). In the acute care setting, this model encompasses technology and tools, the system’s organization, those implementing the change, the tasks involved, and their work environment. Understanding each of these and how they interact with another is key to facilitating behavior change in the acute care setting, explained Lane-Fall.

Lane-Fall noted that in acute care, time pressure forces people to use mental shortcuts to simplify problems and avoid cognitive overload, known as “heuristics.” “We tend to fall back on the ways that we always do things because it decreases the cognitive load of making decisions,” she explained. “So as we are trying to change behavior in acute care, as we are trying to implement serious illness interventions in acute care, that is what we are fighting against.” Thus, it is important to reduce the cognitive strain on the people who are the focus of an intervention.

One way to better comprehend this process is to understand the specific workflow for each individual who will be affected by implementation, explained Lane-Fall. If the goal is to have the nurse do one thing, the physician do another thing, and the nurse practitioner do something else,

___________________

⁴³ See https://iea.cc/about/what-is-ergonomics/ (accessed March 6, 2024).

for example, it is necessary to understand where they are and what they are doing at any given time. To ensure the desired behavior change actually occurs, the new behavior must be compatible with the current workflow.

Another important consideration is the principal-agent problem, which Lane-Fall explained is that clinicians in the acute care setting are acting on behalf of people. In most situations, the right course of action is aligned with what the patient wants and what the clinician or agent wants. However, conflicts will arise, and the right thing to do is to slow down, talk to the patient, and talk with their family to understand their needs and evoke their preferences for care, even though the provider has 20 other patients to see. However, workflow—what the agent needs to do—wins. Lane-Fall noted that part of the objective of aligning implementation approaches with workflow is to bring together what the principal needs and what the agent needs, which can only happen with an understanding of the workflow. “It is important to decompose the workflow to be able to design these interventions appropriately,” she emphasized.

Qualitative methods, such as talking to and interviewing people and conducting focus groups, can help with understanding where interventions are being deployed. Observation, or learning by seeing, is powerful when it comes to determining what people are working with before asking them to change their behavior, explained Lane-Fall. She described how she and her colleagues go into different settings, take pictures, talk to people in their natural environment, and ask them if a potential intervention makes sense in that work setting. If it does not, they consider whether it is possible to re-engineer the environment or adjust the intervention—keeping the function but adjusting the form—so that the intervention fits into the work environment.

Lane-Fall noted some ethical issues relating to implementation in acute care, such as determining who serves as a gatekeeper to the setting and who provides consent. Other issues include determining how participants can opt out of research and how to prevent retaliation. She noted that it is important to consider the risks of an intervention to the people whose behavior it is trying to change. “What is the risk to the bedside provider who decides not to do the intervention, and what is the risk to the person who discloses that they are actually doing something that might be seen as dangerous or not evidence-based?” asked Lane-Fall.

The same issues apply to health care equity and considering the fit or appropriateness of an intervention for a particular population, especially those that have been deprioritized in prior research, Lane-Fall explained. It is also important to account for equity as it applies to the people who might work in a given setting. “We want to think about power hierarchy. We want to think about that certified nursing assistant who cannot speak up to their supervisor but has thoughts about how we ought to be doing things,” noted Lane-Fall. She stressed the importance of bringing everyone to the table to get a variety of perspectives on an intervention and how it is being implemented, both because it is the right thing to do and because asking people to change their behavior without first giving them the opportunity to hear about and share their views on the intervention will not work.

In closing, Lane-Fall shared three key takeaways:

• Inpatient settings are characterized by a fast pace and competing priorities.
• Human factors engineering holds promise for characterizing workflow and complementing implementation.
• Conducting implementation research in an acute care setting has ethical implications.

Serious Illness Care Research in Nursing Homes

Kathleen Unroe, associate professor of medicine, Indiana University School of Medicine, and research scientist, Indiana University Center for Aging Research, focused on specific implementation challenges in nursing home settings. She began her remarks by pointing out the institutional bias of the U.S. long-term care system, which features 16,000 nursing homes or skilled nursing facilities, nearly all Medicare and Medicaid certified. She noted that despite the push to change the long-term care system to be home- and community-based, institutions will continue to have an important role to play. More than 1 million people currently receive care in rehabilitation and long-term care facilities in this country. Skilled nursing facilities provide both short stay, post-acute care and long-term care. Unroe described the average skilled nursing facility as having approximately 100 beds, one-fifth of which will be occupied by individuals receiving post-acute care at any one time.

Unroe pointed out that 80 percent of all nursing home residents require assistance with one or more activities of daily living, and more than 60 percent have moderate to severe cognitive impairment. Approximately 40 percent self-report either persistent or intermittent pain; racially and ethnically minoritized people and severely cognitively impaired residents are more likely to have untreated or undertreated pain⁴⁴ (Hunnicutt et al., 2017). Given the burden of disease in nursing home settings, residents with dementia have many unmet palliative care needs. Symptoms are undertreated, and the quality of care can be poor, with burdensome interventions near the end of life, including transfers to the emergency department and hospital. Unroe noted that one study, for example, found that 41 percent of nursing home residents with dementia underwent at least one burdensome intervention in the last 3 months of life despite preferences by surrogate decision makers to focus on comfort (Mitchell et al., 2009). In 2017, 55 percent of deaths of people with dementia occurred in nursing homes (Cross et al., 2022).

Unroe pointed out the many types of evidence-based interventions to improve the consistency and quality of care in nursing home settings, but high rates of staff turnover present a significant barrier to implementation.

___________________

⁴⁴ For more information, see https://www.cms.gov/Medicare/Provider-Enrollment-and-certification/CertificationandComplianc/Downloads/nursinghomedatacompendium_508-2015.pdf (accessed February 10, 2024).

She added that high turnover of nursing home executive leadership makes it difficult but not impossible to establish research partnerships.

Unroe described the implementation of the Optimizing Patient Transfers, Impacting Medical Quality, and Improving Symptoms: Transforming Institutional Care (OPTIMISTIC)⁴⁵ program, a $30.3 million clinical and payment demonstration project funded by the Center for Medicare and Medicaid Innovation. With this large grant, she and her collaborators were able to embed staff in nursing homes, take a proactive role in dealing directly with gaps in hands-on care, and bring nurse practitioners into the facilities. Nineteen facilities received this in-person model of care for 8 years, Unroe explained, and 40 facilities received resources and support for 4 years to provide acute care in place with the goal of reducing avoidable hospital transfers of long-stay residents (Unroe et al., 2018).

The model relied on several evidence-based practices for implementation, such as having nurses conduct robust advance care planning and document goals of care with residents and families, evidence-based practices focused on high-quality transitions of care and communication, nursing assessment, and common clinical conditions. “There are so many predictable common clinical conditions that need to be treated and need to be treated consistently and quickly in the nursing home setting,” explained Unroe. She noted that experienced nurses served as mentors and supported frontline nursing home staff.

Unroe noted that implementation of OPTIMISTIC had several important positive outcomes, which included reducing Medicare costs. The project also reduced hospitalizations by one-third compared to control group facilities, according to an external evaluation (Ingber et al., 2017). Unroe and her colleagues have applied lessons learned from the OPTIMISTIC project to the Utilizing Palliative Leaders in Facilities to Transform Care (UPLIFT) program. UPLIFT is an ongoing Phase 3 clinical trial in eight facilities in Indiana and eight facilities in Maryland. The trial’s core elements focus on a palliative care lead and all-staff education occurring over time to account for turnover (Unroe et al., 2023). “We are trying to build internal capacity for palliative care in the facility, recognizing the population that is there and creating an environment where palliative care can be delivered and supported,” explained Unroe.

___________________

⁴⁵ For more information, see https://www.probarisystems.com/demonstration-project (accessed January 18, 2024).

The key to the palliative care lead role, Unroe noted, is systematically screening residents and identifying those who have palliative care needs. For those who do have palliative care needs, UPLIFT brings in medical palliative care specialists who provide onsite visits with residents and, whenever possible, their families. UPLIFT focuses on residents with moderate to advanced dementia in nursing homes. Unroe and her collaborators chose End-of-Life in Dementia—Comfort Assessment in Dying with Dementia (Kiely et al., 2012) and End-of-Life in Dementia—Symptom Management (Klapwijk et al., 2021) outcomes that researchers used in a large palliative care trial in European nursing homes.

Unroe explained that connections and collaboration are critical to conducting implementation work and research in nursing home settings. “I do not think it is possible to do this work without having strong relationships or strong connections across these different domains,” she emphasized. Unroe noted that getting letters of support from the leader of a nursing home chain is great, but buy-in from the executive director and director of nursing of the local facility is essential. In addition, it may be necessary to ensure that the physician and nurse practitioner provider groups support or at least will not serve as a barrier to research. Supportive physician and nurse practitioner provider groups can serve as leaders and advance program implementation.

Partnerships with community groups, universities, and trade organizations can be beneficial and useful, added Unroe. “Trade organizations for nursing homes have been a great partner in research because they are so connected to the stressors, realities, and the rapidly changing regulatory environments of their members,” she explained. “If there is a good working relationship there, they will let you know what [their] priorities are and can identify that what you are working on is aligned with what is needed in the field.” Unroe pointed out that trade organizations can also help identify and recruit facilities that might be willing to participate in research, and they are important partners for disseminating an intervention once it has been tested and shown to be effective.

In Unroe’s view, nursing home residents undeniably have substantial unmet palliative care needs, and evidence shows that palliative care is effective for this population. However, consistent implementation and delivery of these interventions remains challenging. Areas of needed research include symptom assessment and management, models for delivering supportive care, targeted trainings, and financing serious illness care delivery. Implementation planning is critical, said Unroe, and interventions must be tested in the nursing home setting.

Multidisciplinary collaborations and partnerships with industry and ongoing investments are needed to test interventions and improve care in nursing homes. “We need to work with the industry, side by side, to test in the setting and deal with, navigate, and identify all of these implementation challenges up front to get evidence-based therapies to this population,” noted Unroe. “We have to do it with collaborations. We have to do the hard work building relationships, but it is an investment worth making,” she emphasized.

Implementation of Care Delivery Models in the Community and Home: Lessons from the 20-Year Odyssey of Hospital at Home

Al Siu, professor in the Brookdale Department of Geriatrics and Palliative Medicine at the Icahn School of Medicine at Mount Sinai, likened Hospital at Home’s journey to Odysseus’s 20-year journey home from the Trojan wars, given that the foundational paper demonstrating the effectiveness and feasibility of implementing Hospital at Home was published in 2005 (Leff et al., 2005), and only today is the program on the verge of national dissemination aided by Congress passing the 2022 Omnibus Reconciliation Act, which included a waiver to pay for Hospital at Home.

Siu said that the typical episode of hospital care starts in the emergency department, transitioning to an inpatient bed and then into post-acute care in a skilled nursing facility. Hospital at Home aimed to replace the last two steps by providing post-acute care at home, along with transitional services at home when needed. Siu listed the similarities between Hospital at Home and community-based palliative care in that both focus on bringing services into the home, care for patients with serious illness with complex interventions involving multiple disciplines, and lack a dedicated fee-for-service payment model. In addition, both may encounter barriers to referral, staffing, payment, regulation, and supply chain and have had difficulties scaling.

Tracing the history of Hospital at Home, Siu explained that it followed a version of the NIH stage model discussed earlier in the workshop and began with foundational studies on the adverse outcomes of hospitalization for older persons. At the time, health care providers at Johns Hopkins were doing home visits for the homebound and would sometimes improvise and provide acute services in the home when individuals developed acute illnesses. In the early testing and development stage, the Johns Hopkins team identified the conditions and criteria for Hospital at Home and then tested

those eligibility criteria to understand patients’ acceptance of care at home. After a year of trying to get their protocol approved by their IRB, the team conducted the first clinical pilot, which was successful.

A larger multisite clinical trial involving managed care and the Veterans Administration revealed the program to be effective; Siu pointed out that the trial showed that the program achieved the triple aim of improving the experience of care, improving the health of populations, and reducing the per capita costs of health care several years before that concept was ever discussed (Leff et al., 2005). Nearly a decade later, Siu and his colleagues conducted a test of the intervention to reduce 30-day hospital readmissions. Although they had early successes in identifying the right patients and getting good outcomes, they encountered problems because the program starts in crowded emergency departments, where staff are busy and do not have time to screen and arrange home services and ensure that insurance will reimburse for care. “Hospital at Home clogs the system, and it did not work,” observed Siu. Another confounding factor was that many admission decisions are made late in the day, when it is challenging to initiate an episode of Hospital at Home. “I can get pizza delivered in Manhattan at midnight. I cannot get an oxygen tank delivered if I do not order it during business hours,” explained Siu.

Going back to the drawing board, Siu and his colleagues adapted the workflow and provided new clinical services and conditions. “We did palliative care at home that we would otherwise not have provided except for their goals of care, and we did rehab at home. We decided to introduce using telehealth technology, and we were able to replicate the results of the Hopkins studies in terms of showing improved patient outcomes, improved satisfaction, decreased cost, and decreased readmissions in this case,” Siu said.

Nonetheless, when the federal contract ended, Siu and his colleagues needed to find commercial contracts to keep the program in operation. They also formed a users’ group to provide technical assistance, develop program standards, and set quality indicators. The CMS pandemic waiver opened the door to disseminating Hospital at Home nationwide.⁴⁶

___________________

⁴⁶ CMS announced the Acute Hospital Care at Home waiver program in November 2020 to help expand hospital capacity to cope with the COVID-19 pandemic. For more information, see https://agsjournals.onlinelibrary.wiley.com/doi/10.1111/jgs.18071 (accessed February 15, 2024).

In terms of what he learned from these experiences, Siu noted that first, formal consideration of implementation should be done earlier in the process. Second, scaling and sustainability needs to be considered up front, not at the tail end of the NIH stages. “We would have made different programmatic decisions up front if we had known about the barriers of scaling and the needs for adaptation,” he said, noting that his team did not try any adaptations until 2015, after the program had been in the works for 15 years.

In closing, Siu said that Hospital at Home had the opportunity to consider both effectiveness and implementation at each stage of research. He reiterated the need to frontload implementation issues early into the development of an intervention. In the same way, scale and sustainability should be discussed earlier, with focused consideration to enable more rapid adoption of evidence-based innovations at the population level. “I do not know whether that would have reduced the 17 years or not, but I would argue, based on this experience, that perhaps we could have shaved a couple of years off,” observed Siu.

Panel Discussion

Session co-moderator Aslakson asked panelists to identify actions, given the knowledge they have, that they believe would help overcome the barriers that have led to the field making only incremental progress on implementing evidence-based interventions. Bartels highlighted Siu’s suggestion that study designers start with scalability, sustainability, and implementation at the beginning of developing interventions, using user-engaged and user-centered design principles from the get-go and thinking beyond effectiveness as an outcome. He reiterated his opinion that the NIH stage model is anachronistic: “I do not think anybody who runs a health care organization thinks that is the way that science should happen, except for CEOs of large academic health systems that accrue lots of grant funding from that approach. I think we need to be much more agile, much more rapid, and much more capable of thinking about these issues right out of the gate.” Bartels added that incremental science has many good attributes, but people who run health care systems, and patients and communities, think that researchers are not relevant in part due to the incrementalist approach.

Unroe pointed out the tension between wanting interventions to be scalable versus simple. She believes this desire for simplicity is what is

behind the excitement about technological solutions, as the research community develops the multicomponent interventions that are necessary to improve care for people with complex serious illness. For her, the right approach is to develop something that works and then figure out how to get it into practice. Siu added that the main challenge is developing interventions that are safe and effective and work when put into practice. “With Hospital at Home, we created a model that was safe and effective, but it only worked for patients who showed up between 8 AM and 2 PM on weekdays,” he said.

Lane-Fall recalled how a senior scientist at her institution recently said that his job was to discover what worked, and then it was someone else’s job to translate that into something that had impact, until he realized that his job was to make sure that what worked had impact. “I think that is part of what we need to do, to take on the mantle of making sure the interventions we are testing have impact,” said Lane-Fall. She acknowledged this does not mean every scientist needs to be an expert in implementation science, but team science can address this challenge. Many of the interventions in serious illness with which she is familiar already involve multidisciplinary teams. “It is not a far leap to think that your research teams ought to be multidisciplinary as well,” observed Lane-Fall. She also echoed calls for designing for implementation and dissemination from the beginning of a project.

Aslakson asked panelists to suggest one or two partners they would recommend be part of a research project. Bartels emphasized that including as diverse a group of patients and families as possible is essential and can provide a perspective that helps develop the capacity to scale and have relevance. He would also include people who are skilled in user-centered design and human systems engineering. Lane-Fall also recommended including human systems engineers and someone with deep expertise in qualitative and mixed methods who can ask questions that will deepen the impact and add nuance to a research project.

Unroe reiterated her suggestion to include trade organizations and added individuals from the Department of Health, Division of Aging, or state Medicaid program, especially for interventions relevant for nursing home care. “They are responsible for the costs of care in this setting, and if you can navigate that, they can be good partners both to write letters of support that can be compelling and as a dissemination channel,” Unroe noted. Siu pointed out that the John A. Hartford Foundation has been a partner from the very beginning of his work on Hospital at Home. When CMS

issued waivers during the pandemic, he was able to turn to the foundation and ask for technical assistance that would enable hospitals to apply for the waivers. Payers are another potential partner, given that dissemination is often thwarted by payment issues, said Siu.

Session co-moderator Nguyen asked panelists for their ideas on how to ensure that implementation plans include early input from systems and sites that are under resourced. Lane-Fall said that is a tough issue to address, but a good start is to understand the pain points for implementation of a given intervention. Questions she would ask include what things such systems wish they could do and services they could provide and how she can partner with them to address some of their challenges. “It is fine if you bring an idea to the table, but in that spirit of stakeholder engagement, be willing to adapt or pivot if what they are telling you is something different from what you are expecting,” she said.

Unroe noted that Mor has been leading an effort, the Long-Term Care Data Cooperative,⁴⁷ which is gathering EHR records from some 3,000 nursing homes. She believes that this type of data pooling will help nursing homes adopt the concept of a learning health system. Unroe added that the data cooperative is working with the larger EHR vendors and that nursing homes using those EHR systems will have systematic differences from those that do not have robust EHRs and may be working with smaller, homegrown products.

What concerns Siu is that many interventions require some organizational investment, but the organizations that have resources to invest are not necessarily those treating the most disadvantaged patients. Hospital at Home, for example, does a better job for disadvantaged patients, but some Medicaid programs still do not include it as an option. “It is hard to approach questions of equity when you do not have Medicaid on board,” said Siu. Similarly, he added, many safety net hospitals do not have the funds for these interventions.

Bartels said that the lack of a cadre of individuals who not only can walk in the shoes of the operational medical director or executive director but also have the scientific skills to conduct quality improvement research is creating challenges for implementation. “We need to train those individuals and develop a new generation of scientists,” said Bartels. The chal-

___________________

⁴⁷ Additional information is available at https://www.ltcdatacooperative.org/Pages/default.aspx (accessed February 24, 2024).

lenge, he noted, is that academia does not reward quality improvement research, so finding people who can walk the boundary between empirical science and embedded trials and operational execution is difficult. Addressing that issue will require new metrics for promotion and tenure that would recognize and value the impact that such individuals have in their fields. Lane-Fall agreed with Bartels and said that a related issue is how community engagement plays out in academic advancement, given that developing relationships in a community takes time that might not translate quickly into publications.

Aslakson asked panelists for ideas on how to balance the need for both fidelity and adaptation in implementation. Unroe replied that this is a difficult issue because it is difficult to turn down a request from the director of nursing, for example, who might want to include some feature. “You are trying to listen and be respectful of these engaged partners, but you cannot do everything they want,” Unroe explained. She suggested carefully documenting the research team’s decisions and their reasons to enable continued learning about the implementation.

Lane-Fall said that fidelity to function is crucial because that is the mechanistic link between the intervention and the outcome, even though the form of the link may vary. This requires having key insights into what features of an intervention are form versus functional elements. She noted the work of Shannon Stirman, associate professor in the Department of Psychiatry and Behavioral Sciences at Stanford Medicine, who has built frameworks to help people document planned and unplanned adaptations and understand what their consequences might be (Holtrop et al., 2022; Kirk et al., 2020; Mui et al., 2023). “I do not think we should shy away from adaptation, but there does need to be some rigor and structure behind it,” stressed Lane-Fall. Siu pointed to the importance of holding oneself accountable to the core elements of an intervention and collecting data on the effect of adaptation to add to the evidence base.

Bartels described a large implementation study of an intervention for cardiovascular risk reduction in people with serious mental illness. He and his team were astounded to find that the vast majority of the 48 organizations involved did adapt and alter the intervention. The study included measurements of fidelity-consistent and fidelity-inconsistent adaptations and found that one of the “inconsistent adaptations” produced the best result. This finding raises the question of who determines what a core feature is and the need to better define that from the perspective of the

researcher and the community affected. The lesson, noted Bartels, is to be humble and recognize that providers, patients, and end users are often smarter than an intervention’s developers.

Audience Q&A

Diane Meier, director emerita and strategic medical advisor for CAPC, professor of geriatrics and palliative medicine, and the Catherine Gaisman Professor of Medical Ethics at the Icahn School of Medicine at Mount Sinai, said that her work on scaling access to palliative care followed business principles rather than the NIH stage model, such as research to identify the issues important to particular audiences and the related necessary behavior changes. For example, she explained, CEOs care about length of stay, readmissions, and hospital mortality and complications; clinicians care about time; patients and families just want someone to sit with them, make eye contact, and listen; and payers want reduced utilization of health care services. “Every one of those audiences’ behaviors need to change to get palliative care to happen,” said Meier.

To accomplish that, she and her collaborators conducted audience research and developed messages that address the issues identified for each audience. They established the business case for palliative care, acknowledging that it was not going to scale based solely on its moral imperative. “Palliative care scaled because important people [said] that cost savings were important,” explained Meier. She noted that for-profit hospitals are the one group of acute care hospitals that has not scaled access to palliative care.

Meier said that the field could learn much from business schools about implementation and scale. She also pointed to the importance of trusting local adaptations and the local knowledge of people working in a given context regarding how to take an innovation and implement it successfully in their context. “That has worked for palliative care, and I think it likely could have benefit for many other interventions,” emphasized Meier.

Lane-Fall agreed with the importance of considering organizational constraints. She noted that in addition to human factors engineers, she partners with management scientists to better understand those constraints. She also noted that audience segmentation is a key principle when dealing with CEOs and frontline clinicians. This is not about being duplicitous but rather meeting those two different groups where they are, speaking to

the issues they care about, and using that knowledge in the service of the goal. Lane-Fall added that achieving big change requires figuring out how to communicate effectively with policy makers.

Jodie Dionne-Odom, associate professor of medicine in the UAB Division of Infectious Diseases, asked the panelists what they would include as a service or core function in a national serious illness care research and implementation infrastructure. Lane-Fall said that the NIH Implementing a Maternal Health and Pregnancy Outcomes Vision for Everyone initiative and NIH-funded Research Centers of Excellence work with NIH’s implementation science hub and data innovation and coordination hub, both of which provide technical assistance. She would like to see this model for centralizing expertise play out in other areas of care and can see a national serious illness network supported by an implementation science hub. Siu added that the National Cancer Institute’s Training Institute for Dissemination and Implementation Research in Cancer is another relevant model.

Morrison, noting that a substantial body of evidence shows that quality of care in not-for-profit health care systems is significantly higher than in for-profit systems, asked panelists to comment on how academic medical centers and academic health care can partner with for-profit organizations to ensure that the quality of care they deliver remains high. Bartels said that this is where the team science approach is so critical. He described a team at his institution that looks at outcomes from different models of care in Medicare Advantage plans and can provide real-time data that could help a learning health system understand what works and what does not. However, that is difficult for private enterprises to do. “I do think it is the role of academic health centers to work collaboratively and objectively to develop the methods and the approaches to look at variations in intensity of care and whether or not they make a difference and whether or not the types of tools that are being used in these Medicare Advantage plans actually make sense,” said Bartels.

Siu pointed out the difficulty of scaling any intervention in an academic medical center or health system where Medicare Advantage or other commercial payer is a tiny slice of the institution’s business because each commercial entity will want a slightly different adaptation to an intervention. Unroe said that the most effective role academia can play is developing quality metrics for accountability. Lane-Fall agreed, saying that academic researchers excel at measurement and interpretation. She also noted that implementation is meant to be iterative, so academic researchers can help

ensure that an intervention is still being implemented according to plan with adjustments as needed.

Nguyen described Kaiser Permanente’s Medicare Advantage plan and how it focuses on the quality metrics that power the CMS five-star rating system. Anything that does not fit with promoting a five-star rating will fall on deaf ears, she said. Some optimism exists that Medicare Advantage plans are starting to be concerned about the ZNA that Winn mentioned. This could lead to evaluation on how they are performing in relation to social and health equity metrics and underserved populations. “There is some positive movement in that regard around quality metrics and accountability for Medicare Advantage plans,” noted Nguyen.

Jori Bogetz, associate director of research at the Treuman Katz Center, assistant professor in the Division of Bioethics and Palliative Care at the University of Washington, and pediatric palliative care physician at Seattle Children’s Hospital, said that she is struck by some of the similarities across the age spectrum in serious illness care, and she asked panelists for their thoughts about how the lessons they have learned can be applied at all ages. Lane-Fall said that all the principles apply across the age spectrum, with the additional piece of the parent and care partner role. The key is to understand who the partners are for an intervention and consider that child–parent and child–care partner dynamic. One issue might be the relative lack of evidence in pediatrics because of the problems with doing research on children, so it may be challenging to understand when an intervention is ready for implementation. However, she added, pediatrics has well-established research networks that effectively deploy principles from both implementation science and quality improvement to strengthen confidence in the evidence base.

Donald Sullivan, associate professor of medicine in the Division of Pulmonary and Critical Care Medicine at the Oregon Health Sciences University School of Medicine, said that an important factor influencing implementation is organizational culture and wondered what panelists think about influencing and operationalizing organizational culture in the setting in which they work. Lane-Fall said that good places to start include qualitative and quantitative methods to assess culture, such as the Organizational Readiness for Change Assessment.⁴⁸ She noted that given provider or care partner autonomy, as is true with the acute care nurses she works with, organizational culture may be less immediately relevant.

___________________

⁴⁸ The Organizational Readiness for Change Assessment is a quantitative measurement instrument. For information, see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7881456/ (accessed January 24, 2024).

Lane-Fall added that organizational readiness for change is not the same as organizational readiness to do research, but academic investigators often end up in places that are ready to do research rather than those that are ready to change. Bartels agreed that the Organizational Readiness for Change scale is a wonderful tool. He added that a real challenge arises when an organization does not have a culture, is not ready to change, and does not have a champion to support change.

Susan Hurley, senior scholar at Care Dimensions, created the embedded role of director of research in a community-based hospice organization that could speak to the research community and have an operational role in workforce training and development. She noted that community-based organizations do not have a negotiated and direct rate with NIH that normally provides additional funding for infrastructure. She suggested that implementation projects include funds to support the mentorship and recruitment efforts at which community-based organizations excel.

Grudzen asked Siu how he would have done things differently if those developing Hospital at Home had considered implementation when they started their work. Siu replied that he would have considered scaling earlier and would have developed some mechanism by which patients who entered the system late in the day could be held over until morning. He noted that although he agreed with the emergency department director about implementation, he did not get buy-in. “I think it would have been important to have included somebody from emergency department leadership in the leadership of Hospital at Home and to get buy-in from the very beginning of the implementation strategy,” explained Siu.

As a final comment, Lane-Fall pointed to the importance of questioning assumptions about interventions and thinking about what would make an intervention fail. “That will give you some insight into what you need to dig into to make implementation successful,” she explained.

LOOKING TO THE FUTURE: RESEARCH PRIORITIES FOR SERIOUS ILLNESS CARE

The final session began with co-moderator Halpern summarizing some of the key takeaways from the presentations. He noted that the opening lived experience session featured some great examples of how to involve patients and other partners in research, PCORI has changed researchers’ behavior by requiring them to engage partners to secure funding for their projects, and some NIH study sections are looking at engaging partners as an important consideration in grant proposals. Halpern pointed to the broad spectrum of research methods that serious illness care researchers

should use in their studies discussed in the second session and the thorny issues regarding patient-reported outcomes and the tradeoffs involved in using them discussed in the third session. He said that the fourth session presented critical issues relating to integrating health equity into research, and the fifth session focused on implementation science, both of which had “actionable insights for this community.”

Offering a number of observations about the presentations, Kutner noted that the serious illness care research community is truly special in its vibrancy, enthusiasm, sense of community, focus on collaboration, shared purpose, engagement across multiple disciplines, and deep commitment to improving care for patients and caregivers. “I cannot imagine another area of science that is this welcoming, this interdisciplinary, and this engaged,” she said. “That is something we need to embrace, nurture, and continue to carry forward.” She added that she was impressed by how far the field has come in one generation of researchers and was excited by its innovative and cutting-edge science. She noted that an important indication of how far the field has come is that it is no longer necessary to explain what palliative care is in grant applications.

Kutner highlighted that a theme across the sessions was that serious illness care research is anchored in patients and their care partners. “That is our ‘why,’ and it continues to hold across all our research,” said Kutner. Researchers are getting better at including those who are the focus of an intervention at the start of designing it and when implementing and disseminating it. She noted that when thinking about the value of serious illness care, it is imperative to consider multiple partner perspectives across the lifespan, in care settings, and at the societal level. “We need to be very purposeful about assuring inclusion of diverse perspectives, especially from historically marginalized populations,” emphasized Kutner. She shared several phrases from throughout the workshop that resonated with her, including centering equity at the beginning of the research design process, unmuting voices, engaging those unique and diverse voices and perspectives, and approaching research design through an equity lens.

Other important perspectives to be considered in addition to patients and their care partners include health systems, payers, frontline clinicians, advocacy groups, and trade organizations, Kutner stressed. She noted that scalability, adaptation, implementation, and sustainability should be considered up front, and the goal should be to make an impact, not to publish a paper that sits on a shelf. “As we design our research, we need to be thinking about the multidimensional components of quality of life,” said Kutner.

Intervention design should include testing the mechanisms driving those multidimensional components as a project proceeds. Doing that without reinventing the wheel requires leveraging complementary areas of expertise and including psychometricians, social scientists, health economists, and others on the research team.

Kutner also pointed out that the field needs the right methods and measures to answer important research questions. If they are not available, it is important to develop them in a rigorous manner to ensure that what is being measured is what an intervention is supposed to be affecting. Research also needs to address important, high-priority, and immediate problems that will make a difference to patients across the age spectrum and care partners and think strategically about the long game. “What does it look like to have sustained support for serious illness science as an area of focus for funding so that we can continue to do this work in the long term?” asked Kutner.

Kutner noted that the presentations pointed out the need to focus the field’s resources on

• generating evidence,
• enhancing methodology,
• developing the research workforce,
• enhancing the capabilities of lower-resourced and community-based settings, and
• increasing opportunities for patients and care partners to participate in research and share their expertise.

NIA’s Commitment to Advancing Serious Illness Research

Amy Kelley, deputy director of NIA, noted the importance of always keeping the patient, care partner, family, loved ones, and the community central to all serious illness care research and of considering how serious illness care evolves over the course of the lifespan. She called on the field to overcome some of the silos in the early stages of research and think about synergies across the lifespan, across diseases, and across the diverse populations that serious illness affects.

Kelley commented on the breadth of opportunities featured in the presentations, particularly regarding multicomponent interventions and how to analyze them most effectively. She said that the field is on the cusp of new opportunities arising from the ability to access and use data in many ways and capitalizing on collaborative efforts, such as “team science,”

that leverage expertise from a variety of disciplines. Kelley shared that she was struck by the range of sites that are the focus of research and how the field is embracing the work needed to improve care in each one, and she was excited to hear repeatedly about the critical importance of inclusion to advance health equity and attention to stakeholder and community engagement. She emphasized that it is critical not to perpetuate inequities or misunderstandings and to partner early and bring humility to those relationships, all with the goal of creating an evidence base and practice that positively affects the lives of all people, regardless of the type and context of their serious illness.

Kelley also shared that she is impressed by the momentum that is driving the field and expanding the number of people who are ready to partner and be invited to the table. This is critical for catalyzing the next phase of research focused on palliative care and serious illness care. She noted the large number of NIH institutes that have joined with NIA to collaborate on palliative care research (see Figure 16) prompted in part by congressional language calling for the trans-institute collaboration referenced in Morrison’s earlier presentation.

Kelley called out the range of existing funding opportunities, including those that can complement the work on palliative care and serious illness care. Kelley noted opportunities focused on the early stages of the research

pipeline⁴⁹ and opportunities for K–12 science teachers, undergraduates, postdocs, and medical residents to participate in research. Kelley emphasized that an important funding mechanism is the diversity supplement⁵⁰ attached to current grants that can be used for individuals at any life stage.

Kelley explained that thinking about the growth in the field and the need to bring in more diverse perspectives and diverse life experiences, one approach might be to include high school students in summer research projects, which might encourage them to pursue a career in palliative care, nursing, social work, or community health work, all of which would benefit the field. “I would encourage you all to think about those additional ways we have to leverage the work you are already doing,” said Kelley.

Kelley briefly discussed the new U54 Cooperative Agreement for the Consortium for Palliative Care Research Across the Lifespan and encouraged participants to review the Notice of a Funding Opportunity.⁵¹ The consortium, she explained, will focus on a range of serious illnesses across ages and populations, with a significant focus on building capacity for, facilitating, and supporting research to understand and address health and health care disparities in palliative care. There will be a core that supports training and mentoring activities and another that supports studies informed by the consortium’s research priorities (see Figure 17). Kelley suggested that a good way to keep up-to-date on NIA developments is to subscribe to the NIA blog.⁵²

Caring for the Most Vulnerable Populations

Meier opened her remarks noting that she would focus on the significant threats to caring for the most vulnerable populations, including people with serious and complex illness at all ages and stages and racialized, marginalized, and historically underserved communities. She identified the incursion of the profit motive into U.S. health care as a key threat, one which she characterized as “hiding in plain sight.” She noted that the public

___________________

⁴⁹ For additional information, see https://www.nia.nih.gov/sites/default/files/2023%2005/Training_Career_Dev_portfolio_May_2023.pdf (accessed February 21, 2024).

⁵⁰ For additional information, see https://grants.nih.gov/grants/guide/pa-files/PA-23-189.html (accessed February 21, 2024).

⁵¹ For more information, see https://grants.nih.gov/grants/guide/rfa-files/RFA-AG-25-002.html (accessed February 21, 2024).

⁵² Available at https://www.nia.nih.gov/research/blog (accessed February 21, 2024).

is unaware of the growth of for-profit health care, as are many clinicians, especially those in academic medicine. “What is happening in the real world is quite different from the cultures in which we work, which are almost all nonprofit academic medical centers,” she noted.

Emphasizing that financial incentives are key drivers of behavior, Meier explained that the top concern of many patients and families with serious illness is financial pressure. This is also true for clinicians, both physicians and nurses, who are under tremendous pressure to do what aligns with the financial priorities of their employers and do more in less time, noted Meier. Therefore, any intervention that does not solve those problems is not going to have uptake, according to Meier.

As the population ages, more people are on Medicare, and fewer have higher-paying commercial insurance, Meier explained. The increasing

penetration of Medicare Advantage plans⁵³ and their lower reimbursement rates is making it difficult for hospitals to stay open. “This is why there are so many hospital closures going on around the country and why so many heretofore untouchable large academic medical centers are under major financial stress,” said Meier.

Meier noted that CMS is focused on moving Medicare beneficiaries into managed care plans because fee-for-service is not financially sustainable. She pointed to the danger of this in a totally underfunded oversight and regulatory infrastructure that places no constraints on the profit motive. She referred to a recent exposé in The New Yorker that highlighted the negative impact of private equity ownership of hospices on the quality of care that patients receive (Kofman, 2022). “It is not that CMS does not want to do oversight. They are not adequately funded to do oversight, and that is a political issue that has to do with the electoral college, gerrymandering, and things the serious illness community cannot directly address,” explained Meier.

Palliative care can help by developing quality measures that (1) illustrate the effect of financial constraints and incentives on behavior and outcomes and (2) do not conflate utilization outcomes with quality outcomes. “CMS is under enormous pressure to reduce utilization and, for good or ill, much more focused on cost containment than quality,” emphasized Meier. The available measures such as the Medicare Advantage star ratings, were designed without considering the serious illness population. Meier called for the field to focus on developing quality measures for the highest-risk, most vulnerable populations and ensuring they are incorporated into the star ratings.

One way to do that, said Meier, is by calling attention to the issue, as The New Yorker article did and as colleagues in the field are doing by publishing in top-tier journals on the impact of the profit motive, particularly in for-profit Medicare Advantage plans.⁵⁴ She noted that nonprofit plans do not serve anywhere near as many people as the top five for-profit Medicare Advantage plans. “Our job as researchers is to name this, study it, publish about it, shine a light on it, talk about it, and disseminate those findings to the press and others,” said Meier.

___________________

⁵³ According to the Kaiser Family Foundation, in 2023, 30.8 million people were enrolled in a Medicare Advantage plan, accounting for 51 percent of the eligible Medicare population and $454 billion (or 54%) of total federal Medicare spending (net of premiums). For more information, see https://www.kff.org/medicare/issue-brief/medicare-advantage-in-2023-enrollment-update-and-key-trends/ (accessed January 24, 2024).

⁵⁴ See, for example, Teno and Ankuda (2022).

Another key threat is that for-profit Medicare Advantage plans “over target racialized minorities,” which, cautioned Meier, runs the risk of “amplifying longstanding preexisting inequities.” She underscored that research reveals lower quality in Medicare Advantage plans as well as “much higher rates of admission to the lowest quality hospitals and nursing homes.”

Counterbalancing the incentive to cut costs requires financially incentivizing quality for the seriously ill population in the same way that the Medicare star ratings do for the healthy population. That takes appropriations by Congress, which require advocacy, explained Meier. “CMS cannot just decide to do this because it does not have the money. That money has to be appropriated” by Congress, Meier pointed out.

Developing interventions to address inequities requires identifying gaps in what is known about beneficiary experiences and using mixed methods to understand specifically the ways in which the system affects racialized, marginalized, and underserved populations. “We have to understand what the problem is before we design interventions, and understanding the problem means listening, through qualitative and quantitative research, to populations that we have not studied before,” emphasized Meier.

Addressing racism requires “centering efforts at the margins,” noted Meier, which is something palliative care and serious illness research is doing already by focusing on the small but highest-cost and highest-need population. It also requires having equity as the field’s “North Star.” “We should not be thinking about our research question if we are not thinking about equity at the same time. I think many of us think of ourselves as allies, and of course we support reducing disparities, but our research does not address that,” observed Meier. She called for the field to be held accountable regarding including equity in all research. “If our recruitment is not oversampling and substantially inclusive of previously unincluded populations, we should not get funded,” she argued. As an example, she noted how PCORI was able to drive accountability for engaging the end receivers of care by linking funding to doing so.

Mistrust cannot be used as an explanation for why people do not want to participate in studies, because that mistrust was earned, in Meier’s view. She said that the problem is the research community, so accountability for developing research through an equity lens has to sit with the research community, funders, publishers, payers, and CMS. “These are the things that are a step toward healing what ails us as a society and that manifests the values of end-stage capitalism, which is basically profit drives everything, and government at the moment is incapable of balancing that

priority with the needs of the whole,” said Meier. She acknowledged that government once did a better job of balancing competing priorities, which indicates it is possible and thus raises hope for the future. However, that will require the research community to do the work that will influence how people vote and what they see on social media, read in the paper, or hear on the news.

Panel Discussion

Halpern opened the discussion by asking Meier if she had any ideas on how the serious illness care research community can make a greater impact and develop a core message that can be well received across party lines. Meier replied that research teams should include experts in social marketing. “We have to up our game on the messages that speak to universal human values and desires and human-to-human messaging,” she said. Palliative care needs to get the message across that palliative care teams “are the people that remember that you are a fellow human being, who work alongside our colleagues to focus on your disease or your organ to accompany you through serious illness,” said Meier.

It is essential, said Meier, to communicate more effectively to the public about what serious illness care provides. She and her colleagues once estimated it would take $40–60 million to conduct an effective national marketing campaign to multiple audiences on what people should be demanding from their health systems, their clinicians, and every setting in which they receive care. Although that funding may not be available, the field, in Meier’s view, can at least get its message straight and spend more time communicating it outside of the ivory tower.

Kutner pointed out that part of getting the message across to different audiences is to build that into the beginnings of a research project and the dissemination plan. She noted that Candrian turned a research paper she wrote on the experiences of older LGBTQ women into a movie.⁵⁵ The problem, she said, is that investigators are not trained to think about publicizing their work and getting important messages out to the public. One way to address that is by partnering with patient advocacy groups, which have a great deal of experience with messaging.

___________________

⁵⁵ For more information, see https://www.cpr.org/2023/06/19/just-us-lgbtq-documentary/#:~:text=In%20earlier%20parts%20of%20the,from%2026%20people%2C%20mostly%20Coloradans (accessed January 24, 2024) and https://www.westword.com/arts/carey-candrian-documentary-just-us-queer-community-16870463 (accessed February 15, 2024).

Kutner wondered how to balance talking about the positive aspects of palliative care with discussing the burdens and troubles people with serious illness face. Meier replied that nobody wants to hear about how much suffering there is in the world and how bad end-of-life care is, but that is what the field talks about with the public. “We have to talk about how we help people have a regular, normal life,” said Meier. “That makes it attractive.” Bogetz added that keeping the patient and family front and center and partnering early with the community can help get the message out too. Kutner suggested capturing people’s stories while doing mixed methods research, getting consent to use them, and sharing those stories with the public.

Halpern asked panelists for their thoughts on what the field should be measuring that will help with augmenting messaging and buy-in among diverse communities while also advancing the broad range of health care goals discussed throughout the workshop sessions. Kelley said that measures should be developed based on what matters to key partners and what a meaningful outcome would be for them. The answer may be to measure different things for different audiences.

Bogetz asked what CMS would need in a set of measures that would enable it to change a quality measure, benefit design, or demonstration, and he wondered if NIH-funded studies are generating the necessary evidence. Meier reiterated that financial toxicity is a major source of misery for patients and families, making it imperative to measure financial burden and get the results into local newspapers where elected officials will read it. “By shining a light on the things that are of greatest concern to the people we care about and then making sure our message goes beyond the journal, it starts to shift public opinion, and that starts to shift congressional behavior, especially when combined with partnering with entities like the Parkinson’s Foundation or the American Cancer Society or others,” explained Meier. Getting the message out in a way that influences policy requires partnering with powerful and much larger organizations and coalitions that have the ability and skills to affect congressional behavior, she added.

Bogetz asked Kelley for ideas on how to break down the silos she had mentioned, especially given commonalities across complex chronic conditions, such as recurrent loss, grief, and bereavement. One way to do so, Kelley replied, is to look beyond one’s research and look at what people in other fields are doing that is similar. What is challenging is operationalizing that, particularly given the tenure and promotion system. Influencing the ivory tower, said Meier, will require getting more people in this field to accept institutional leadership roles.

Halpern observed that he is frequently contacted by a new start-up with a new advance care or palliative care project. This raises concerns

about private-equity-funded initiatives crowding out opportunities for real, evidence-based improvements in care, and he asked what the scholarly community can do to combat that trend. Meier said that she did not know whether better partnering with private equity or becoming more entrepreneurial is the best approach. Kelley noted a small business grant mechanism that can fund technology transfer. NIA, for example, leads the Start-Up Challenge, which is designed to train and promote the work of early-career entrepreneurs from a wide array of diverse backgrounds by providing them mentors, coaches, and resources to apply research to the problem they want to solve. Meier noted that entrepreneurs are focusing on addressing the needs of 2.5 percent of patients with long-term, high-cost chronic illnesses, such as dementia, because that is what Medicare Advantage plans care about.

Bogetz asked panelists for their advice on how to advance the idea of creating core or shared resources for the serious illness research community. Kelley replied that the consortium she discussed is one opportunity for doing that. She noted that palliative care is still a small field with the opportunity to engage more people in serious illness research. Kutner identified the need for technical support in methodology, measures, statistics, and even career development paths. Creating community is important for enabling shared learning, she noted.

Kelley pointed out that there is the opportunity for people in the early stages of their careers to think creatively about how they can partner with someone in the business school or experts in behavior change. “Unlike many other fields, where there are concrete paths for a particular research question, this is an open landscape for people to innovate, partner, and grow in whatever direction they want to,” Kelley noted. Speaking as a mentor, Kelley would advise early-career investigators to pick a subject about which they are most excited and passionate. Meier agreed with this advice because it can be motivation to tolerate the challenges they will inevitably face as young investigators and enable them to contribute and feel most successful.

Audience Q&A

Rodgers pointed out that palliative care’s power is in its ability to improve how patients and their care partners experience care throughout their serious illness journey. Rodgers noted that part of improving the care experience is reducing the low-value care that costs money but does not add to quality of life. Given that, he encouraged the research community to focus on the characteristics of palliative care that maximize the reduction of low-value care. The resulting data will bolster the argument to include quality measures that have some teeth into payment programs, he observed.

As someone who is involved in advocating for policies that would benefit serious illness care, Rodgers said what gets policy makers’ attention are the experiences of their constituents. Leading the policy change argument with stories of how palliative care improves those experiences and then making the cost-savings argument has a better chance of success than making the financial argument alone. “If we can align the incentives in the direction that we know we can support with evidence that palliative care makes better care for our patients and their care partners, then our business case will make itself,” said Rodgers.

Kirch pointed out that what sets palliative care apart from other medical specialties is that it normalizes humanity in medicine. The field needs to take advantage of that and let patients tell the story of what it means to be heard and understood, something that other fields do not focus on, she emphasized. Palliative care, for example, considers financial hardship for patients and their care partners, and the field needs to enable them to give voice to that issue. “We can partner with them and think of them as a shared resource that can substitute for the $40 million marketing campaign,” said Kirch, referring to Meier’s statement. Her suggestion to the research community was to let patients tell their stories and then study what it means to be heard and understood.

Kirch called for every specialty to be accountable to the same quality measure against which palliative care is judged, and she noted the new physician fee-for-service rule that will start paying for some aspects of palliative care. She mentioned the role that advocates and patients can play in creating a grassroots movement that can share stories, write opinion pieces, and build the echo chamber to support palliative care. As final comment, she welcomed the research community to think of the Patient Insight Institute as a resource;⁵⁶ it focuses on how the patient experience can inform health care policy. “Use us at every turn to help you find those populations for advising you on how to build a study that matters and that shows this heard and understood piece,” said Kirch.

Aslakson commented that one frustration she experiences as a board member of the Anesthesia Research Foundation is that its grants always seem to go to the same small number of institutions. She wondered what panelists thought about how the new U54 grant will play out in terms of whether it supports the same places that are already funded or if it should

___________________

⁵⁶ Additional information is available at https://www.patientinsightinstitute.org/ (accessed January 24, 2024).

expand beyond the usual recipients. Kelley replied that the concentration of NIH funding is a frequent topic of conversation at NIH and in the research community. Some initiatives exist to address this, most of which aim to build the infrastructure that can help institutions with more limited resources, or that do not currently have much ongoing research, to become more involved in research. The goal, said Kelley, is to bring more diverse perspectives to bear on research. Her suggestion for the palliative care community is for institutions with large, robust research programs to partner with smaller institutions, perhaps by offering pilot grants or career development opportunities, that can grow and bring new perspectives to the field.

Kutner remarked that the field should take it as an obligation to do the best science and uplift those diverse perspectives. For example, when PCRC established its pilot grant award program, it looked specifically for institutions that had not yet been funded by such an award in order to increase the diversity of the field. The Cambia Health Foundation’s Sojourns Scholar Leadership program⁵⁷ has taken the same approach in selecting individuals to receive its awards.

Halpern wondered whether dispersing money to more institutions is truly the most efficient way to promote the shared goal of increasing the diversity of ideas and opportunities for researchers. Instead, he proposed that a hub-and-spoke model—one that brings diverse scholars from multiple institutions to train at institutions that already do high-quality research—would be a more efficient way to achieve that goal. “This is not a crazy or impossible concept as long as we are willing to recognize that Zoom, FaceTime, and the other efficient, technology-enabled ways of collaborating, mentoring, and sharing ideas is a new way of having robust mentorship,” said Halpern. “There is nothing that mucks up the efficiency of getting answers to important questions more than disseminating money to institutions that do not have the infrastructure to use it properly,” argued Halpern.

Arora commented that rigorous, real-world trials take time, with the large pragmatic trials PCORI funds typically taking at least 5 years. As a result, when the research results are ready for large-scale uptake, at least 10–12 years have passed since completing the original research that led to the trial, despite urgency to improve the lives of seriously ill patients.

___________________

⁵⁷ See https://www.cambiahealthfoundation.org/focus-areas/sojourns-scholar-leadership-program.html (accessed January 5, 2024).

He asked panelists for their thoughts on how to resolve this tension. Kutner said that this is a question that she struggles with daily as someone who lives in both hospital executive and researcher roles. For Kutner, the answer is “yes, and.” Yes, it is possible to do rigorous quality improvement work to address current issues and generate evidence to inform the field at the same time, and these two facets need to be linked more clearly. The idea is to learn from rigorous quality improvement work, get those results into the peer-reviewed literature, and improve clinical practice based on emerging evidence. Kelley agreed with Kutner and noted opportunities with existing data and real-world data applications for AI that can power natural experiments that can influence policy.

Bakitas observed that in rural areas, palliative care is not the extra layer of support or designed to be the exclusive layer of support for individuals with serious illness and their care partners. She questioned how the field should develop a model of care. “Should we as palliative care be primary care? Should we be the Medicare Advantage plan for people who have not found a way to have a stable life insurance kind of plan?” she asked. Kutner replied that because of economic and workforce pressures, care in the future probably will look different. “This is where there is opportunity for innovation, as long as we are demonstrating that we are achieving the outcomes we are setting up to achieve,” Kutner noted.

As a final question, Kluger asked the panelists for their ideas on how to create bridges between primary palliative care, surgery palliative care, neurology palliative care, cardiology palliative care, and others without creating silos. In his view, the future of palliative care is going to be powered not by palliative care specialists but by primary palliative care clinicians. Kutner agreed that specialists will always be too few to meet all the serious illness community’s needs. She noted that the overarching goal is to get the right level of care to the right people at the right time, so it is critical to demonstrate the models of care that can ensure that goal is met. Meier added that CAPC has invested tens of millions in an online clinical training model aimed at frontline clinicians, not specialists. “While we can say with certainty that over a million courses have been taken by hundreds of thousands of clinicians, we cannot tell you whether [these courses] had any influence on the care that their patients received,” said Meier. “I am hopeful that primary palliative care models will show benefit.” She noted that PCORI is funding a project comparing a primary palliative care training model to specialty palliative care.⁵⁸

___________________

⁵⁸ For more information, see https://www.pcori.org/research-results/2023/comparative-effectiveness-generalist-versus-specialist-palliative-care-inpatients (accessed January 24, 2024).

Meier recognizes that this will be a significant challenge, made more difficult by the time and performance pressures that specialists are under, the cultures they work in, and the values of those cultures, which are arrayed against the likelihood of success. “The incentives and culture need to change for a cardiologist to think it is a good use of their time to have a half an hour discussion with a patient about what it is like living with heart failure,” said Meier. Producing the needed culture change, she said, will have to start during residency and fellowship training to get highly honed patterns of practice to change. “It is not going to be data-driven,” she added.

Planning for the Future

Noting that “we are at a tipping point in the field,” Kutner pointed to the “tremendous opportunity to codesign, implement, and disseminate rigorous science to meet the priority needs of patients, their care partners, and the communities in which they live.” She referred to Morrison’s remarks, reminding all that the goal is ultimately to improve quality of life for people with serious illness, and emphasized that the field needs to think about the multidimensional components of quality of life and design interventions to incorporate them. Reflecting on the broad range of research priorities in methodology, metrics, and specific populations identified by the speakers, Kutner shared her optimism for the future: “They have given us a road map, and now we need to follow it . . . we have changed the dialogue, and, as an inclusive research community, I am confident that we will affect important change through science.” She emphasized that “patients and their care partners are depending on that happening.”

This page intentionally left blank.