5
Evaluating and Communicating Impact

Colleges and universities want to be able to document and share the value of their partnerships and collaborative activities. Determining how to measure the impact of their engagement at the individual, project, program, organizational, and systems levels is a foundational step in the process of evaluating and communicating public value. However, measuring and communicating impacts is increasingly difficult at the highest levels of complexity, and there is no standard practice for making clear, comparative assessments of their value. Identifying accurate qualitative, if not quantitative, indicators of impact from collaborative partnerships is an active subject of creative scholarship and innovation (van Tulder et al., 2016).

MEASURING AND COMMUNICATING IMPACT

In a panel presentation, Lina Dostilio, vice chancellor for external relations at the University of Pittsburgh, posited that communicating impact is both a technical task and a political function to bring clarity to complexity: “We have to be rigorous without being reductive, and we must be able to tell the stories that resonate with both our communities and our governing stakeholders.” She added that it is critical to recognize that measuring impact is not the same thing as measuring effort: “Policymakers and communities are less interested in what we did and more interested in what changed and what became possible when we showed up.” Another way to characterize impact is to articulate the counterfactual—that is, what would not have happened if the collaboration had not happened. Antonio McLaren, vice president of programs at the 1890 Universities Foundation, emphasized the importance of being able to tell the story of collaborative impacts with appropriate highlights and data points in a way that is personalized to match the receiver of the story (such as a policymaker); he said telling these stories is pivotal to successful advocacy for engagement.

When assessing and communicating the impacts that are relevant to these different interested parties, it is critical to understand how the information will be used and what is going to be actionable. Susana Campbell, provost associate professor at American University, noted that it is useless to have more data “without a clear decision-making structure that [the data are] going into. . . .We always need to think about, what am I measuring? What am I capturing? How are we going to use it?” As such, communicating compelling measures of impact will involve a sustained, iterative approach that evolves alongside all interested parties and is embedded within ongoing collaborations.

A holistic story to enable learning includes indicators related to the purpose and process of a community partnership or collaborative activity, together with indicators of community impact. The purpose of a collaboration might be indicated by public statements reinforcing the public intent of the partnership and the institutional incentives and supports put in place to enhance capacity for collaboration. Indicators of the process could include aspects such as the level of involvement of community members, participation by students and faculty, and the activities that were carried out. The scale of the desired impact is another important aspect

for assessment that relates to the scope of a proposed collaboration, and to the number and breadth of partners involved. Finally, the story must attempt to identify measures of impact attributable to the collaboration, which may be more difficult to pin down, so evaluators need to consider what different impacts might be anticipated at different stages of the collaboration (Wanjiru and Xiaoguang, 2021).

USING VIGOROUS RECIPROCITY TO DEFINE WHAT MATTERS IN COLLABORATIONS

Principles of “vigorous reciprocity” can help guide what to measure in each type of collaboration, focusing on what is important from each invested partner’s perspective.

Community partners are in the best position to describe their observation of the impacts of the partnership, even if indicators were not anticipated prior to the collaboration, because they can be incorporated in the future. Dostilio emphasized the need to ensure that higher education partners do not unilaterally determine the metrics for measuring impacts. She stressed that public partners must be actively involved to ensure the selected measures are relevant, to prevent “missing what matters most and what resonates to our community and public audiences.”

At times, a university partner may lack of familiarity with the community for which it is trying to create value. McLaren observed this in partnerships between an 1862 university and a community served by an 1890 land-grant institution; he said this lack of familiarity impeded the ability to find metrics of success to both university and public audiences. This underscores the critical importance of building relationships and establishing mutual engagement first before beginning collaborative projects.

Measurable, actionable outcomes are different at the level of projects, which are completed on a shorter time frame and typically have well-defined goals that change little. On the other hand, institution-level collaborations have a much longer time frame and may evolve as multiple efforts build on one another. An institution’s capability to effectively measure and communicate impact can be limited by resource constraints, time constraints, changes in the political landscape, a lack of personnel with appropriate training in communications, and a tendency to stick with established communication approaches rather than critically evaluating what approaches are best suited to the audiences, messages, and goals at hand.

CAPTURING THE PUBLIC VALUE OF THE LAND-GRANT SYSTEM

Measuring the collective impact of an institution’s engagement activities is an important way to demonstrate clearly and transparently the accumulative effects of a university’s—and by extension, the land-grant system’s public value. Sharon Paynter, chief innovation and engagement officer and interim chief research officer at East Carolina University, emphasized ongoing discussions around the concepts of “value, trust, and return on investment [ROI].” As a whole, Generation Z feels higher education is important but may be unaffordable.¹ In addition to affordability, discussions have expanded to issues regarding quality, curriculum, and infrastructure. Students engage with their universities as a mechanism

___________________

¹ See https://www.deloitte.com/global/en/issues/work/genz-millennial-survey.html (accessed August 3, 2025).

for increasing their prospects for “finding a job, being satisfied with their careers, and increase their economic security,” Paynter added. She highlighted the work being done in North Carolina. The University of North Carolina (UNC) System² commissioned a study to assess the ROI at their institutions from three perspectives: individual institutions, students, and the state (see Box 5-1). The results showed that students pursuing bachelor’s and graduate degrees in the UNC System will have higher lifetime earnings than nondegree holders. Furthermore, the study found that many of their degree programs aligned with critical workforce needs in the state. It also identified areas where similar programs across the system showed different ROIs. The dashboards generated by the study serve as valuable tools for the UNC System, students, and state, to support informed decision making on, for example, educational programs, funding, economic mobility, and workforce alignment.

The need for measurements of impact is not only of utility to the parties invested in a particular land-grant institution and its community partnerships. The tools, models, and concepts that are being developed by scholars and practitioners of evaluation have potentially broader use for funding agencies as they try to understand the impact of their grant portfolios.

___________________

² See https://www.northcarolina.edu/institutions/ (accessed August 3, 2025).