CHAPTER 2

Research Approach and Findings

This chapter will describe Phase I of this project, which included gathering data on each Snapshot topic, assessing that data for application within Snapshots, and developing a consistent design and content template for each of the four Snapshot products delivered under this effort.

There were several iterative phases of content generation as part of this phase. Initial work produced a synthesis of practice documentation, which functioned similar to a review of literature. Following this synthesis, researchers engaged practitioners and stakeholders to determine the leading practices and how to develop a product that would be useful to the target audience. Finally, this information was used to produce proposed Snapshot content, which highlighted key examples of the practice.

Snapshot Literature Review

The purpose of the literature review was to gather, assess, and document existing research and resources on each Snapshot topic and to identify core needs, gaps, and best practices for further development. The literature review focused first on state DOT, MPO, and city and county research and best practices. As needed, the research team also incorporated resources and research from a variety of nontraditional practitioners. This review examined topical, timely research that could inform the practitioners to engage and the topics to cover in practitioner interviews.

For each Snapshot topic, the literature review presented the following information:

• Key Practitioner Questions: To determine what practitioners might be most interested in learning about through each Snapshot, the research team used online tools, Google Bard and Google Trends, to summarize search engine inquiries related to each topic area. This information provided information on potential focus areas or sections for later Snapshot content.
• Topic in Practice: To begin developing a list of potential agencies for outreach, this part of the literature review catalogued agencies implementing emerging practices in the field. The research team identified several potential case studies for further investigation and research. Case studies were identified based on:
  • Achieving a mix of state, regional, and local agencies operating in rural and urban communities, and
  • Achieving a mix of agencies of different relative sizes, program maturity, and funding and resource constraints.
• Topic in Research: To leverage current research on Snapshot topics, current research is summarized on each topic that informed subsequent tasks.
• Next Steps: Recommendations are summarized for practitioner engagement based on findings of the literature review.

After the literature review was completed, the research team requested feedback from the project panel to clarify the focus of each topic area and to recommend potential agencies or programs for engagement.

State of the Practice Assessment

The practitioner engagement undertaken as part of this project was designed to gain a better understanding of leading practices related to each Snapshot topic and to gather feedback on the content and design of the Snapshots. To reduce survey fatigue, engagement was focused on virtual fact-finding and direct conversations.

The project team engaged practitioners through a multifaceted approach that included focus groups, interviews, and outreach to committees and industry organizations. The first section of this summary is organized according to type of outreach conducted, which included the following:

• Committee and Industry Organization Outreach: Presenting to AASHTO groups and soliciting input on interviewees and thematic areas of focus. Interviewed representatives from the National Association of City Transportation Officials (NACTO) and the Association of Metropolitan Planning Organizations (AMPO) on all topic areas and gathered ideas for potential interviewees.
• Focus Groups: Convening two focus groups with practitioners and facilitating breakout groups organized around Snapshot topic area. The focus group invitation and interview guide can be found in Appendix A and Appendix B.
• Interviews: Conducting targeted conversations with key agencies in each topic area to gather input on leading practices as well as Snapshot form. The discussion guides for interviews can be found in Appendix C and Appendix D.

Committee and Industry Organization Outreach

The research team delivered a presentation on the project in two AASHTO forums. The team also met with representatives from AMPO and NACTO to identify leading organizations and practices for each Snapshot topic area. Table 1 summarizes that outreach.

Table 1. Summary of committee and industry organization engagement.

Focus Groups

To gather insights from industry leaders and practitioners, the research team organized two focus groups. Practitioners identified in the literature review but not selected for interviews were invited to these focus groups. Additionally, the research team engaged with applicable committee leaders from TRB and AASHTO to distribute invitations. The invitation included a registration survey that participants could use to indicate the Snapshot topic they were most interested in discussing. This information guided the creation of breakout groups, which allowed for more focused conversations on each topic. The invitation for the focus groups and the discussion guide used to support the breakout groups can be found in Appendix A and Appendix B, respectively.

The focus group meetings began with a brief presentation and overview of the research topics. Participants were then split into two breakout groups based on topic. In total, the research team contacted 72 individuals or agencies for participation, resulting in two focus groups with 25 participants in total. Table 2 summarizes the organizations represented at each meeting.

Key Focus Group Takeaways

The following subsections summarize takeaways from the focus groups by topic area. The only topic not discussed explicitly during the focus groups was economic analysis. The research team found that few participants were interested in this topic on its own, but many were interested in its relation to other topics (e.g., economic measures as they relate to Complete Streets or freight movement).

Complete Streets Implementation

Participants highlighted key challenges related to Complete Streets implementation and examples of leading practices. Implementation challenges occur when:

• There is a state road that is designed for higher-speed traffic but also has destinations that need to be served by transit. This is also a safety concern.
• There is a need to justify Complete Streets’ economic impact because of the ongoing maintenance implications of these projects.

Table 2. Focus group dates and attendees.

• There is a need to create bike lanes that are safe and appealing to cyclists and accommodating to people with disabilities.

One participant noted that transportation professionals can build relationships with local health partners to talk about Complete Streets in terms of public health.

Leading Complete Streets practices identified during the focus groups include those detailed in the Minnesota DOT (MnDOT) Complete Streets guide (MnDOT n.d.-a) and Complete Streets Transportation Hierarchy Tool (MnDOT n.d.-b), the Washington State DOT (WSDOT) legislative Complete Streets mandate (WSDOT 2023) and Washington State Complete Streets statute (Washington State Legislature 2022), and the Florida DOT (FDOT) Context Classification Guide (FDOT 2022).

Data Sharing for Performance Management

Participants noted that mapping tools should be used as a central repository for multiple data sources to create one accurate source. Examples of mapping tools and initiatives highlighted by participants include the Sun Cloud Explorer [Maricopa Association of Governments (MAG) n.d.], the Virginia freight plan interactive map (VTrans n.d.), the Philly Freight Finder [Delaware Valley Regional Planning Commission (DVRPC) n.d.], and the Atlanta Regional Commission Freight Cluster Plans (Atlanta Regional Commission n.d.).

In terms of data collection and sharing, participants mentioned data sources such as individual stakeholders (e.g., rail carriers) but highlighted the need for data from the private sector, businesses, and local stakeholders. Participants also expressed a need for data sharing agreement templates, resources to determine estimates and impacts of freight projects, and data-sharing and analysis performance methods. For successful partnerships, DOTs must be able to easily share data with MPOs and other entities.

Collaboration on Local Freight Delivery

Practitioners focused on leading practices in the freight industry and on the overlap of freight delivery with data sharing and Complete Streets. Practitioners feel they have a limited view of freight industry activity. Data resolution is not currently fine enough to assess local delivery, and obtaining better data will likely require enhanced stakeholder involvement. Key stakeholders to engage include carriers like United Parcel Service (UPS), FedEx, Amazon, less-than-truckload carriers, and utilities (power companies, plumbers, etc.) with large vehicles.

A participant from the Port of Oakland marine terminals explained how the port presents an end-to-end use for customers by tracking freight metrics through staff input and cameras (Port of Oakland n.d.). Similarly, the city of Portland’s newly updated freight plan is unique in that it looks at measures for curb usage and availability, especially in the central commercial district (Portland Bureau of Transportation 2023).

Practitioners also discussed the overlap between Complete Streets and freight delivery. One participant noted that their agency reaches out to developers and stakeholders in walkable urban areas to ask how they are considering freight movement.

Interviewee Selection

The literature review informed the selection of an initial set of interviewees. The research team’s goal was to speak with two to three individuals in each audience category (e.g., state DOT, MPO, city) for each topic. These interviewees were identified based on a few characteristics:

• Association with innovative practices identified in the literature review
• Feedback from subject-matter experts and the project panel
• AASHTO and TRB committee outreach

During the interviews, participants offered further recommendations for agencies they considered leaders in one of the four topic areas. This resulted in additional interviews being conducted for each topic area. The initial list of interviewee recommendations and interviews that were completed can be found in Table 3. Additional interviewees were identified by people interviewed during the first round or from the project panel. A prioritized list of interviewees by Snapshot topic as they were presented to the panel is summarized by topic in Appendix E.

Practitioner Interview Takeaways

Initially, the research team determined an interviewee list from the results of the literature review for each Snapshot topic. The initial round of engagement resulted in eight agencies identified for interviews and seven interviews scheduled. After the initial round of engagement, additional leading agencies were identified through past practitioner interviews and through focus group feedback. As a result, the research team scheduled several additional conversations to gather more information. Each of the practitioner interviews by topic area is listed in Table 3.

Complete Streets

The research team conducted interviews with representatives of Arlington County, Virginia, on March 19, 2024, the Metropolitan Transportation Commission (MTC) on April 9, 2024,

Table 3. Initial interviewee recommendations and completed interviews.

Kansas DOT on May 3, 2024, and Michigan DOT on June 6, 2024. Table 4 summarizes findings from these conversations.

Data Sharing for Performance Management

The research team conducted an interview with representatives of Utah DOT on March 28, 2024, MnDOT on May 7, 2024, Michigan DOT on June 6, 2024, and DVRPC on June 7, 2024. Table 5 summarizes findings from these conversations.

Table 4. Complete Streets—interview summaries and findings.

Table 5. Data sharing for performance management—interview summaries and findings.

Collaboration on Local Freight Delivery

The research team interviewed planners from MAG on March 20, 2024, New York City DOT on April 9, 2024, Michigan DOT on June 6, 2024, and Seattle DOT/Urban Freight Lab on August 22, 2024. Table 6 summarizes findings from these conversations. The team also interviewed Virginia DOT on April 24, 2024, but found that there was not substantial overlap between the Snapshot topic and Virginia DOT’s activities.

Programmatic Economic Measures to Evaluate and Prioritize Transportation Projects and Investments

The research team met with representatives from Ohio DOT on March 20, 2024, Broward MPO on March 25, 2024, and Michigan DOT on June 6, 2024. Table 7 summarizes findings from these conversations.

Table 6. Collaboration on local freight delivery—interview summaries and findings.

Table 7. Programmatic economic measures to evaluate and prioritize transportation projects and investments—interview summaries and findings.

Overall Engagement Findings

Information collected through practitioner interviews and focus group discussions yielded several key conclusions regarding Snapshot content and design:

• Practitioners prefer simple, visual content with external links: Practitioners want resources that summarize topics at a high level in a visually engaging but simple manner. Practitioners likened the Snapshots to the function of executive summaries in reports; those summaries are easily digestible and give readers enough information to be informed but not overwhelmed. Practitioners also noted that they wanted these simple visual products to offer readers the opportunity to learn more; links to external content, raw data, maps, and other supporting documentation are appreciated.
• Practitioners want resources that offer solutions for different types of agencies: There is a desire to see a range of options highlighted in the Snapshots so that any agency—regardless of urban or rural context, agency resources and capacity, or other factors—can engage in some part of the Snapshot’s findings. It was suggested that different Snapshots be developed for different audiences. At the very least, they should convey the extent to which more cutting-edge practices could be implemented on a smaller scale in another state, city, or region.
• Practitioners want help sifting through resources and guidance: Practitioners noted that there is already a great deal of guidance available on all the Snapshot topics. What would be most helpful would be to create a repository of the more authoritative sources of guidance rather than a new guidance document itself. This approach could be described as a “curated list of resources.”
• A key feature of dissemination should be webinars and training: Practitioners often attend webinars and training, and these would be useful forums for distributing information on the Snapshots. The American Planning Association, various industry committees and groups, and FHWA community planners might also be other resources for getting the word out.

Based on these takeaways, the research team developed a set of Snapshots that offer building blocks along a spectrum of resources, practices, and leading examples. The organizing principle for this spectrum was a maturity model to describe the evolution of a person or group over time, mapping capabilities to different maturity levels. The interviews and focus groups provided many different practice examples that could have been used for the case studies. The final case studies were chosen by selecting practices or programs that showed exemplary practices across a range of agency scales, maturity levels, and program designs. This method allowed the Snapshots to present a diversity of case studies with the goal of allowing users to see an agency similar to theirs reflected in the case studies.

Analysis of Sample Formats

In a memorandum, the research team investigated characteristics and design options related to the maturity model and building-block concepts. First, the research team examined existing approaches for developing maturity models. Maturity models generally describe the evolution of an organization over time, mapping capabilities to different maturity levels. This framework provides a tool for assessing current capabilities and understanding how a person or group can move to a subsequent maturity level. Originally developed to improve software development processes, different maturity models are now used across a range of industries and organization types. The following list identifies several maturity models that are commonly used today:

• Capability Maturity Model (CMM): The CMM is a process of identifying the current condition and how to move the product to the next specified level. The CMM grew out of the Software Engineering Institute and was developed to assess process maturity levels in software.

• Capability Maturity Model Integration (CMMI): CMMI was an effort out of the Software Engineering Institute to improve on weaknesses of the CMM model. CMMI considers process maturity more broadly and adopts a results-oriented approach toward key performance areas. CMMI is usable by industries besides the software industry.
• Business Process Maturity Model (BPMM): The BPMM framework further builds on the CMMI model. Specific to BPMM are Domain Process Areas, which describe the types of new measures that should be taken to lift the organization to the next maturity level.
• People Capability Maturity Model (PCMM): This model aims to assess the maturity of an organization’s workforce practices to integrate workforce development into efforts around process improvement.

The research team then examined approaches within the transportation field and beyond toward placing organizations along a de facto maturity spectrum but with language and information that focuses more on ongoing process improvement than on fixed levels of maturity. Other research products and guidance have adopted a range of approaches toward helping agencies locate themselves on a maturity spectrum and for representing such a spectrum to audiences:

• NCHRP WebResource 1: Reducing Greenhouse Gas Emissions: A Guide for State DOTs (NCHRP Project 25-56): This guide provides guidance on practical and innovative methods for state DOTs to assess and advance emission reducing strategies in their planning and programming. After finding that DOTs struggled to find appropriate methods and programming decisions that match their needs and decision-making context, the research team developed a series of “self-assessments” to assist staff in determining where their agency falls on a variety of spectrums associated with greenhouse gas engagement. Level of DOT engagement was ranked generically from 1 through 4 (Cambridge Systematics, Inc., et al. 2022).
• Data Visualization Methods for Transportation Agencies (Higgins et al. 2016a) and NCHRP Web-Only Document 226: Data Visualization Methods for Transportation Agencies (NCHRP Project 08-36/Task 128 VizGuide; Higgens et al. 2016b): The VizGuide is intended as a resource for transportation professionals who want to use illustrations and visualizations to communicate their ideas to an audience. Derived from NCHRP Project 08-36/Task 128, “Data Visualization Methods for Transportation Agencies,” the site breaks content into six sections: Examples, Process, Charts, Style, Tools, and Resources (Higgins et al. 2016a). The Examples section offers short case studies and details where users can replicate some functionality with varying technical skills or software access. For example, a case study in one tool notes that “an intermediate knowledge of Microsoft Excel or Adobe Illustrator is necessary to replicate the dial, but a choropleth can be created easily in many visualization programs, including Tableau, Microsoft PowerBI, ArcMap, and QGIS” (Higgins et al. 2016a).
• The Analytics Playbook for Cities: A Navigational Tool for Understanding Data Analytics in Local Government, Confronting Trade-Offs, and Implementing Effectively: This report from the Harvard Ash Center is aimed at practitioners who are thinking about choosing to name their first Chief Data Officer, start their first analytics team, or empower an existing group of individuals. The report is offered online in PDF form. Though the format provides limited functionality, it gives an example of integrating examples and external content into an otherwise static document. The report’s body offers sidebars with examples from cities, while an Appendix Expands on these case studies. Footnotes and body text provide clickable hyperlinks, connecting users to additional resources (Mashariki, Diaz, and Eaves 2020).
• Cyber Essentials Toolkit: This resource from the Cybersecurity and Infrastructure Security Agency (CISA) is a set of modules designed to break down the CISA cyber essentials into bite-sized actions for information technology (IT) and C-suite leadership to work toward full implementation of each cyber essential. Each chapter is an individual PDF that highlights key actions. Each key action includes a list of resources for taking action, which link to external resources. All actions are identified either as actions for leaders or discussions with IT staff or with service providers; the guide’s simplicity makes the takeaways and responsibilities clear (CISA 2020).

Finally, the research team looked for visual examples of a maturity spectrum that would best fit the Snapshot format. Some key findings included:

• Strategic Highway Research Program Capability Maturity Framework: This framework provides an example of a transportation-related maturity model developed for the Strategic Highway Research Program. The maturity framework is process-driven and provides self-assessments to identify position along the maturity spectrum. It then provides six process improvement dimensions: business process, systems and technology, performance measurement, workforce, culture, and collaboration, as well as specific actions necessary to progress through four capability levels that range from “Ad-Hoc, Low Level of Capability” to “Optimized, Highest Level of Capability.” This model integrates self-assessments for each process improvement dimension (FHWA 2016).
• Smart Cities Readiness Maturity Model: This model provides five maturity levels and the various dimensions that a city must address/improve to mature along the spectrum. It provides details on different dimensions of Smart City programs, such as strategic intent, data, and stakeholder engagement, and what needs to be done to mature along the spectrum. It features five maturity levels, ranging from “Ad-Hoc” to “Optimized,” and provides an overview of how a level’s approach effects outcomes. This visualization provides a thorough and detailed resource for cities to understand where they fall along the spectrum and what they need to do to advance their programming (Scottish Cities Alliance, the Scottish Government, and Urban Tide 2014).

Panel Feedback

On May 2, 2024, the research team presented initial outreach findings and Snapshot design concepts to the NCHRP Project Panel. Panel feedback and research team responses to that feedback are summarized here:

• The research team should be aiming for convergence: The panel noted that the research team will know when outreach is complete when the team starts hearing the same things repeatedly. Given that this had not occurred at that time, it was clear that additional organizations should be contacted for interviews. Panel members recommended several organizations for additional outreach, including the following:
  • Complete Streets: Florida DOT, Nebraska DOT
  • Data sharing for performance management: Connecticut DOT
  • Collaboration on local freight delivery: New Jersey MPO
  • Economic analysis to support decision-making: Michigan DOT, Nebraska DOT

  The research team subsequently contacted those organizations for interviews, and was able to schedule an interview with Michigan DOT. The interview guide also changed based on the approval of the maturity model framework. The revised Phase I Interview Guide can be found in Appendix D.

• The maturity model and compilation of resources approach makes sense: The panel affirmed that the maturity model approach would allow the Snapshots to supply useful information to different types of organizations. It was noted that many agencies that struggle to move to implementation may remain at the initial stage for a long time; it is useful to provide resources and examples for those agencies as they establish the groundwork for future implementation. Further, the panel agreed that compiling different resources for organizations along the spectrum would be useful.
• Certain Snapshots need to incorporate additional information: While reviewing the draft interview and focus group findings, the project panel identified the need for additional information. For example, Complete Streets should include more information on public engagement, while the data sharing for performance management topic should include more

• information on data-sharing agreements and platforms. The panel was also interested in learning more about where gaps remain, even for organizations identified as leading on the maturity spectrum. Where specific gaps were identified, the research team incorporated additional content into the Snapshots. The research team has documented in this report the findings related to the remaining gaps.
• There are certain topics that should be highlighted across all Snapshots: The research team noted that data sharing had emerged as an issue in all Snapshot topics. The panel agreed that this topic should remain an independent Snapshot but that it should be discussed as a distinct component of each of the other Snapshots. The research team responded to this by developing a data subsection (Data and Performance Management) for the practices summarized in each of the final Snapshot designs.