• This Conduct of Research report, NCHRP Web-Only Document 440: Developing a Guide for Incorporating Uncertainty into Long-Range Transportation Planning, including full findings from the literature review, questionnaire, interviews, case studies, agency testing, and peer exchange workshop
• A PowerPoint Presentation with summary and more detailed “how-to” content
• An Implementation Plan, including a two-page fact sheet

This report is organized into three chapters:

• This first chapter introduces the research and its objectives.
• The second chapter summarizes the Phase I research completed in the Task 1 foundational research, Task 2 outreach (questionnaire and focus groups), and Task 3 case studies. Complete findings from these tasks are included in an Appendix, which also incorporates edits in response to panel feedback on prior deliverables.
• The third chapter describes the Phase II research, including the way the research report and toolkit were developed, the testing workshops that allowed the research team to gather feedback on the proposed research report and toolkit from partner agencies, and the high-level findings of the peer exchange between partner agencies after the testing workshops.

Summary of Phase I: Foundational Research, Outreach, and Case Studies

Our work in Phase I of this project provided a basis of knowledge from which we developed the Phase II research products. Specifically, our work in Phase I (a) identified core information and resources that could feed into the development of guidance and toolkit components and (b) identified the most pressing needs and gaps within the state of practice with respect to planning for and managing uncertainty.

Foundational Research

The Task 1 foundational research was organized to address four core areas, each summarized in respective chapters (see Appendix A):

• Sources of Uncertainty contains an overview of a range of types of uncertainty that influences state DOTs and MPOs in their ability to carry out their missions and adhere to their long-range plans or investment programs.
• Uncertainty in Decision-Making provides an overview of decision-making frameworks and processes used historically and at present to understand and manage uncertainty.
• Data and Methods to Assess Uncertainty and Its Impacts addresses data, methods, and tools that are widely available to transportation agencies to address the effects of uncertainty.

• Regulator Context and Requirements describes where transportation agencies, primarily state departments of transportation (DOTs) and metropolitan planning organizations (MPOs), must consider and incorporate uncertainty or risk in their required planning documents and highlights selected opportunities for additional consideration beyond requirements.

Each section of this foundational research provides “ingredients” for the subsequent guidance development, as described in Table 1.

Table 1. Task 1 foundational research components and guidance goals addressed

Outreach

We expanded on the Task 1 foundational research by conducting outreach to state DOTs and MPOs to learn more about the ways they manage uncertainty. The Task 2 scan of practice was comprised of two components:

• A questionnaire designed to scan the landscape of practice for managing uncertainty within state DOTs and MPOs.
• Focus groups and interviews designed to engage with experts from diverse backgrounds outside of the state and regional transportation planning organizations.

Questionnaire

Data Collection.

The research team used Survey Monkey to create the questionnaire to collect information from organizations that may have been involved in developing multimodal long-range transportation and capital investment plans to meet future transportation needs. The questionnaire gathered information on individual organization’s service area(s), type, level of

importance for consideration of each type of uncertainty in future planning, methods and tools used to address uncertainty, and needs and opportunities. We received and analyzed 136 survey responses, representing 48 U.S. states and territories, including Puerto Rico, as well as Canada and other countries. Representatives of State DOTs and MPOs or RPOs comprised the largest group of respondents, with additional responses from other government and non-governmental organizations. The responses included significant coverage of both urban and rural areas.

Planning Processes.

The questionnaire revealed that uncertainty is most frequently addressed in current practice within long-range plans and strategic plans. To a lesser extent, respondents report addressing uncertainty in asset management plans, model plans, and TIPs and STIPs.

Source of Uncertainty.

The top sources of uncertainty respondents report addressing in recent planning efforts are funding, policies and regulations, costs, and natural/environmental hazards. Also considered are mode choice, vehicle electrification, safety technology/behavior, automation, economic change, land use, workforce, other disruptions, location choice, telework. Respondents ranked funding, government regulations and policies, infrastructure and service costs, economic change, and natural/environmental hazards as the most important sources of uncertainty that should be addressed in transportation planning.

Methods and Tools.

The most reported methods to address uncertainty were scenario planning, risk management, and sensitivity analysis. Travel demand models, pavement management systems, and bridge management systems were the three most commonly reported tools used to address uncertainty (Figure 1). These systems are core technical resources regularly employed by State DOTs to manage their assets and plan and prioritize their infrastructure investments. Other tools that are more specific to risks and uncertainty are also employed in some cases.

Challenges and Opportunities.

Respondents reported a range of barriers and challenges faced when planning for uncertainty, many of them associated with resource constraints and complexity. In their responses, individuals expressed a desire for both improved technical resources (tools, data), as well as avenues toward improved knowledge sharing, partnerships, and communications (Figure 2).

Focus Groups and Interviews

Process.

Two focus group sessions were held to gather information on how different industry leaders manage and address uncertainty. One organization was not able to attend the focus groups and opted to participate in a follow-up interview instead. Participants were drawn from a range of industries including automotive manufacturing, TNCs, utilities, research and non-profit organizations, consulting companies, multi-state agencies, freight industry organizations, and airports. The focus groups involved a brief up-front presentation on the research project, followed by a facilitated discussion, including the use of interactive polling technology through Menti.

Findings.

Participants were asked probing, open-ended questions regarding challenges, effective approaches, methodologies, tools, and data based on their knowledge and experience with uncertainty. The discussions can be summarized into the themes in Figure 3. Partnerships, communication, and building flexibility were key areas of emphasis in the discussion.

Case Studies

Objectives

The goal of the case studies was to focus on “deep discovery” within examples of state DOT and MPO approaches to managing uncertainty. Through each case, we sought to understand the impacts of these approaches on decision-making and operations and to identify lessons learned, including gaps in practice. This supplements the broad picture of practice and knowledge developed through the questionnaire and focus groups in the Task 2 outreach described above.

Case Study Overview

Below is a summary of the selected case studies that include examples of state DOTs and MPOs working in a variety or urban and rural contexts, with varying levels of technical complexity.

Table 2. Case Study Summary

Key Findings

The case examples discussed reveal several insights about how agencies plan for and manage uncertainty, including impacts on decision-making and operations, and lessons learned that could benefit other transportation organizations conducting similar efforts in the future. Overall, the individual agency experiences demonstrate that engaging with uncertainty can be a catalyst for unearthing necessary process improvements that were already important even without uncertainty at issue and further amplifying them. Specific insights derived from the cases include:

Planning for uncertainty accentuates existing needs to connect long-range planning to programming.

Long-range plans provide a key opportunity to consider sources of uncertainty in future transportation needs and performance. However, there is more work to be done to connect planning recommendations to project-level decision-making and prioritization.

Alaska DOT, for example, created policy level recommendations from its scenario process. Connecting these insights to investment decisions is still a work-in-progress and part of the department’s overall effort to be more performance- and data-driven in planning.

Ohio DOT did create an Implementation Plan following its scenario based LRTP that grouped actions into four categories: Monitor, Accelerate, Launch, and Defer. For the first three, the agency has designated points of responsibility within the organizations to track and report annually. The agency would like to move towards project-based long-range planning in the future and has found that stakeholders increasingly ask for this. This will require new levels of coordination with Districts and other Divisions within the organization, as well as MPO planning partners. There is no uniform mechanism for allocation of funding and prioritization of resources across the department.

The Hawaii DOT Highway Division effort occurred outside long-range planning, but the resilience tool and associated data are intended to inform upcoming planning as well as specific project-

level design decisions. Implementation will require the planning office to communicate and train about the developed tool and how it can and should support specific decisions by district engineers or others in the agency.

Finally, Hampton Roads TPO, as an MPO, is required to develop a long-range plan that includes a fiscally constrained prioritized project list. Performance of individual projects under multiple scenarios is formally incorporated into the scoring and ranking of projects. Inclusion in the LRTP is then a precondition for funding regionally significant projects within the TIP.

Data management and technical capacity are challenges and key to facilitating and perpetuating institutional learning.

All four case study agencies described challenges related to data management and technical capacity and emphasized the importance of building these capabilities both to support analysis of uncertainty, and to disseminate and share information across an organization and with planning partners. Spatial analytics and data are very important to investigating differential impacts of potential future changes across various geographic areas.

In Alaska, consideration of uncertainty affirmed a need to develop additional capacity for data management. The agency identified the need to manage data consistency, including how data can be shared across departments and updated based on clearly defined data ownership, rather than generating multiple versions of conflicting information.

In considering implementation of its scenario insights, Ohio DOT is very interested in ways to produce, package, and deliver data sets or information to its districts, agency divisions, and planning partners. Rather than creating more burden with additional things to consider, ODOT planning seeks to save others time and effort and alleviate the staffing burden of needing to have experts on-hand covering various drivers of change and sources of uncertainty. The department has a strong record of working to keep modeling tools, data, and procedures consistent across the state and can build on this foundation. ODOT is also beginning to consider the idea of a single database of projects across MPOs/RTPOs. At present, understanding planned investments requires manual review of individual planning documents.

Hawaii DOT’s resilience efforts foregrounded limitations in GIS capacity within the agency, as well as the need to enhance the feedback loop between data collected by staff in the field and information maintained centrally. A key focus of the department is creating shared platforms that can be used at multiple levels within the organization. The department also identified the importance of connecting with outside expertise and other government agencies with specialized knowledge.

HRTPO staff reflected on the significant additional effort required to incorporate scenario planning into the long-range plan and project prioritization. While the agency staff see significant value in being able to stand behind and justify investment decisions, they are looking for ways to reduce effort going forward. They emphasized the importance of exploring uncertainty within the

tools and processes already in place within the agency such as the project scoring process and the travel demand model. HRTPO is actively investing in automation for data processing to limit burden on staff time and working to better align data across departments or within other planning partners.

In all cases, data management is closely tied to staff-related challenges. Agencies often have a small number of planning staff with varying degrees of experience with data and capacity to rapidly analyze and combine multiple data sources. Staff retention also presents issues for knowledge retention and can cause an organization to need to “begin again” if there is insufficient knowledge transfer, training, and documentation of procedures. As such, the case study agencies report a focus both on training and retention efforts and on better systematizing and documenting methodological procedures.

There are multiple outcomes and avenues towards implementation of knowledge gained from considering uncertainty.

Engaging with issues of uncertainty can point to a variety of actions and strategies that can be implemented within agency operations. These include supporting prioritization of projects; generating information to support adaptive design; identifying trends that require tracking; recognizing issues that entail communication and coordination with decision-makers, stakeholders, and planning partners; as well as revealing the need for enhancement of technical and workforce capacity. Because sources of uncertainty are diverse, there is no single outcome that can be expected of a planning effort that engages with uncertainty. For this reason, it is important for agencies to think through how their analysis or planning efforts might impact different parts of the decision-making process or operations, and to target outcomes to their desired areas of impact.

In Alaska, the DOT realized that their regions are key partners for implementation of identified strategies, but that they have varied levels of organization and capacity. As a result, the DOT is engaging with both the development of new regional organizations (including, through DOT funding support), and in capacity building to support on-the-ground implementation.

In Hawaii, the HDOT Highways Division is working to incorporate findings on potential impacts of flooding and extreme weather into the design process. This includes consideration of adaptive design solutions, development of a design checklist, and in some cases the implementation of temporary bridges as interim solutions while ongoing planning efforts grapple with how and where organized retreat from sea level rise may occur.

In Hampton Roads, as described above, uncertainty is directly incorporated into project scoring. In this case, the agency found that presenting multiple metrics across multiple scenarios was too much detail and risked confusion by the public and elected officials. Instead, they opted to present average scores and rankings across scenarios. Other less measurable but equally important outcomes of the effort include the ability to communicate to stakeholders that varying issues and trajectories that might affect the region were addressed in the planning process.

Rather than being held to one forecast, HRTPO saw the value in being able to acknowledge and incorporate various issues that are of concern to their public and elected officials. Additional avenues towards implementation include supporting localities within the region in incorporating scenarios into their comprehensive plans, through provision of data and expertise. Efforts to engage with sea level rise also resulted in ongoing efforts to improve available infrastructure elevation data which is critical to identifying flooding risk.

Ohio DOT leveraged its scenario planning process both to support policy-level dialogue around funding and to identify specific strategies and actions. In addition to the Implementation Plan described above, the DOT modeled funding needs under different scenarios. By bounding the range of needs under a wide variety of future conditions rather than providing a single number that is subject to criticism, the agency was able to engage in conversations more effectively with decision-makers around available funding mechanisms, including issues related to the diminishing value of the gas tax. Based on analysis of emerging technology trends, DOT staff describe the need for ongoing engagement with technology companies such as through the Drive Ohio initiative.1 In some cases, analysis highlighted ways in which the DOT should avoid being “locked in” to a particular technology solution before the market has matured.

Considering uncertainty can broaden the lens of planning and cause an agency to explore new issues, grow their knowledge, and engage with new stakeholders and partners.

As agencies engage with uncertainty, many of them outside the purview of traditional engineering and planning, they often must seek information from new sources. This process can strengthen linkages within an agency and between an agency and their partners, which has the potential to enhance responsiveness of a department to change.

During their LRTP, Alaska DOT identified the need for more focused engagement efforts. Because of the geographic focus of their scenario efforts to identify impacts on different parts of the state, the planning effort highlighted gaps in input, particularly from more rural and remote communities.

Hampton Roads TPO experienced an unanticipated benefit of the scenario planning process: their call for projects received far more public interest and input that in prior planning cycles. Staff also reflected on how scenario planning forced increased coordination and dialogue between different parts of regional government focused on transportation, land use, economics, resilience, and water resources.

Beyond identification of discrete actions, Ohio DOT staff identified considerable value in the conversations that occurred as part of the scenario planning process. ODOT placed a heavy emphasis on stakeholder engagement, including internal engagement within the DOT, with the steering committee, and with districts and regions. By initiating conversations around uncertainty,

___________________

¹ https://drive.ohio.gov/home