Appendix B
Planning for Uncertainty – Outreach Summary Memo

Contents

1. Introduction

2. Questionnaire of Practice

2.1. Overview of Questionnaire

2.2. Profile of Respondents

2.3. Types of Uncertainty

2.4. Methods & Tools to Address Uncertainty

2.5. Barriers and Resources

3. Focus Groups

3.1. Overview and Participation

3.2. Areas of Uncertainty

3.3. Focus Group Themes

1. Introduction

This appendix summarizes the findings of the component of the Task 2 Scan of practice. Chapter 2 contains findings from the questionnaire of practice, which was designed to provide a broad scan to identify the landscape of practice for managing uncertainty within state DOTs and MPOs. Chapter 3 summarizes findings from the focus groups and interviews that were designed to engage with experts from diverse backgrounds outside of state and regional transportation planning organizations.

2. Questionnaire of Practice

2.1. Overview of Questionnaire

The research team used Survey Monkey to create the questionnaire entitled “NCHRP Uncertainty in Long-Range Transportation Planning” to compile information from organizations that may have been involved in developing multimodal long-range transportation and capital investment plans to meet future transportation needs. The questionnaire gathered information on individual organization’s service area(s), type, level of importance for consideration of each type of uncertainty in future planning, methods and tools used to address uncertainty, and needs and opportunities.

To distribute the questionnaire, the research team solicited the assistance of the several TRB Committees to send the questionnaire link to their contacts and distribution lists. The Survey Monkey questionnaire accepted responses between February 16, 2023, and March 20, 2023. In the time it remained open, the questionnaire collected 141 responses. Out of these responses, we excluded five responses as they were either incomplete or did not provide accurate information.

Questionnaire Distribution Channels

The following is a list of organizations that assisted the research team in the distribution of the questionnaire. Contacts from these organizations typically shared a link to the questionnaire via email to their distribution lists. The questionnaire was also shared via team members social media (LinkedIn). The research team is very appreciative of the support provided by these organizations and recognizes that the questionnaire would not have had the success it did without their efforts.

Table 1. Distribution Channels for Questionnaire

2.2. Profile of Respondents

The questionnaire respondents cover 48 U.S. states and territories, including Puerto Rico. The states most represented are Massachusetts (7), North Carolina (6), California (6), and Texas (6). The questionnaire also drew international attention collecting responses from Canada (9) and other countries (4) as well. To further understand the geographic distribution, responses were grouped by economic regions defined by Bureau of Economic Analysis (BEA):

• Far West: Alaska, California, Hawaii, Oregon, Nevada, Washington
• Southeast: Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, West Virginia
• Southwest: Arizona, New Mexico, Oklahoma, Texas
• New England: Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
• Plains: Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, South Dakota
• Great Lakes: Illinois, Indiana, Michigan, Ohio, Wisconsin
• Rocky Mountain: Colorado, Idaho, Montana, Utah, Wyoming
• Mideast: Delaware, District of Columbia, Maryland, New Jersey, New York, Pennsylvania
• Other: Puerto Rico¹

Because this was a voluntary questionnaire designed primarily to collect examples and insights from respondents who had specific experience with uncertainty and resilience planning, the profile of respondents is not intended as a representative sample. Note that economic regions may be over- or under-represented in our questionnaire respondents due to the success or shortcomings of the questionnaire distribution process. One quarter of the United States

___________________

¹Note: Puerto Rico is a territory and not within the BEA defined economic regions.

respondents were from the Southeast region, followed by Mideast (18 percent) and then Southwest (12 percent, Figure 1).

The survey welcomed respondents with different backgrounds and professional positions including researchers, planners and decision makers, as illustrated in the word cloud in Figure 2.

The questionnaire respondents represent several types of agencies, ranging from transit organizations to government agencies, academia, research institutions, and private sector organizations. Representatives of DOTs constituted most of the questionnaire respondents (73), making up 54 percent of all who answered the questionnaire (Figure 3). Respondents from other organizations (28) account for 21 percent of all questionnaire respondents, while representatives from MPOs or RPOs (20) and from local government (5) account for 15 and 4 percent of questionnaire respondents respectively (Figure 3).

Respondents were asked to identify the type(s) of areas that best describe the communities in which their organizations operate. Respondents had the option to select multiple service area types and write-in other responses. Of all responses, 79 percent (108 respondents) indicated that their service area includes urban areas (Figure 4). 62 percent of respondents (84) indicated that their organization’s service includes rural areas, while 23 percent noted their organization includes tribal areas. Under the other category, some respondents wrote that they operate in suburban and statewide service areas.

Note: Numbers do not add to the total as questionnaire respondents were able to select multiple options.

2.3. Types of Uncertainty

The questionnaire respondents represent organizations that face different types of uncertainty across broad themes including technology and behavior, policy and regulation, context and environment, and overall agency capabilities. Respondents were asked to indicate the level of

importance for consideration for each type of uncertainty in future planning. The categories provided in the questionnaire with accompanying descriptions and are as follows:

Technology and Behavior:

• Vehicle automation and related technologies: This includes driver assistance as well as the emergence of connected and autonomous vehicle technologies.
• Household and firm location choice: Location preferences are subject to change—for example, from urban to rural or from offshoring to nearshoring strategies. These uncertainties may reflect generational or other demographics shifts, changing priorities, and alterations of industry technology and operational parameters among others.
• Mode choice: Modal preferences and choices can shift unexpectedly over time in ways that are shaped by cultural norms, demographics, and generational attitudes, among others.
• E-Commerce: The evolving dynamics of how goods and services are sold online are still taking shape and are affecting patterns of land use and travel.
• Telework: Rapidly accelerated during the pandemic, the shift towards full or part time telework has not reached steady state and introduces a range of uncertainties in future travel patterns.
• Vehicle electrification: The motor vehicle market is undergoing a rapid transition from fossil-fueled Internal Combustion Engine (ICE) vehicles to Zero Emission Vehicles (ZEVs). The timing and nature of this transition remain uncertain, with implications for infrastructure and planning.
• Changing safety technology and behavior: New transportation technologies present unique challenges as well as solutions to road safety issues. Human behavior is also subject to shifts, such as the sudden uptick in crash rates in 2020. Uncertainty can make it hard to anticipate and manage safety performance.

Policy and Regulations

• Funding: Funding available for transportation is subject to uncertainty due to the dynamics of the political process, as well as variations in other sources such as the fuel tax and other fees.
• Government regulations and policies: Government regulations and policies change over time and can result in shifting organizational priorities and requirements for transportation agencies, as well as new limitation incentives on the transportation industry.

Context and Environment

• Economic change: The pace and nature of economic change is dynamic and subject to much forecasting effort but remains uncertain.
• Infrastructure and service costs: The cost of building, maintaining, and operating transportation infrastructure and services is subject to external factors such as shifts in specific supply and labor markets, and market dynamics like inflation.

• Land use patterns, regulations, and constraints: Land use patterns influence travel patterns by affecting the number and types of trips made (trip generation), trip origins and destinations, and mode choice. Land use patterns evolve over time and carry with them a degree of uncertainty.
• Climate/natural/environmental hazards: The increasing frequency and intensity of extreme weather events, along with sea level rise, pandemic disease, and other natural disasters create an evolving landscape of risks and system stressors.
• Other system disruptions: Cyber-security threats, supply chain interruptions, and other disruptive events contribute to stress on transportation systems for transportation planners to navigate.

Agency Capabilities

• Workforce: The ability to attract, train, and retain the necessary workforce in transportation to meet future needs is itself a source of uncertainty, as generations transition and industries compete for talent.

Figure 5 presents the distribution of importance indicated by respondents for each type of uncertainty. The chart is ordered by the number of respondents that indicated a given source of uncertainty was “very important.” Based on selection of “very important,” funding, infrastructure and service costs, and government regulations and policies were identified by respondents as the top three most important. When considering sources selected as either “very important” or “somewhat important,” economic change and climate/natural/environmental hazards emerge as the highest ranked. As can be seen from the chat, more than 70 percent of respondents indicated that every listed source of uncertainty was either “somewhat important” or “very important.”

To better understand the application of each type of uncertainty, respondents were asked to identify specific planning or business processes where each type was incorporated in the last five years. The planning and business processes identified included Long-Range Plans, Asset Management Plans, Modal Plans, TIP or STIPs, Strategic Plans and Other.

Among all the responses, Long Range Plans were the most selected planning process followed by Strategic Plans (Figure 6). Within Long Range Planning, most of the planning has focused on climate/natural and environmental hazards (67), followed by funding (66) and economic change (63) uncertainty. Strategic Plans have mostly addressed uncertainty around funding (45), climate/natural/environmental hazards (43), and government regulation and policies (41). Among all the types of uncertainties listed, funding, government regulation and policies, infrastructure and service costs, and climate/natural/environmental hazards stand out and are addressed the most in different planning and business processes.

2.4. Methods & Tools to Address Uncertainty

To better understand the application of different methods and tools used to address uncertainty, the survey asked respondents to highlight methods that their organization uses to analyze or manage uncertainty. The categories of methods provided in the questionnaire with accompanying descriptions and are as follows:

• Risk Management: Risk management involves identifying and assessing risks and vulnerabilities, as well as developing procedures to reduce their likelihood or mitigate their impacts.
• Sensitivity Analysis: Sensitivity analysis involves examination of the range of impacts that a variation in a given variable can have on an outcome.
• Causal Diagramming: A method for diagramming the relationships between causal factors and potential outcomes. Can be used in a workshop setting to identify and think through key factors that may drive uncertainty or affect desired outcomes.
• Scenario Planning: Scenario planning involves the analysis of and preparation for multiple potential futures.
• Monte Carlo: Monte Carlo methods rely on repeated random sampling.
• Use of contingency factors: Contingency factors can be added to a budget to account for unknown costs.
• Tradeoff and exploratory analysis: Tradeoff analysis examines the relative utility of different outcomes to decision makers in a given situation. Exploratory analysis is intended to uncover interactions or potential relationships among variables for further study.

The most reported methods to address uncertainty were scenario planning, risk management, and sensitivity analysis (Figure 7).

Note: Numbers do not add to the total as questionnaire respondents were able to select multiple options.

The categories of tools in the questionnaire administered in February and March 2023 with the wording of the descriptions are quoted below:

• Visioning and Sketch Planning Tools: Visioning tools help stakeholders develop a shared vision for the future, while sketch planning tools can illustrate the types, direction, and size of impacts of external factors and agency strategies on outcomes.
• Pavement Management Systems: Pavement management systems assist in pavement management decisions, often by storing data on pavement inventory and conditions, storing information on inspections, forecasting pavement condition, and selecting optimal preservation activities.
• Bridge Management Systems: Bridge management systems assist in bridge management decisions, often by storing data on bridge condition and inspections, forecasting bridge condition, and selecting optimal activities to maintain a state of good repair.
• Transit Asset Management Systems: Transit asset management systems store condition data and maintenance activity, prioritize maintenance and replacement activities within limited funding, and calculate investment needs.
• Travel Demand Models: Travel demand models predict future travel activity within their modeling region under specified conditions.
• Risk and vulnerability screening/assessment tools: These tools allow users to screen for vulnerabilities of the transportation system and assess risks, particularly from climate change, weather events, and other natural disasters like earthquakes, floods, tsunamis, and hurricanes. Hazus is an example.

Travel demand models, pavement management systems, and bridge management systems were the three most commonly reported tools used to address uncertainty (Figure 8). Risk and vulnerability screening/assessment tools and visioning and sketch planning tools were also commonly used by respondents in planning for uncertainty.

2.5. Barriers and Resources

In addition to understanding the methods and tools used to address uncertainty, the survey also asked respondents about the constraints they face in doing so and the resources that could assist them in better planning for uncertainty. These two questions were open-response questions, allowing respondents to share in greater detail. Open-ended responses were categorized based on identification of common themes.

The first question: “What are the most significant barriers or challenges that your organization has faced when planning for uncertainty?” received over 67 unique responses, spanning across 7 key themes, as noted in Figure 9 below.

Over a third of respondents noted aspects of decision-making to be a significant challenge to addressing uncertainty within their organizations. This includes planning difficulties presented by collaboration, coordination, and communication with stakeholders with differing perspectives. Other respondents note inertial effects and the challenges of adopting practices and processes that have long been in place.

Following decision-making, over a quarter noted that the lack of funding for planning is also a significant challenge for organizations, with many also mentioning that the lack of funding also contributes to other resource constraints – such as data (19 percent), staffing (16 percent), and tools (15 percent).

Nineteen (19) percent of respondents also mention that various forms of compounded or deep uncertainty are a challenge. These responses refer to interactions among many uncertain factors and unknown relationships or dependencies between factors. One respondent describes the challenges surrounding “the ability to estimate ranges of future outcomes and have confidence in those future values.” Another describes how “uncertainty can be overwhelming when many factors are considered at once.”

The second open-ended question — “What types of resources, data, tools, methods etc. would help your organization to better manage or plan around uncertainty?” — received 53 unique responses, spanning across 8 key themes, as noted in Figure 10 below.

Over a third of respondents suggested that more sophisticated modeling tools and routinely updated data sources could help their organizations better plan for uncertainty. In addition, over a fifth of respondents also mentioned that shared best practices, and potentially, future collaboration and communication, across organizations can also help better plan for uncertainty.

As with the prior question, 21 percent of respondents and 11 percent of respondents suggested that increased staffing and funding resources would help with planning, as the two remain a significant constraint in planning. Nine (9) percent of respondents also suggested that more sophisticated project management tools and visualization tools can also help streamline planning efforts.

3. Focus Groups

3.1. Overview and Participation

Two focus group sessions were held to gather information on how different industry leaders manage and address uncertainty. Team members from EBP, High Street, and Michael Baker reached out to their contacts across different industries including transportation, utilities, management consulting, and others. Participating organizations are listed in Table 2. One organization was not able to attend the focus groups and opted to participate in a follow-up interview instead. The focus groups involved a brief up-front presentation on the research project, followed by a facilitated discussion, including the use of interactive polling technology through Menti. Findings discussed in this section reflect both verbal and written feedback provided by the focus group participants.

Table 2. Focus Group Participating Organizations

___________________

²https://www.nikolamotor.com/investors/

³https://www.bge.com/AboutUs/Pages/default.aspx

⁴https://nacfe.org/our-story/

⁵https://www.rand.org/about.html

⁶https://www.georgetownclimate.org/about-us/index.html

⁷https://theray.org/

⁸https://wstc.wa.gov/roles-and-responsibilities/

⁹https://www.mckinsey.com/about-us/overview

¹⁰https://www.lyft.com/

¹¹https://tetcoalition.org/about-us/

¹²https://www.flysfo.com/about/about-sfo

3.2. Areas of Uncertainty

Participants were asked which areas of uncertainty they were currently investigating or most concerned about. Areas included funding, electrification, climate, market dynamics, and other technologies, to name a few. More areas can be found in the word wall below, which was generated using collected Menti inputs.

3.3. Focus Group Themes

Participants were asked probing, open-ended questions regarding challenges, effective approaches, methodologies, tools, and data based on their knowledge and experience with uncertainty. The discussions can be summarized into the following themes:

• Build and leverage partnerships
• Focus on outreach, education, and communication
• Adapt and prepare for change
• Use up to date data and projections
• Incorporate long-range lessons in stress tests

3.3.1. Build And Leverage Partnerships

Building, maintaining, and leveraging partnerships is key to address uncertainty. This includes partnerships both within the traditional transportation planning ecosystem (e.g., between State

DOTs) and with external transportation partners and those from other industry sectors. Since other industries are also dealing with uncertainty, transportation agencies can develop partnerships to learn and adapt successful strategies from those industries. Transportation agencies are also increasingly called upon to interact and coordinate with other sectors, such as organizations in the energy sector and technology space. Partnerships help share expertise or bring stakeholders together. Learning from industry leaders and their best practices can drive problem-solving and efficiency. It is easier and more effective to address challenges and uncertainty as a team. Partnerships which bring diverse ideas result in better solutions. Organizations may leverage partnerships to manage the resource requirements required to stay abreast of a constantly changing environment—by sharing knowledge rather than “recreating the wheel.” Partnerships can also reduce the risk to a public sector organization of being first adopters of new ideas if they are able to learn from case studies of best practice or build upon experience of other organizations.

An example of these partnerships is the Center of Resilience Excellence South Carolina (CORE SC), which uses collaboration and partnerships to provide best practices on resilience. CORE SC’s partners include federal, state, and local organizations; academic institutions; researchers; and non-profit organizations. The U.S. Department of Energy’s Clean Cities initiative is another example that brings together stakeholders around a specific topic, in this case energy-efficient transportation and domestic fuels. Technological advances such as electric and autonomous vehicles may require public agencies to develop new kinds of relationships with original equipment manufacturers (OEM) and private industry to stay abreast of dynamic market shifts and align planning with private-sector trends. As another example, the Joint Office of Energy and Transportation was created through the Bipartisan Infrastructure Law and facilitates collaboration between the U.S. Department of Energy (DOE) and the U.S. Department of Transportation (DOT) around issues of electrifying the transportation system. Multi-state organizations like the Eastern Transportation Coalition provide a forum for states to collectively work through emerging and often contentious issues like Road Usage Charges.

Relationships can be relatively more straightforward to start and maintain when each side benefits or has a common goal. However, it is just as important to work with and assure those that might feel threatened by a potential change or dynamic emerging future. Techniques like stakeholder mapping can be used to understand who stands to benefit from or be burdened by an emerging future. An organization can then devise a partnering and coordination strategy accordingly.

3.3.2. Outreach, Education, And Communication

Outreach.

Most Americans have some sort of social media presence. Social media has brought with it increased scrutiny of public institutions. Focus group participants report concerns about degraded trust by the public of public institutions. When public trust in national and state leaders is lacking, lawmakers have been hesitant when it comes to addressing challenges or stating new

initiatives. However, trust can be built and mended through education and helping to spread basic facts underpinning decisions through public outreach.

Education.

States, organizations, and individuals can all be resistant to change, especially if it is going to affect daily operations or one’s lifestyle. Many leaders avoid discussing uncertainties because it might be seen as indecisiveness, and some staff members are too occupied with urgent tasks to allocate their resources toward uncertainty. In the face of resistance to considering uncertainty, education focused on prior successes, successful early adopters among peer agencies, and use cases can help get more people on board with changes. It is generally easier to be a follower; more people and organizations are receptive to examples from others who have already demonstrated benefits.

Communication.

As the transportation industry continues to innovate, communication between sectors, industries, and peers will be important for achieving some transportation goals. This may include communication between the transportation and telecommunications industries to facilitate connected vehicles, or communication between the transportation and energy industries to facilitate vehicle electrification. Communication among industries is complicated by the fact that different industries use unique terminology and word definitions which can cause misunderstandings. Additionally, each industry is very complex, making it hard for people in other industries to truly understand. On a smaller scale, there can be issues within organizations if expertise is siloed and staff don’t talk to one another across divisions.

Integrated conversations between technical staff, boards, and the public can help bridge these divides among sectors and industries. Sometimes institutionalized communication mechanisms can break down silos and help keep this communication going. For example, the federal Joint Office of Energy and Transportation helps bring together experts from multiple sectors to lead on topics such as vehicle electrification.

3.3.3. Adapt And Prepare for Change

Change is unavoidable in all industries, including transportation. Unsurprisingly, the COVID-19 pandemic was a major talking point among the focus group participants since it has changed how we all work and live and we are still seeing the effects today. Participants also discussed uncertainty around other ongoing issues, such as the war in Ukraine and how that continues to affect the supply chain and economy. Participants noted how different sources of change have different timelines. For example, climate chance was described as a “long ramp crisis” while the war with Ukraine or the COVID-19 pandemic broke more quickly. The challenge is to prepare in a way that does not require specific prediction, but nevertheless allows an organization to address ongoing change “before it breaks on top of us.” A participant noted that there are definitely lessons to be learned from the COVID-19 pandemic in terms of flexibility, but they may not have been fully incorporated into agency practices and culture as of yet.

Focus group participants pointed to political uncertainty and the need to track policies on an ongoing basis. For example, requirements for GHG performance measures may be challenged in

court and change depending on who is in office; by tracking policies and political trends, transportation agencies can prepare for these changes. Adaptability and a willingness to change will always be needed since there will always be factors out of transportation agencies’ control.

Decisions can be made in ways that explicitly allow for flexibility and adaptability. For instance, making infrastructure modular can allow for adding onto it later to increase capacity if needed. Similarly, designing infrastructure so that multiple technologies can be used avoids locking into a technology prematurely when it is not clear which one will become dominant. Making investment decisions in increments allows decision makers to benefit from the new information that has become available at each step. It can also be helpful to explicitly try to understand the “worst case” – i.e., the future that requires the most extensive action or investment by an organization.

In practice, an incremental approach can be challenged by the existing planning process that makes it difficult to reevaluate projects that have been in the project development pipeline for a long time. A shift towards flexibility requires focused attention and leadership.

One participant noted how organizations can define “on ramps” and “off ramps” for projects—aka key decisions points and actions over time that can allow an investment to either be moved into the pipeline for action, or put on pause if conditions warrant it.

Organizations can decide in advance what information will signal that an outcome has reached a tipping point or level of maturity that requires course correction. Decisionmakers can also review prior predictions and assess their accuracy to help improve future forecasts. Transportation agencies must adapt to changes, monitor where we are now, and build a culture of flexibility and learning.

3.3.4. Use Up to Date Data and Projections

Current and accurate data can be a great resource for planning; however, incomplete or old information is common and increases the probability of challenges to planning and developing courses of action. Many existing transportation improvement programs (TIP) and statewide transportation improvement programs (STIP) are based on projections from 5-10 from years ago which may no longer be sufficient for planning. Compounding the challenge of tracking and forecasting is the fact that average conditions can often obscure the dynamics of change at the “cutting-edge.” For example, the North American Council for Freight Efficiency (NACFE) shifted their analysis of commercial truck fleets from a focus on on the average vehicle to a study of the “best of the best” to understand what’s coming to the future of the industry. NACFE recognized that it should not be using the average vehicle, which didn’t accurately represent the way the fleet has been changing recently, but rather look at the cutting-edge vehicles. In doing so, NACFE can plan around what the future is more likely to look like instead basing their decisions on a lagging average.

3.3.5. Incorporate Long-Range Lessons in Stress Tests

Transportation planning, and related forecasting activities, traditionally involve very long timelines. Long range plans can look at years and decades into the future; but things are changing monthly. Transportation agencies may need to re-evaluate projects that have been in the pipeline for a long time because circumstances change, and it is difficult to predict needs far into the future. Stress testing is one approach to managing the unavoidable uncertainty of forecasting.

Stress testing involves evaluating outcomes and performance under a wide variety of potential future conditions, often with the help of models where inputs and assumptions are varied widely. Stress tests can play an important role in planning by helping to identify potential risks as well as forecasting the probability of missing or meeting goals under different conditions. The resulting forecasts can be a basis for developing plans that can respond under these highly varied conditions. Transportation Improvement Plans (TIPs) can benefit from stress tests against projections. Use of real-time data can also allow us to track change more effectively. Stress testing plans can help move the locus of certainty from a prediction (e.g., I am certain that the prediction is accurate) to the plan (e.g., I am certain that my plan is robust in the face of many possible futures).