NCHRP 23-12

ARTIFICIAL INTELLIGENCE OPPORTUNITIES FOR STATE AND LOCAL DOTS – A RESEARCH ROADMAP

Appendix D: Workshop Report

Prepared for NCHRP
Transportation Research Board
of
The National Academies of Sciences, Engineering, and Medicine

Aditi Manke
Abhijit Sarkar
Matthew Camden
Alejandra Medina
Tammy Trimble
Christie Ridgeway

Virginia Tech Transportation Institute
Blacksburg, VA

Introduction

Figure 35 shows the details of the technical tasks and their dependencies. Task 2 summarizes the literature and trends in AI and transportation. Task 3 aims to summarize recent practices in DOTs. The workshops from Task 4 aim to facilitate knowledge transfer and planning. Task 5 aims to summarize learning from all the tasks to create detailed research needs report and a research problem statement with a proper dissemination plan.

Project Progress and Scope of This Document

As part of literature review task, the team conducted the trend analysis of AI applications in transportation using topic modeling and a co-occurrence matrix. This task was divided into two parts. Part, one aimed to understand the relationship between a certain transportation research problem (e.g., traffic monitoring) and its solution using AI techniques. The task further identified transportation research trends and AI-based solutions’ maturity. The team summarized the research over the last 11 years and found more than 65,000 research articles. Part two of the task focused on identifying transportation topic trends that are highly

researched for AI-applications. Part two is also aimed at identifying AI application trends in transportation research projects sponsored by state DOTs.

The findings from part one of the literature review showed that the topics within the transportation domain are interlinked but the relative interdependencies within the topics vary. For example, the transportation topic “traffic management” is highly related to other transportation topics, whereas the transportation topic “winter road management” is much less studied and does not show much dependence on other transportation areas. The team also found that AI topics like advanced machine learning, neural methods, and optimization are widely used in most transportation research areas. Part two of the literature analysis explored the extent of AI applications in transportation research in the last 5 years using the Transportation Research Board (TRID) database. The results show that most AI applications research is in the area of traffic management and transportation infrastructure. Since urban areas constantly face traffic congestion issues, AI tools can provide real-time information from vehicles for traffic management. The team explored 17 state DOT-funded transportation research projects that looked at applying AI tools in the last 5 years. Overall, the literature analysis presents the interrelationships within transportation areas and the various AI applications that are available for state DOTs, local DOTs, and the stakeholders to apply in research areas. The detailed report is now submitted.

As part of our outreach efforts, the Virginia Tech Transportation Institute (VTTI) team conducted eight interviews with individuals who work for or with state DOTs to obtain a snapshot of where state DOTs are in their adoption, understanding, and needs for successful implementation of AI-based methods to solve their state’s transportation problems. The team compiled the data and wrote a report outlining (1) the current priorities of state DOTs and their state of using AI to address these priorities, (2) the challenges associated with integrating AI into DOT work, (3) the workforce and infrastructure needs for AI work, and (4) where DOTs hope to be using AI in their work within the next 5–10 years. Results from this task show that state DOTs are currently looking forward to using more AI and ML functions in their daily work to address transportation problems. Some DOTs are already using AI tools and methods. Traffic management seems to be the main area of current work and future interest for AI opportunity. Incident detection and pavement performance were the areas where DOTs indicated they have incorporated AI methods. Researchers found that there is an active collaboration among DOTs and academic institutions and private sector companies for research and development of AI-based methods, but more collaborations are required. However, overall, there is a general lack of understanding, education, and support in terms of how AI can be used to help solve transportation problems. To move forward with AI work, state DOTs have several needs to fulfil in their workforce and infrastructure.

Scope of the Two Workshops

To inform the AI Research Roadmap, the research team conducted a series of workshops to engage stakeholders from DOTs on the current and future use of AI in their agencies. The first workshop had two purposes: (1) to allow representatives from state and local DOTs to discuss and validate the results from the literature review and interviews, and (2) to facilitate discussions regarding primary needs for research and advancement related to AI in state and local DOTs as well as regional transportation agencies. The second workshop focused on presenting the draft Research Roadmap ideas to the representatives of state and local DOTs and gathering their feedback on each of the ideas. This report summarizes the discussion that occurred during Workshop 1 and Workshop 2.

Methods

For this task, VTTI researchers conducted two workshops involving individuals associated with academic institutions, regional transportation agencies, and state and local DOTs. The interviews in Task 3

and Workshop 1 helped researchers to refine topics for the Workshop 2, which also included personnel from other federal and state agencies including the Federal Highway Administration (FHWA) and the National Highway Transportation Safety Administration. Workshop 1 was divided into two sessions: the first session occurred on October 3, 2022, and the second session took place on October 12, 2022. Workshop 2 occurred on March 7, 2023.

Step 1: Participant Recruitment and Outreach

Researchers recruited participants for the workshops via email. Apart from DOT representatives, researchers worked with the following Transportation Research Board committees to disseminate recruitment materials: AED50 Standing Committee on Artificial Intelligence and Advanced Computing Operations, ACS20 Safety Performance and Analysis, ACS10 Safety Management Systems, ACP15 Intelligent Transportation Systems, AP020 Emerging and Innovative Public Transport and Technologic, and AED30 Statewide/National Transportation Data and Information Systems.

For Workshop 1, the team reached out to 88 individuals for participation, including the 10 project panel members. These individuals were told that the workshop would be held on two separate dates for 4 hours each. This was designed to accommodate more DOT personnel across multiple time zones and to better distribute participants’ daily time commitment. These individuals were further asked to forward workshop information to other interested individuals. Follow-up emails were sent to the individuals after 1 week if the researchers did not receive a reply. Thirty individuals indicated that they would participate in the October 3, 2022, workshop and 26 confirmed participation for the October 12, 2022, workshop. The interested workshop participants represented state level DOTs, local DOTs, metropolitan planning organizations, and academic institutions.

For Workshop 2, the team reached out to 137 individuals from state and local DOTs, the FHWA, the American Association of State Highway and Transportation Officials (AASHTO), and 10 project panel members and individuals who had previously participated in the interviews and Workshop 1. Twenty-three individuals responded stating that they would attend the workshop on March 7, 2023.

Step 2: Finalize Topics

Workshop 1

As part of Task 4b, the research team finalized the topics at the first workshop as well as the workshop agenda. The team mainly considered the following questions:

1. How would state and local DOTs evaluate the proposed AI solutions?
2. How would state and local DOTs know they are ready to use AI (e.g., prerequisites for using AI solutions)?
3. What guidance, policies, and/or standards are needed to assist transportation agencies in successfully applying AI?
4. What are the potential risks, limitations, and challenges of AI for transportation and transportation agencies, transportation modes, and transportation systems?
5. What are the ethical, data security, and privacy challenges of AI, and how they can be addressed and overcome?
6. What is the diversity, fairness, and equity implications of AI?
7. What are the workforce development implications of AI, including preparing and training the workforce?

After discussion amongst the researchers at VTTI and with feedback from the advisors at VTTI, The team finalized four key topic areas. This topic areas:

1. Current and future focus areas of transportation development at DOTs: This discussion topic aims to identify research areas within transportation where AI could be useful. (Answering question 4)
2. Challenges in adopting AI-based solutions: This discussion topic aims to understand what limitations and challenges transportation agencies expect while integrating AI in their operations. (Answering questions 5,6)
3. Sustainable workforce and infrastructure development within DOTs for AI readiness: This discussion topic aims to understand what structures support a strong AI group within an organization. (Answering question 7)
4. Readiness of AI, evaluation, and third-party collaboration: This discussion topic aims to understand where organizations are struggling to evaluate, implement, and identify useful AI. (Answering questions 1,2,3)

Workshop 2

The topics for Workshop 2 were finalized based on the discussions that took place during Workshop 1 and the feedback that the research team received from the DOTs during the interview process under Task 3. Apart from the feedback received from the previous outreach efforts, the research team also referenced the National Artificial Intelligence R&D Strategic Plan that came out in 2016. The eight key strategies highlighted in that report are as follows:

• Make long-term investments in AI research.
• Develop effective methods for human-AI collaboration.
• Understand and address the ethical, legal, and societal implications of AI.
• Ensure the safety and security of AI systems.
• Develop shared public datasets and environments for AI training and testing.
• Measure and evaluate AI technologies through standards and benchmarks.
• Better understand the national AI R&D workforce needs.
• Expand public-private partnerships to accelerate advances in AI.

These eight strategies were considered while creating the final list of roadmaps. The team at VTTI created 14 draft Roadmap ideas that were presented during the Workshop 2:

1. Conduct case studies of successful AI program implementation at state DOTs.
2. Develop a Roadmap for successful collaboration with industry partners that provides AI-based solutions.
3. Create a toolbox to guide the selection and deployment of AI technologies at state and local DOTs.
4. Develop an educational toolkit to accelerate the adoption of effective AI programs.
5. Identify workforce needs and development to prepare transportation agencies for the application of existing and emerging AI approaches.
6. Identify state and local DOT funding strategies for AI opportunities.
7. Develop a guidebook to understand the vulnerability and security concerns for AI-based solutions to accelerate adoption.
8. Develop a Roadmap to create shareable, reliable sources of data.
9. Create a framework to process and manage data collected by DOTs.
10. Develop an equity plan for AI integration across DOTs.
11. Develop a research plan to include AI in less explored transportation research fields.
12. Include integration of AI-based methods in multimodal transportation planning.

13. Explore NLP-based methods to help solve problems at DOTs.
14. Explore use of Blockchain and AI in DOT research (such as Asset Management).

The research team came up with seven research areas where they felt the proposed draft roadmap ideas would address the problems in those areas. The seven research areas are: workforce development, infrastructure development, readiness and evaluation of AI, challenges in adopting AI, current practices and prioritization, external collaboration, and equity, policy & planning. Table 20 shows how the draft roadmap ideas falls into one or more research areas.

Table 20. Grouping the roadmap ideas by research focus areas.

Step 3: Conduct Workshops

The two workshops took place via Zoom e-meeting. During the workshops, researchers gave a brief introduction to the purpose of workshops and an overview of the research tasks that are part of the project. The first workshop included four sessions: (1) current and future focus areas of transportation development at DOTs; (2) challenges in adopting AI-based solutions; (3) sustainable workforce, and infrastructure development within DOTs for the implementation of AI; and (4) readiness of AI, evaluation, and third-party collaborations. Each session was scheduled for 45 minutes and included an open discussion to gather thoughts or comments from participants. In the second workshop, the research team presented 14 Research Roadmap ideas. Approximately 10 minutes were given to present each idea and gather feedback from the workshop attendees. After presenting all the roadmap ideas, researchers facilitated a discussion to understand how the participants would like to prioritize the research problem statements. Polls were administered in all the workshops to gather participant opinions.

Step 4: Analysis of Outcomes

Researchers performed a content analysis of the workshop transcripts, reviewing each session and gathering the information regarding the four main topics that were presented during Workshop 1. Researchers then compiled the information and developed a list of all the topics that were discussed in Workshop 1. For Workshop 2, the researchers followed a different method, summarizing the feedback and comments for each Roadmap idea that was presented. This report gives a brief summary of the roadmap ideas.

Results

The results for workshops are divided into two parts. The first part summarizes the outcomes of discussions from Workshop 1. The second part summarizes feedback from Workshop 2, where the draft Research Roadmap ideas were shared with the workshop attendees.

Part I: Workshop 1

Current and Future Focus Areas of Transportation Development at DOTs

This session focused on identifying transportation areas where DOTs can benefit from AI tools and where the deployment of AI-based applications should be prioritized. The session discussed DOTs’ plans for AI over the next 5 years and research areas that would require funding support. To guide the conversation, VTTI researchers presented a few key transportation areas identified during a literature review. Participants were asked which other transportation areas DOTs would like to see included in the Research Roadmap. In this session, one poll was administered to ask participants about what the top three research areas would be and where they would like to see AI integration in the next 5 years. The results are displayed in Figure 36.

Transportation Research Areas

Below are some of the key areas within transportation that participants suggested they would like to see the use of AI resources.

• Document Management: According to participants, AI can help with information mining and organization, records management, and the development and delivery of organizational practices across all DOT functions.
• Maintenance: One participant stated that DOTs have a specific maintenance function. The participant asked whether that maintenance function would be covered as part of transportation infrastructure or if the roadmap would have a separate maintenance topic.
• Winter Road Maintenance: Participants would like to see AI applications when it comes to winter operations, snow plowing, and roadway maintenance.
• Plan Review: One participant suggested adding plan review under the topic of Highway Management and Design.
• Natural Language Processing: One participant suggested an NLP tool could be used in interacting with the systems and in transcription of videos and meetings.
• Pattern Recognition: A pattern recognition tool, such as crack detection on bridges, could be useful for asset condition monitoring.
• Construction operations: One participant wanted to know about the use of AI in quantifying the material needed for construction, delivery of materials on site, and tracking reports to maximize the quality of operations.
• Project Management: One participant suggested project management as a topic for resource allocation since it takes different types of resources to complete a project and sometimes projects are put on hold because certain resources are not available.

Current State of Research

This portion of the Workshop explored the transportation areas identified by DOTs where they would like to prioritize the application of AI tools and the areas where some AI applications have been implemented.

• Performance Measures (PM3): According to one participant, there is a need for AI tools in PM3 measures—for example, tracking system reliability, congestion, and high linear-nonlinear behaviors where the data looks complex—and these techniques might be helpful.
• Safety: One participant highlighted the urgent need to address safety from the infrastructure as well as the driver behavior side. For example, it might be possible to use AI tools in combining geometric data gathered by some DOTs’ data collection vehicles and observing that versus crash modification factors and trying to filter out scenarios resulting from various interventions.
• Non-recurring Congestion: There is a need for specific AI applications that can be used in examining data to identify the delay caused by non-recurrent congestion.
• Pedestrian Safety: One participant shared the successful application of AI in pedestrian safety using video analytics to determine relevant safety issues. This might include, for example, looking at what times most pedestrians are present and changing the signal timing accordingly to reduce the conflict between pedestrians and vehicles.
• Definition of AI: A few participants highlighted that to prioritize AI-based deployment in any transportation research area, it is important to understand what is considered to be “AI.” According to participants, AI can perform various functions, but the important question for DOTs is what they should use it for and where the benefits would outweigh the costs.

Future Scope

The portion of the Workshop focused on the areas of AI concentration for DOTs in different transportation areas.

• Simulators: One participant underlined the challenge of having simulators in the future that can create counterfactuals in various driving scenarios. Stating further that AI-based algorithms are generally data hungry, and the problem with providing data under current conditions alone does not allow us to explore what/if situations. For example, if signal timing is changed, what happens to pedestrian safety and traffic volume, or what happens if the weather conditions change. It is difficult to apply these variables to real data because of various limitations, and simulators can help bridge this gap.
• Partnerships: One DOT member shared information about the work they have done in the classification of text detection with intelligent transportation system imagery video through partnerships with universities. Since this DOT lacks the necessary technical hardware resources, they are trying to pursue more partnerships with industry and academia in the future.
• Business and Information Practices: One DOT emphasized the lack of standardization in some of the practices for which the DOTs want to use AI. This can be problematic when trying to develop a tool for use. According to this DOT, the real opportunity would be to take a step back and understand the flow of data and information starting from the data collection process all the way through to the uses of the data and using that data for feedback into the DOT system. There is also an opportunity to improve employee awareness, knowledge, and access to information.

Challenges in Adopting AI-based Solutions.

The session focused on the potential risks, limitations, and challenges that transportation agencies expect while integrating AI into their operations. The purpose was to expand on the ethical, data security, and privacy challenges of AI and how DOTs can address these challenges. The poll administered in this session asked participants to select some of the challenges that they face at DOT level from the given options. A total of 20 people out of 29 responded to this poll. The results are shown in Figure 37.

The discussion during this session was steered around five key challenges identified by the research team: (1) availability of data, (2) data security, (3) computing resources, (4) workforce, and (5) trust in AI. The points discussed in each of these challenges are reviewed below.

Availability of Data

• One participant spoke about having confidence, or lack thereof, in the data. Often, data measurements have errors and are not predictable. Even trying to measure things in multiple ways and trying to determine which of the measurements are correct or incorrect is very difficult and thus the agency finds it challenging to train the systems.
• A few participants emphasized having quality data. One participant further elaborated that quality also extends to the variability of the data received at times, which is not only related to network stability but also has to do with what happens on the roadside. Systems break down and may be down for periods of time, and this is where the challenge lies in having AI that is stable and does not overreact. One way to address this, according to participants, is to have a human in the loop to prevent unanticipated consequences when the quality of data starts to decline.
• On being asked whether DOTs would be interested in sharing data platforms or creating platforms to collaborate, one participant’s opinion was that, at a broad level, having data alone without any practical application for those datasets won’t be useful. There might be a perception that if one entity starts collecting data then others would join in, but this often doesn’t happen due to associated costs. The participant suggested that instead of having datasets alone, it would be good to develop interesting applications and to then sell those case studies, at which point they can become transferable between DOTs.
• Another challenge that came up during the discussion was regarding the need for labelled data, which is expensive from the collection perspective, as DOTs are unaware of the kind of data they are receiving.

Data Security

• In terms of security, one DOT shared their experience of running into issues with migrating data to the cloud while trying to make it more accessible. While talking about operating systems, there were concerns that DOTs do not want anything cloud hosted touching something that has an operational impact, and that architecture then makes it difficult to immediately push something to the cloud. The workaround has been to migrate data using hard drives, which is less secure. When it comes to personally identifiable information, there were concerns related to connected vehicles, and specifically, signal request messages.

Computing Resources

• From the DOTs’ standpoint, the challenge is lacking knowledge about which resources to buy into for the DOTs to be effective based on their operational objectives. One participant shared that the agency was able to do some small-scale testing to verify what hardware is necessary and was then able to wrap it in through normal procurement. The hard part for DOTs is getting over the hurdle of knowing what resources are required.
• One DOT participant classified advanced computing projects into two categories. For example, exploratory projects develop an approach to understanding things versus, while applications are more likely production-based and apt to use machine vision. According to this participant, the computing resources would be different for these two kinds of projects. The challenge for DOTs is to define what and how many resources are needed to address exploratory projects.

Workforce

• The ability to find and maintain a reliable workforce is challenging for DOTs. In addition to this, there is the question of communicating to the agency at a broad level and communicating what is expected from employees. As per one participant, there is an education and communication issue, which is perhaps rooted in some of the data governance approaches that DOTs have been trying to use. Further, this participant added that management usually needs a very condensed version of requirements to get the needed funding. It is important to figure out how to communicate opportunities to the managers and state legislators who are providing funding for some of the emergent activities.
• One participant elaborated that people who have AI-based training are in high demand today. DOTs and even universities are facing challenges in attracting and retaining these people.

Trust in AI

• One participant shared an example of trying to use computerized information at their agency, noting that there was resistance from employees to rely on what they see on the screen versus what they see in the field. This participant identified a need to incorporate some human factors considerations into the use of AI products.
• Another participant shared their learning from a research presentation about medical imaging to detect cancer where researchers tried everything to eliminate the variable of race from the data, but it could still be determined by the AI. According to the participant, there are inherent biases in the dataset that the DOTs don’t even know about and AI is able to pick these up in some way that we don’t understand. There are a number of ethical issues and inherent risks in the use of these kinds of algorithms to make decisions about people. The agency needs to be cognizant of these issues and risks since they will be using these algorithms in ways that might affect people differently.
• The issue of trust arises from the inability to really understand exactly what the AI model is doing. It is imperative to think of these AI models as supporting decisions rather than making decisions for DOTs.
• To include AI in transportation, it is probably best to create a paradigm or steps where, in the beginning, AI performs data summarization. At a second level, there may be diagnostics to show where the problems are. A third level could be predicting problems and having a human in the loop to address them. It’s after these steps that AI can be considered complete automation.

Sustainable Workforce and Infrastructure Development within DOTs for AI Readiness

Another piece of the AI Roadmap is understanding what structures support a strong AI group within an organization. Session three focused on identifying the infrastructure, workforce, and partnerships that participants used within their respective areas to build their AI groups. The purpose of identifying these supporting elements was to understand what areas organizations are struggling with when implementing AI. Participants were encouraged to share their experiences with these topics.

Challenges With Sustainable Workforce

• To gain more insight on the issue, participants were asked about the policies currently in place at their organization regarding AI workforce development and how successful they thought these policies were at building their AI department. Many participants agreed that the private sector is more appealing to new job applicants, so the DOTs struggle to find qualified people.
• One participant stated, “I just don’t see how we can compete. Companies are paying three times what we can offer. We just can’t pay enough.” Several others agreed with this sentiment and commented that because they can’t pay enough to keep in-house data scientists, they tend to outsource their workforce. However, several participants commented that outsourcing can have major drawbacks because the consultants often have no experience in the transportation sector.

• One participant shared that their company invested a large sum of money for an AI professional to analyze crash information. The professional’s conclusion was that when there are more cars on the roadway there are more crashes. For professionals in the transportation industry this conclusion seemed obvious and was not worth the money. This exemplifies how AI scientists and transportation researchers can sometimes have a disconnect when explaining problems. Another participant supported this argument, stating that “there are many nuances in this field, and someone needs to take the time to understand the problem more in depth than an outsider ever could.”

Approaches to Address Workforce Issues

• Participants were asked what they thought the best approach was to solve the issue of workforce, or if any company had found strategies to mitigate the negative impacts of consulting. One participant began the discussion by redefining the problem. They claimed that the issue isn’t just the lack of workforce, but to move forward, each sector needs to learn how to better describe their problems to the other party.
• Furthermore, another participant added that the companies need to understand what they are asking for specifically before “throwing money at it.” One solution brought to the table was involving universities. Several participants agreed that universities often have interdisciplinary students with diverse backgrounds, such as transportation, who are looking for experience. Participants greatly supported partnerships like these. They valued a trustworthy partner who can bring in consistent work.
• Lastly, participants were asked how they thought DOT leaders would react to AI pervading the industry. One participant claimed that upper management often doesn’t understand the amount of effort it takes to plan an AI project. They stated that they think the organization will support the involvement of AI, but they may not be prepared for the resources required to follow through.

Readiness of AI, Evaluation, and Third-Party Collaborations

The final session of Workshop 1 addressed the implementation of AI. The purpose of the session was to understand where organizations are struggling to evaluate, implement, and identify useful AI. Participants were encouraged to share their experiences with these topics. Two polls were administered during this discussion session. One of the polls asked what type of organizations their DOT would need support from to implement AI programs. Participants chose from the options given to them, with 10 people responding to that poll. The results are shown in Figure 38.

The second poll asked participants whether their DOTs had difficulties in determining if an AI program was ready for implementation. There were 11 responders to this Yes/No question, with results shown in Figure 39.

Identifying an Opportunity for AI

• One DOT participant explained that they first identify a particular problem in their field—the example given was an increase in lawsuits due to a lack of guardrail treatments. They realized this problem involved a lot of data assessment to figure out where guardrails currently existed and whether they had end treatments. They concluded that because the problem involved such a large amount of data, AI seemed like an obvious answer.
• Another DOT participant added that AI has strengths and weaknesses and should be used in areas where AI excels. Further elaborating about AI’s strengths, the participant asserted that AI is better utilized for operational situations and understanding where to deploy resources. Similarly, it is best used when there are big questions that need to be answered with real-time, descriptive analytics.

Determining the Readiness of AI Programs for DOTs

• One DOT participant commented that understanding the maturity cycle of AI would be useful for knowing when it should be implemented. This participant also emphasized moving more towards data standardization so that it can be clearer in the future when AI is ready for launch.
• Many participants mentioned education as a solution for not understanding when AI is ready. For example, one participant suggested educating upper management on AI planning specifically for traffic management and data prediction. They further noted that the reason behind this suggestion was that they felt they were ready from a technical standpoint, but not from an organization standpoint.
• One DOT participant shared that they were struggling to scale AI up to larger projects and noted the value of identifying when something does work, then explaining why it worked, so that they can recreate it.
• Additionally, several DOT participants stressed the importance of collaboration. One participant discussed that collaboration with the civil engineering department, and IT would be beneficial due to the types of problems being worked on.

• Another participant stated that their organization has workshops with the different divisions for demos so they can share ideas and enhance collaboration. Many of the other participants thought this internal collaboration idea was valuable. A topic also discussed in the third session, this group discussed using universities with interdisciplinary backgrounds to find interns and specialists for AI projects.
• Overall, the participants felt their organizations could benefit from increased collaboration both internally and externally to see how other people are using AI to solve big issues. They also expressed concern with evaluating the effectiveness of AI and suggested education on an operational level within the organization. Lastly, participants suggested standardization of more common transportation issues to allow scalability and better collaboration.

Part II: Workshop 2

The following are the 14 Research Roadmap ideas that were presented during Workshop 2. Each Roadmap idea is briefly summarized, followed by feedback or comments that the team received from participants.

Conducting Case Studies of Successful Implementation of AI Programs in State and Local Transportation Agencies

Summary

The goal of this project will be to conduct case studies where DOTs have successfully implemented an AI program to improve transportation efficiency or safety. The rise of AI has led to the creation of new programs and countermeasures, which experts at DOTs usually lack evidence for the success of. Documentation of successful AI-integration within transportation-related programs could instill trust and push DOTs to incorporate AI in their operations.

Feedback/Comments

• One of the participants suggested defining AI programs more specifically since “AI programs” comes across as a broad term.
• Another suggestion highlighted that case studies will have limitations due to the rapid change in algorithms. Instead, the focus of the project could be more applicable to an organizational integration rather than as a technical project.
• Another suggestion was about the use of quantitative measures to track the success of AI programs. Examples should be established that explain how people have incorporated AI processes into their business and how often they have to update the areas in which they are applied. A broad definition of success might not be useful in this case.

Develop a Roadmap for Successful Collaboration with Industry Partners That Provides AI-based Solutions

Summary

Currently, private companies are leading in providing AI-based solutions. From our outreach efforts, we learned that many individuals at DOTs lack knowledge about existing AI resources and often face problems integrating these solutions into their programs. The objectives of this research would be to understand the growth of industry in the intersection of transportation and AI. This research will also create a plan that could encourage partnerships between DOTs and the industry. The project should also focus on building criteria that could aid DOTs in efficiently choosing an AI solution partner.

Feedback/Comments

• Benchmark tests could be used to help agencies in assessing different solutions. This kind of approach would require an independent third party to run testing. Using AASHTO could be one approach, as could having a pilot project using a transportation pooled fund with a subset of DOTs.
• The project should involve creating a categorization of AI solutions mapped to operation and safety strategies, and identifying issues in implementation would be helpful for agencies and AI stakeholders to react. This would help prioritize early AI implementation opportunities as well as prioritize research. The participants were interested in learning about the applicability of the solutions and not just the solutions themselves.
• One participant highlighted their interest in the use cases of what others are thinking about and would be interested in hearing the results of collaboration with industry partners.
• One participant noted that collaboration with industry partners would work but the issue is that DOTs may not understand how AI could help them, so providing examples would be a good extension of this idea. It will be important to provide clarity with partners about data ownership, use/reuse, record management, privacy, product management and using products in feedback/continuous improvement, and the people side of AI/ML as well as machine-to -machine interaction.

Toolbox to Guide the Selection and Deployment of AI Technologies at State and Local DOTs

Summary

The primary objective is to develop guidelines to help engineers to decide in which areas and under which conditions the state DOT will benefit from implementing AI technologies. It is expected that these guidelines will help to identify DOT readiness, potential alternatives to address the AI project, and prioritize the deployment of AI projects.

Feedback/Comments

• One suggestion highlighted that many surveys have been done on AI in transportation regarding maturity levels. Inclusion of such surveys in this project could be an add on and could refresh the information since AI changes so rapidly. Using the Capability Maturity Model would be helpful.
• One comment supported the need for AI and ML to be used to solve DOT problems. This participant’s concern was that DOTs need guidance to filter out marketing schemes from actual real applications of AI since the term AI is often used as a marketing buzzword.

Develop an Educational Toolkit to Accelerate the Adoption of Effective AI Programs

Summary

Conduct outreach and awareness of state-of-the-art AI technology guidance and identify champions for transportation AI applications and sponsor peer exchanges to allow newly interested agencies to learn about their noteworthy practices and lessons learned.

Feedback/Comments

• One participant shared that there is enough knowledge regarding AI at the leadership level. The problem lies at the lower staff level, since the operator who actually has to use the system lacks the required education. The focus should not be on leadership alone; the DOT also needs to work with various levels of staff. AI/ML is used to predict crashes and the staff looks at use cases from other DOTs. They are often unaware of what to expect in terms of results, what information they need, and what input they need to give to the algorithm.

• Educational material could be beneficial for state legislators in addition to the individual organizations to garner awareness and support for the resources and knowledge needed to develop AI at agencies.
• Another comment elaborated on how a DOT needs to prepare and staff to maintain an AI/ML solution. How can the DOT manage the training data? Research is needed to understand cases when a ML model may not fit “reality” vs the original training data.
• Educational material could also include how to train staff to understand when the model is over or under fit. It is easy to create a model but difficult to maintain and communicate its use and importance to everyone.

Workforce Needs and Development to Prepare Transportation Agencies for the Application of Existing and Emerging AI Approaches

Summary

The objective of this project is to identify the needs of workforce personnel who will be in charge of the operations that incorporate new technologies and to provide recommendations of how to develop and deploy the required training/certifications. The research must identify the current workforce and strategies to build future capacity as technology evolves. Note that the workforce needs include skill sets and education requirements for supporting personnel (i.e., data collection).

Feedback/Comments

• One participant suggested that integration of AI training into existing discipline-based training will be critical. There will be both new positions and new skills for people in existing disciplines and job classes.
• Identifying needs and responsibilities for the data collection/management/use would be beneficial to include in the workforce needs.

State and Local DOT Funding Strategies for AI Opportunities

Summary

There is a need to identify how states and local governments can use existing funding mechanisms and new grants to test and incorporate AI into the transportation processes. This project will conduct outreach to state and local agency staff on available funding opportunities for the incorporation of AI in existing and future processes, best practices for estimating project costs, and identifying matching funds.

Feedback/Comments

• There was a lot of support for the idea of using pooled funds.
• One participant summarized the advantages of pooled funds having more flexibility and duration than NCHRP projects. These projects could take some time to set up but are beneficial, especially for early adopters.
• One participant shared their perspective that AI should be treated as a tool to accomplish things that we already do and have funding for versus being an objective in its own right.

Develop a Guidebook to Understand the Vulnerability and Security Concerns for AI-based Solutions to Accelerate Adoption

Summary

AI-based solutions are often governed by black box models. Modern AI-based methods provide high performance, but at the price of explainability. AI methods can also result in unwanted biases, depending on how they were developed. Adversarial attacks can make AI methods produce incorrect results, as well as create security vulnerabilities. Lastly, most AI methods perform well in a specified domain, but fail to generalize in other domains. These limitations can make them vulnerable in many ways. Therefore, a deeper understanding of AI methods and limitations is very important. This project aims to create a guidebook of these possible limitations, biases, and vulnerabilities. These methods can vary by their application use at a DOT, the complexity of the application, and the methods and toolsets used for the targeted solution. This part of the project will highlight the risk of these limitations for various applications and create a guidebook for an explainability and testing regime that will guarantee efficient AI deployment.

Feedback/Comments

• There was a suggestion to have the goal be “explainable AI” as opposed to “understandable AI.”
• Explainability is a separate issue from bias, as is security. There are a small set of vulnerabilities that are specific to ML in addition to broader IT vulnerabilities.
• It would be good to look at more specific applications and the environment where these applications take place.

Roadmap to Create Shareable, Reliable Sources of Data

Summary

The current revolution of AI is driven by large-scale data, and most recent AI models are also data driven. However, there is a scarcity of reliable large-scale data sources in transportation research that can help solve problems at a state level. This is due to two main issues. First, the existing data lacks enough metadata. Secondly, these datasets lack the proper quality control, diversity, and annotations required for wide-scale AI deployment and testing. Many DOTs already collect their own data using advanced sensors; however, these data are often targeted for very specific applications and geographic areas. The goal of this research is to first identify already existing datasets along with the transportation research areas for which these datasets are applicable. The project will focus on selecting attributes that define the data quality and provide a path for improvement in the existing data resources. The project will also identify the data gaps that exist in research. This will help to identify data needs across state DOTs. Finally, the project will develop a Roadmap on how to collect new data (including that from industry partners), manage the data, and make datasets sharable across DOTs.

Feedback/Comments

• Creating metadata frameworks for specific applications could be helpful for agencies to document and share information.
• It could be useful to leverage the known underlying relationships to create artificial data.
• Include sharing of data codebooks and peer exchanges of people that code data can aid in data transferability. Even if people are collecting the same data, it is difficult to share because there are slight differences that create nuances. Sharing requires a great level of detail.
• A key suggestion that would be adding to any AI proposal specific contractual language and means of checking quality of delivery. It is important to understand how to write contracts that include AI specifics.

Framework to Process and Manage Data Collected by DOTs

Summary

There is no dearth of data in today’s world, but we often lack in-depth knowledge on the diversity and nature of the data that is being collected. It is still unclear in what aspects of data analysis AI can be used, and which tools are best suited to manage large volumes of data. This study will help in creating a manual and identifying resources and AI tools that would help data engineers in understanding the type of information that is collected and how it can be analyzed. This project will also create a guidebook that emphasizes human-AI interaction to ensure there are no ethical biases during decision-making.

Feedback/Comments

• A key consideration is that documentation on third party collection and data cleaning is important since agencies are not going to manage the entire workflow in-house.
• It is necessary to have human review as an element in the guidebook. It would therefore be useful to document effective practices.
• One suggestion was that access to data is critical for developing any AI application.

Development of an Equity Plan for AI Integration Across DOTs

Summary

There is a big disparity of AI inclusion across DOTs. Will collaboration between DOTs help bridge the gap? The project should address some of the challenges and possible solutions for such collaborations. Further, the project will choose five DOTs that are most advanced in AI implementations, and five DOTs that are not. We will hold interviews and workshops with these DOTs to better understand the gap. The project will develop a set of parameters that will define equity amongst DOTs and will create a plan for how to measure those parameters across time to evaluate improvement.

Feedback/Comments

• There was only one comment on referencing the Capability Maturity Model to assess an organization’s level of AI readiness, which could be useful in defining some of the collaboration opportunities.

Research Plan to Include AI in Less Explored Transportation Research Fields

Summary

The objective of the research would be to identify areas where AI has not been applied but has the potential to be implemented. During the outreach efforts, the team identified that DOTs are looking for technologies that could help them in document management, automating customer services, winter operations, etc. AI-based solutions are often concentrated in a few application areas like traffic management. The project should focus on identifying solutions that could be used by DOTs in their daily operations and maintenance.

Feedback/Comments

• One participant shared an FHWA funding opportunity that focuses on AI applications on winter maintenance. See https://www.fhwa.dot.gov/publications/research/ear/21090

Integration of AI-based Methods in Multimodal Transportation Planning

Summary

For a transportation system to be efficient and fair, it must serve diverse demands. For most of the last century, transport planning has heavily centered around automobiles. As a result, most communities have well-developed road systems that allow people to drive to most destinations. This kind of planning has left out non-automobile travel demands, and over the years the automobile-focused road system has seen an increase in travel time and traffic congestion. The objective of this project is to study some of the predictive models that look at reducing travel time and peak period congestion, determining some of the gaps and limitations in the existing models, and identifying if new variables need to be considered in the predictive models. The project should also focus on data analysis techniques that model the travel demands of bicyclists and pedestrians.

Feedback/Comments

• One suggestion highlighted that this seems like a promising field but that DOTs still need to take baby steps, as AI cannot be relied upon to do everything.

Exploring NLP-based Methods to Help Solve Problems at DOTs

Summary

In recent years, NLP has become immensely successful. NLP helps to automatically study texts and documents. This shows promises in minimizing manual efforts in document management, text summarization, and customer service. Large language models like ChatGPT have shown potential to automate tasks like customer service and can provide quick automated responses and custom messages, minimizing human interaction time. DOTs handle a large volume of documents every day. This may include project reports, environmental assessments, traffic studies, contracts and agreements, budget and financial reports, and employee information. NLP can significantly help in maintaining and interpreting these documents while reducing human hours. The goal of this project will be to identify key areas and tasks at DOTs where NLP can be useful. The project will also identify a list of available NLP tools. Finally, the project will demonstrate the benefits of NLP in several examples of use cases.

Feedback/Comments

• One participant suggested that this could be an interesting area of exploration. This participant has seen major issues with these types of services, including instances where an NLP tool has literally made up information, references, and even fictitious newspaper articles. Thus, any research on NLP tools should also look at how to confirm their validity and accuracy. It is also important to address data privacy and ownership issues.
• One participant noted that the federal agencies are starting to look at NLP in rule making.
• One participant noted using the Transportation Research Thesaurus (TRT) but according to this individual, it is still small compared to resources in other fields such as agriculture or health. The participant noted that the European Union vocabulary site could help in expanding this idea. See https://op.europa.eu/en/web/eu-vocabularies

Use of Blockchain and AI in DOT Research (Such as Asset Management)

Summary

Blockchain is an emerging field of study with a promise in maintaining secure transaction records. Blockchain technology has the potential to revolutionize the transportation industry by providing solutions for issues such as supply chain management, security, and data sharing. Blockchain technology, in

conjunction with AI tools, can immensely benefit asset management for DOTs. Specifically, it can help in asset tracking and maintenance, risk management, fraud detection, and investment management. Blockchain also provides a secure and efficient way of data sharing, increasing transparency. The primary goal of this project is to study the current advancements in Blockchain technology in relation to research needs at DOTs. The project will also perform a feasibility analysis of the usage of Blockchain technology in supply chain and asset management.

Feedback/comments

• No specific suggestion or comment was received on this idea.

Results from Polls

Three polls were administered during Workshop 2. The first poll asked participants to prioritize the ideas from one to six. These ideas were related to workforce and infrastructure, and readiness of AI implementation. The results are shown in Table 21.

Table 21. Prioritize the ideas from one to six for workforce and infrastructure needs, readiness, and evaluation of AI programs.

The second poll asked participants to prioritize the ideas from one to eight. These ideas were related to current practices of AI within transportation and challenges in adopting AI-based solutions. The results are shown in Table 22.

Table 22. Prioritize ideas from one to eight for current practices of AI in transportation and challenges faced by DOTs.

In the third poll, participants were asked to rank all 14 Roadmap ideas based on the likeliness of them receiving funding on a scale from 1 to 7, where 1 was not likely and 7 was extremely likely. The results are shown in Table 23.

Table 23. Rank the Roadmap ideas based on the likeliness of receiving funding.

Conclusion

This task attempted to investigate the research needs to support the integration of AI tools and resources across all levels of DOTs in the United States. To achieve this, the task involved two workshops, the first of which was conducted in two sessions, and the second in one session. Results from Workshop 1 included participants recommendations for a few transportation topics where applications of AI could be resourceful such as document analysis, project management, and plan review for highway maintenance. Discussion also highlighted that participants thought that the most important step forward would be learning how to describe the problems they need to solve in better terms. Further, they were looking for guidance and documentation on how DOTs could use AI and ML applications. Another notable issue concerned collaborating with people in the AI/ML space over time so each partner could learn about the nuances in the other’s problem space. Results from the polls showed that 40% of participants expressed challenges in availability and quality of data, 40% of participants stated that DOTs lack awareness about the use of AI, and 70% of participants expressed that they lacked the workforce to execute AI applications in their projects. The team found that DOTs often outsource their needs to universities and consulting companies since these organizations can provide DOTs with interdisciplinary workers.

During Workshop 2, the team presented 14 draft Research Roadmap ideas, which were created based on the discussions during Workshop 1 and the interviews that were conducted with state DOTs under Task 3. In this session, brief background information, research objectives, and a research plan were presented for each Roadmap idea, followed by feedback and comments from participants. Results from the discussions and polls show that participants were looking for case studies that provide examples of how AI has been incorporated into different organizations’ practices and why was it used. Many participants agreed that Roadmap ideas that focused on creating educational materials regarding the adoption of AI programs and toolkits to select appropriate technologies would be very useful. Few participants showed interest in ideas that focused on defining datasets that could be shared across DOTs for AI analysis and use as well as for creating standardized data management practices. Overall, the results from both workshops tell a story that representatives from state and local DOTs as well as regional transportation agencies are looking into AI-based solutions to address problems within transportation. To achieve that goal, participants’ first priorities were to understand and learn from some of the successful cases of AI integration, to have access to educational materials to spread awareness, and to have a guidebook to identify which resources would best fit their needs. fulfill.

The next step of the project involves taking these comments into consideration to refine the research roadmap ideas. The research team will identify unanswered questions and research opportunities regarding how AI will converge with state and local DOTs, including creating knowledge base, the role of the employee(s), the skills/training needed for the integration of AI into DOT practice, and barriers to effective integration. The final research roadmap will provide a broad overview of existing literature, the state of the art of AI within states and local DOTs, knowledge gaps, constraints, research, and project opportunities to fill these gaps, and potential synergies between state and local DOTs and other agencies. The roadmap will be inclusive enough to reflect the broader vision and initiatives of states and local DOT’s. The roadmap will also focus on accelerating adoption and implementation of AI in the next 5–10 years. The team will prepare a research needs report that will include general description of research that should be conducted within the next 5 years, and the team will provide a minimum of 10 research problem statements suitable for NCHRP or other funding sources.