The research team for NCHRP Project 08-119 was charged with developing multiple standalone (and largely unrelated) research products that addressed the data sharing, integration, and management needs of transportation agencies, as identified in Phase I of the research, to advance transportation operations and planning use cases. These stand-alone research products have been compiled into this NCHRP Research Report to keep the research outputs together under one cover. Each stand-alone research product is provided in a separate chapter in Part II so that readers can easily identify products of specific interest without reading the Research Report in its entirety. This chapter provides the reader with an overview of each product.
State and other government planning agencies explore big data sources to facilitate better freight transportation planning and operations decision-making. However, limited funds, differing planning horizons, proprietary information, insufficient human resources, and misinformation about available data have resulted in challenges related to the acquisition, integration, and use of freight-related big data. The freight data decision tree is an interactive tool developed to illustrate how agencies can integrate and use big data sources to address freight planning and operational use cases.
The freight data decision tree is for freight transportation planners seeking guidelines on which data sources best address specific planning and operations use cases. The tool includes the following freight planning and operational use cases:
The tool includes information on the following datasets:
The freight data decision tree uses a JavaScript library that renders an embedded data structure into a collapsible tree view. A subtree expands when the user clicks on each node on the tree until the applicable list of data sources for that use case appears. When users select a data source, they are taken to the section describing the content of the data, including metadata and use cases for the data source as identified from the literature.
Visit Chapter 6 in Part II to learn more about the freight data decision tree.
Previous research has highlighted the need for an interoperable freight data architecture to help strengthen multimodal freight data collection efforts; enable interoperability between multiple systems; and aid with data accuracy verification, validation, gap identification, and integration for seamless exchange of information. While attempts have been made to address differences in freight data sources as well as data collection, querying, and fusion methodologies, most studies address specific data sources or aim to address a specific use case. The freight data interoperability framework is a simplified, generic, and robust freight data querying methodology that data enthusiasts can leverage to encourage the implementation of an interoperable freight data architecture.
This chapter summarizes the state of the practice in freight data interoperability in the areas of data processing, data fusion, and data querying; presents a proposed freight data querying methodology; and demonstrates the querying methodology for various use cases. The chapter also presents a suggested approach for reconciling differences in location and vehicle classification as well as an approach to addressing vehicle routes by inference from the start and end locations.
The proposed querying methodology assumes four base parameters for most freight data sources: location, time, vehicle classification, and use case. The tool infers other parameters, such as route and geographic extent, from the start and end locations specified as part of the data query. The proposed methodology was tested with the following use cases:
The goal was to identify commonalities and differences in data elements when working with multiple freight data sources.
A web-based geographic analysis and data-querying tool was developed and employed to query multiple freight-related databases by using the base parameters. The validity of the proposed methodology was evaluated by assessing the outcomes of the queries.
Multiple databases were used in the analysis, including origin–destination datasets such as Geotab, StreetLight, and INRIX; point location datasets such as crashes and bridges; route-based datasets such as the NPMRDS; and geographic area datasets such as population data from the U.S. Census Bureau or GDP data from the U.S. Bureau of Economic Analysis. The use cases illustrate the feasibility of resolving some of the differences in definitions of data elements and the shift toward interoperability in freight data sources.
Potential next steps include developing a complete web-based application that uses the methodologies developed from this study and enhances the recommendations provided through testing of additional use cases.
Visit Chapter 7 in Part II to learn more about the freight data interoperability framework.
Crowdsourcing is the practice of addressing a need or problem by using technologies to enlist the services of many people. In the context of transportation, travelers generate crowdsourced data both passively and actively (e.g., by using mobile navigation apps). Crowdsourcing can offer low-cost, high-quality operations data (e.g., traffic speeds, slowdowns, events, crashes) and improved situational awareness without the need for costly traffic monitoring technologies. This chapter provides a practical understanding of how state and local transportation agencies can access and use Waze for Cities data.
The goal of the chapter on the use of Waze for Cities partnership data for operations and planning is to support state and local transportation agencies in better understanding the data available through the Waze for Cities partnership and to inform a more robust use strategy for the Waze data. The chapter
The chapter draws from published and unpublished discussions, research, and documentation from the crowdsourcing for operations innovations in FHWA’s Every Day Counts Rounds 5 and 6 (EDC-5 and EDC-6, respectively) including reporting from a peer conference where state and local agencies discussed their experiences using and integrating crowdsourced data; direct experiences from pilot uses of the Waze for Cities data; and other research efforts related to the use of crowdsourced data for transportation applications.
Visit Chapter 8 in Part II to learn more about the use of Waze for Cities partnership data for operations and planning.
Vehicle probe data includes information from mobile cellular devices, fleet vehicles with embedded Global Positioning System (GPS) devices, connected vehicle telematics, and mobile applications (cell phone, tablet, or other devices) that track user locations. These data, which are typically provided by third-party vendors who capture, manage, analyze, and anonymize
the data, provide cost-effective information across the transportation system. Transportation agencies are increasing adoption of the use of vehicle probe data to support the planning and operation of transportation systems. This chapter serves as a primer on vehicle probe data, including the sources of and applications for probe data, the challenges agencies need to address to use the data effectively, and the current and future opportunities for using vehicle probe data.
The vehicle probe data primer includes the following sections:
Visit Chapter 9 in Part II to learn more about the use of vehicle probe data.
Traffic incidents can have an impact on transportation system performance, causing delays and decreasing travel time reliability for the public and in the movement of goods. One of the essential responsibilities of transportation and public safety agencies is to detect, respond to, and clear traffic incidents as safety and quickly as possible through planned, coordinated, traffic incident management (TIM) practices. To understand the effect of traffic incidents and TIM
activities on system performance, agencies need data about incidents and incident response. The chapter provides lessons learned and recommendations for transportation agencies on improving the sharing, quality, and management of data for TIM use cases.
Traditionally, agencies have collected data on traffic incidents and TIM activities in a variety of ways, including crash reports, advanced traffic management systems (ATMS), computer-aided dispatch (CAD) systems, safety service patrol (SSP) programs, and traffic citation systems. Data on traffic incidents exist; however, many of the data are not available and ready for use in analyses. Therefore, agencies and TIM programs infrequently use these data to enhance their understanding of how they could improve TIM practices and policies to reduce system impacts of traffic incidents. There are three primary challenges that contribute to this limitation in traffic incident data:
To maximize the potential for data to improve TIM and to reduce the transportation system impacts of traffic incidents, agencies must improve the sharing, quality, and management of traffic incident data. The purpose of this chapter is to provide lessons learned and recommendations for transportation agencies on improving the sharing, quality, and management of data for TIM use cases. The chapter contains four sections:
Visit Chapter 10 in Part II to learn more about improving the sharing, quality, and management of data to support TIM use cases.
Roadway work zones can create hazardous conditions for motorists, pedestrians, and highway workers, and better, more accurate, and more timely data can reduce the risks of driving in work zones. Collecting, consolidating, and distributing this information, however, has been an ongoing challenge. The objective of FHWA’s Work Zone Data Exchange (WZDx) initiative was to develop and promote the use of a common specification to collect and share data on work zone activities. This chapter presents case studies from five agencies on their past, ongoing, and planned use of smart work zone technologies as a source of data for their WZDx feeds. The intended audience for this chapter includes state, regional, and local agencies seeking to use smart work zone technologies as well as those looking to establish a WZDx feed and those integrating real-time data with other data sources, such as work zone planning and tracking systems. This chapter will assist agencies by highlighting recent and ongoing efforts by peer agencies; summarizing their challenges, lessons learned, and recommendations; and providing additional resources.
Each of the five agencies featured in the chapter has prioritized different smart work zone devices, and each has taken a different approach to prioritizing the incorporation of data from smart work zone devices into their WZDx feeds. Each case study includes a discussion of the technologies used; the type of data collected; how the data are integrated, managed, and applied; and lessons learned from the projects. Each case study also includes available resources. The five case studies are as follows:
After the case studies, the chapter compares the agencies’ approaches to obtaining work zone data through different technologies, explores emerging issues, and compares the relationship between the WZDx and three related data feeds:
The intended audience for this chapter includes state, regional, and local agencies seeking to use smart work zone technologies as well as those looking to establish a WZDx feed and those integrating real-time data with other data sources, such as work zone planning and tracking systems. This chapter will assist agencies by highlighting recent and ongoing efforts by peer agencies; summarizing their challenges, lessons learned, and recommendations; and providing additional resources.
Visit Chapter 11 in Part II to learn more about using smart work zone devices for work zone data feeds.
Shared mobility is the shared use of vehicles to provide travelers with short-term access to a travel mode on an as-needed basis. Its scope includes micromobility services such as bikesharing and electric scooter services as well as car sharing, microtransit, paratransit, transportation
network companies (TNCs), and traditional ride hailing (taxi) services. Shared mobility services have grown rapidly within the past few years. Just as some cities were taken by surprise by TNCs and struggled to put regulatory frameworks in place, many localities have struggled with regulating dockless bikes and scooters. There is a clear need for public agencies to have data to better understand how all these services fit into the overall transportation network. The purpose of this chapter is to provide public-sector agencies with curated reference material to help plan for, manage, and use shared mobility data.
The chapter on shared mobility data includes the following information:
The primary intended audience for this chapter includes both management and staff of public agencies responsible for shared mobility, including those that use data for regulating shared mobility operations and those that use this type of data for broader planning purposes, such as implementing bike lanes or integrating shared mobility with transit operations.
While the chapter summarizes more than 40 references for data management, it is not a comprehensive encyclopedia; rather, it provides an overview on the data management needs related to each topic. The chapter contains modules so that readers can easily locate the specific sections relevant to their topics of interest. The bulk of the resources address micromobility; however, many of these resources have information or recommendations that are equally applicable to other shared mobility services, such as TNCs, ride hailing, and microtransit. Following are the topic areas covered:
Each reference is also tagged with the type of resource it represents. Resource types include
The chapter begins by providing a brief discussion of each topic area. Next, the primary section of the chapter presents a reference sheet for each resource. The reference sheet identifies the title and author, the type of resource, where to obtain it, the topic areas covered, a summary of the content, and a more detailed description. In some cases, links to additional, closely related resources are also provided.
Visit Chapter 12 in Part II to explore resources on shared mobility data.
Public agencies have a need to obtain data from shared mobility providers. Some of these data may be sensitive because they could reveal either personal details of users of the services or proprietary data, including trade secrets of the service providers. By implementing appropriate policies and procedures, agencies can protect sensitive data while using the data to meet their goals and objectives. Building on the information in Chapter 12, “Shared Mobility Data: A Resource Guide,” the chapter on managing sensitive shared mobility data gives public agency managers and staff information on issues relating to the protection of sensitive information that agencies may gather as part of their shared mobility programs.
This chapter provides recommendations and specific guidelines for protecting sensitive information related to shared mobility services, including data associated with specific individuals and data that mobility service providers consider proprietary. The goal of the chapter is not to turn readers into information privacy experts, but rather to provide sufficient depth to familiarize readers with the issues and approaches for resolving them. The chapter aims to enable readers to have more productive discussions, both with interested parties (e.g., mobility service providers, academic researchers, and public advocacy groups) and with domain experts (e.g., IT staff, cybersecurity specialists, and lawyers). This chapter can easily be read in its entirety or used as a reference to find information on specific topics as they arise.
The chapter lays out a process for managing sensitive shared mobility data and gives recommended best practices for each step. The steps include the following:
Examine what other agencies have found a need to collect. Another question to consider is whether the agency needs raw, detailed data in every case, or whether some of the use cases could be met by having the mobility service providers send only preaggregated or preobfuscated data.
The scope of the policy that the agency develops should include all types of sensitive data as well as how the data are exchanged and internally handled by each entity (e.g., agency, law enforcement agencies, other public agencies, researchers, general public).
Those responsible for shared mobility programs should work with their agency’s IT department and others responsible for agencywide security and privacy policies to ensure both that they address the specific needs for shared mobility programs and that the shared mobility programs comply with all relevant agency requirements. By following these practices, agencies can appropriately protect sensitive data, including both personal private data and proprietary data.
Visit Chapter 13 in Part II to learn more about managing sensitive shared mobility data.
Conflation is the process of identifying common points and references with which to reconcile two or more geodatasets across overlapping areas. Because of differences in scale, resolution, and, sometimes, accuracy or convention, data referring to the same location often do not have the same geographic reference and are challenging to combine. This leads to the definition of “near-enough” criteria for expecting that two references represent the same feature. The use of near-enough criteria, while it allows several corresponding features to be merged effectively, is not perfect. LinkerAT is a prototype open-source software tool for transportation agencies and their partners that conflates two different sources of roadway network data.
Specifically, LinkerAT addresses the problem of relating data from two sources:
These network datasets describe roads and connections for all 50 states and the District of Columbia. The RITIS/TMC network is used to segment and report real-time vehicle speed data. ARNOLD is a specification developed by FHWA to inventory each state’s roadway network in terms of road configuration and condition by segment. ARNOLD serves as the mapping foundation of the HPMS. Due to the difference in the intended use and purpose of the RITIS/TMC network and ARNOLD, segmentation of the networks and representation of network features vary significantly.
LinkerAT uses a combination of geographic and thematic cross-referencing to identify how the ARNOLD network represents parallel elements in the RITIS/TMC network. To support LinkerAT and provide a fully open-source software implementation, the networks are input to
the program in GeoJSON format, which is the standard format used for LinkerAT. LinkerAT uses a series of algorithms to identify corresponding elements in the source (RITIS/TMC) and target (ARNOLD) networks. The process involves a series of steps or phases executed to maximize the likelihood of a solution. Much of the process is automated, with the program running in the background and tabulating a solution set. As the program runs, it learns more about the network and how elements can be cross-referenced, so multiple program runs are executed sequentially to effectively apply this new knowledge.
The LinkerAT program code, software tool, documentation, and example datasets for use are in the public domain and can be downloaded for free from the NCHRP 08-119 website (https://data.transportationops.org/linkerat-guidance-and-demonstration-improved-conflation-and-geodata-reference-process). LinkerAT includes all necessary libraries and support to run the program on a standard Windows computer and requires no special libraries or proprietary software.
Implementing LinkerAT for a given state requires both knowledge of principles of geographic information systems (GIS) and associated tools and proficiency in JavaScript software development to customize the LinkerAT tool for a particular state.
Visit Chapter 14 in Part II to learn more about LinkerAT.
The capabilities of data management systems have changed significantly since the rollout of the U.S. DOT pilot Integrated Corridor Management (ICM) Initiative in 2006. This chapter is meant to provide an update on trends in effective data management principles and practices and use of data for ICM systems to help inform transportation practitioners about key areas in the conceptualization, planning, and design of ICM systems. The chapter focuses on research and projects conducted since 2019. Two recent ICM projects are highlighted:
On the basis of a review of recent documents and findings from interviews with the Tennessee and Pennsylvania DOTs, the study team recommended the following:
Visit Chapter 15 in Part II to learn more about opportunities and challenges to improve data use on ICM Systems.
There are multiple use cases for connected vehicle (CV) data for transportation planning, safety management, and operations. This chapter is a primer on establishing uses of CV data for these purposes. The scope of the primer includes an overview of CV systems that produce data; the expectations of that data; methodologies for using that data; and planning, safety, and operations use cases. The chapter discusses these topics in the context of two types of transportation management: vehicular system management and pedestrian system management.
The chapter helps bridge the gap between CV and ITS, and identifies opportunities for developing CV infrastructure while advancing the quality and resolution of data supporting transportation system management. The chapter contains information on the following topics:
The intended audience for this chapter includes public agency staff, consultants, researchers, and educators involved in work related to
This audience includes, but is not limited to, directors of transportation management centers, Transportation Systems Management and Operations (TSMO) professionals, project managers, transportation agency planners and engineers, data analysts, and researchers.
Visit Chapter 16 in Part II to learn more about using CV data for transportation system management.
NACTO (National Association of City Transportation Officials) and IMLA (International Municipal Lawyers Association). (2019). Managing Mobility Data. Retrieved June 4, 2020, from https://nacto.org/wp-content/uploads/2019/05/NACTO_IMLA_Managing-Mobility-Data.pdf.
NUMO (New Urban Mobility alliance), NABSA (North American Bikeshare & Scootershare Association), and OMF (Open Mobility Foundation). (2021). Privacy Principles for Mobility Data. Retrieved March 16, 2022, from https://www.mobilitydataprivacyprinciples.org/.