CHAPTER 6

Real-World Implementation Examples

To illustrate the real-world application of this data fusion framework, here are two use cases:

1. Utah DOTs fusion of point sensor speed data and probe data to create a more accurate statewide speed dataset (Zhang and Yang 2021).
2. The CATT Lab’s use of volume data from point sensors fused with probe-based speed data for computing user delay and then further fusing incident data from CVs and other sources to identify the causes of congestion on roadways (Franz, Pack, et al. 2022).

Both use cases showcase data fusion of probe and point-sensor data. While both follow the broad data fusion framework steps as described in this report, both implement their own, unique use case-specific fusion algorithms as shown in Step #3 of the framework (Figure 20).

For example, Utah DOT implements a set of ML algorithms in Step #3, and the CATT Lab implements a cooperative data fusion algorithm. This Section of the report highlights those differences during a detailed explanation of how these two agencies approached data fusion. For each use case, the following information is provided:

• Purpose of the use case
• Data sources
• Data fusion framework steps that the use case implemented

6.1 UDOT Speed Data Quality Improvement Use Case

The first use case is from Utah DOT and researchers Xianfeng Terry Yang, Zhao Zhang, and Brian Azin. The following information is heavily referenced in Utah DOT reports UT-20.15 and UT-21.03. The research team for this paper thanks the authors of the Utah DOT reports for their permission to present their work and reprint images and code.

6.1.1 Purpose

UDOT has access to agency-installed point-sensor data that reports speed data. The agency also procures probe-based speed data from the private sector. At the time of development of this research project, UDOT determined that generally, point-sensor data provided more accurate speed measurements than probe data due to the probe data being biased. Probe data bias is a result of low penetration rate of probe data (around 2% of total traffic, according to the UDOT use case researchers in March 2021). The bias is based on the type of vehicles which are capable of reporting their GPS location and their geographic distribution, as well as length of the segments on which data are reported. Figure 21 shows how the length of segments on which data are reported results in probe data speeds remaining unchanged over the length of those segments, while the sensors within those segments indicate variations in speed between individual locations.

However, UDOT also realized that ubiquitous probe data coverage allows for statewide monitoring that is not possible using only the point sensors.

Source: Zhang and Yang 2021

To address this challenge, UDOT developed a methodology to process their point-sensor data and fuse it with probe vehicle data using a set of machine learning models to produce a more reliable estimate of statewide traffic speed and flow patterns.

6.1.2 Data Sources

The data used in the fusion process came from two sources:

• Point-sensor data stored in the UDOT Performance Measurement System (PeMS) platform.
• Probe-based speed data stored in the ClearGuide platform.

The UDOT PeMS platform collects point-sensor data consisting of vehicle counts and occupancy in 20-second intervals. Using these two variables, the system then calculates the speed at that location and aggregates the measurements to 5-minute intervals for use in operations.

The ClearGuide platform stores and makes available third party-provided statewide 5-minute probe speed data which is used in operations, specifically in locations that lack point sensors.

6.1.3 Data Fusion Framework Steps

6.1.3.1 Pre-Processing

Prior to data fusion, UDOT performs a set of rigorous cleaning steps to ensure the highest possible quality of sensor data to be used in the ML algorithms. While ML models are good at capturing the stochastic nature of traffic data, they depend heavily on quality and quantity of training data. This step is important because poor quality or sparse training data will severely degrade the performance of the ML algorithm (Figure 22).

Physical sensor devices may become unreliable in reporting accurate data due to lack of maintenance and calibration, or due to field communications issues. The initial screening step uses

If any of the three metrics is zero, then the other two metrics must be zero. If the data record fails any one of these checks, the record is discarded for the purpose of further analysis.

Piecewise Quality Check

After the records that failed the primary screening are removed, the pre-processing algorithm evaluates the remaining data using the detector AEVL distribution and Kolmogorov-Smirnov (KS) test to uncover erroneous records. The AEVL is defined as:

$AEV{L}_i\approx \frac{3o_i{v}_i}{q_i}$

and AEVL distribution of a detector is measured over a specified time interval (e.g., 1 hour). This distribution is then compared to the AEVL distribution of upstream and downstream sensors. The K-S test evaluates the likelihood of these two sample distributions being drawn from the same probability distribution. In other words, the K-S test determines whether the AEVL of the examined sensor statistically makes sense in context of its upstream and downstream neighbors. If the AEVL distributions are statistically different, the resulting difference provides insight in changes in data patterns to flag erroneous records.

Continuous Quality Check

The next check uses multiple comparisons with the best (MCB) method to identify potential bad records not captured by previous checks. The key feature of the MCB method is that it compares AEVL of each lane of traffic at neighboring stations and uses the calculated confidence interval (CI) for statistical assessment. This method requires data collected over a sufficiently long period of time (e.g., 1 week) so that data can be grouped into equal segments (e.g., 1 hour) and the mean and variance of the group can be computed. If the CI contains zero, that indicates that there is no significant difference of AEVL at neighboring stations.

UDOT determined three possible outcomes based on the results of the piecewise quality check and continuous quality check:

1. If sensors show no significant errors in both steps, there is a 95% CI that no other error is present.
2. If the target sensor shows no errors during the continuous comparison check but does indicate some in the piecewise check, it can be concluded that there might be a change in traffic flow patterns such as a special event, or there might be a fixed error within the detector.
3. If the sensor only fails the continuous comparison check, an additional investigation is needed due to two possible scenarios: a) the target sensor may have an error, or b) the adjacent station sensor may not be in proper working order.

If a sensor is found to be faulty during the additional investigation, it is excluded from further quality checks, and the next nearest sensor is used instead.

Example Results from UDOT Implementation Case

This pre-processing algorithm of Figure 23 was applied to an example case to demonstrate the results. The researchers evaluated sensor data on I-15 northbound from January 12, 2019, to January 19, 2019. Note that the UDOT use case implementers use the term “detector” in Figure 23 in the same way the term “sensor” is used in this report.

The demonstration area included 61 sensors across roughly 9.5 miles of I-15 (between mileposts 302.75 and 312.16) as shown in Figure 24.

The single variable thresholds were set as follows:

• Flow: 180 vehs/5 mins/lane
• Speed: 90 mph
• Occupancy: 95%

The results were as follows (Figure 25). Note that the percentages don’t add up to 100% as errors can occur multiple times throughout different stages of the algorithm.

• No error: 85.41%
• Missing data: 10.46%
• Primary error: 1.39%
• Piecewise comparison: 3.62%
• Continuous comparison: 2.74%

The results showed some variation in error rates based on sensor location (e.g., ramp vs. mainline vs. HOV) as shown in Figure 26.

6.1.3.2 Spatial and Temporal Alignment

In this step, the pre-processed sensor data from PeMS are temporally and spatially aligned with probe speed data from ClearGuide (Figure 27). Mainly, the study identified a set of sensors along the corridor and corresponding probe data segments between the first and last detector as shown in Figure 28. The probe data are reported on roadway segments defined using the TMC standard. The latitude and longitude of each sensor is compared to the start and end points of TMC segments along the corridor to identify an appropriate TMC segment that sensor belongs to. Both sensor speed and flow, and probe speed data are reported in 5-minute intervals over the study time period (14 days). For each matching 5-minute interval, a record for a sensor and corresponding TMC segment is captured for a total of 4,032 records for each sensor and TMC segment.

Depending on further use case needs for the pre-processed sensor data, additional steps may be taken. In the case of UDOT’s sensor and probe speed data fusion using ML algorithms, the final pre-processing step includes normalization of raw speed and flow data. This normalization is necessary because of the sufficiently different scales for the two metrics. Normalization allows

UDOT uses several ML algorithms and measures their performance to select the best algorithm for each case.

The SVM algorithm is a supervised model that can be used in a wide variety of applications including regression and prediction. The RF algorithm uses a multitude of decision trees during training to generate predictions that can be averaged for each tree and returned as a result. The GBDT is an algorithm that relies on boosting weak prediction models such as simple decision trees into a one strong prediction. The XGBoost algorithm is a software library that implements a more regularized form of gradient boosting that performs very fast and tends to be better at model generalization. Finally, an ANN uses a connected network of artificial neurons (nodes) that communicate to each other based on the input values and activation functions. ANNs typically contain an input layer, a number of hidden layers, and an output layer, where each layer groups nodes performing a particular type of transformation.

The performance of each of these algorithms can be measured using various statistical indicators. UDOT researchers used RMSE, MAPE, and MAE to evaluate the accuracy of the estimated traffic speed and flow.

The UDOT team implemented the pure ML approach along a two-and-a-half-mile stretch of the I-215 corridor in Salt Lake City, UT. They collected 2 weeks of sensor and probe data from January 7, 2019, to January 20, 2019, and tested all trained models at a location where they had observed sensor data to evaluate accuracy of the estimated values.

The initial implementation did not use probe data as input, and as a result all models performed poorly with all statistical indicators at an unacceptable level:

• Flow indicators:
  • RMSEs > 69 veh/5-min
  • MAPEs > 50%
  • MAEs > 41 veh/5-min
• Speed indicators:
  • RMSEs > 3.35 mph
  • MAPEs > 2.6%
  • MAEs > 1.8 mph

Figure 30 shows the performance of pure ML algorithms without probe data. The observed flow and speed are not accurately reflected by the pure ML algorithms without probe data, failing to capture the variations in observed values.

The next implementation included probe data as an additional input. All models performed significantly better than without probe data, resulting in the following statistical indicator values:

• Worst-case flow indicators:
  • RMSE = 36.26 veh/5-min
  • MAPE = 35.16%
  • MAE = 26.28 veh/5-min
• Worst-case speed indicators:
  • RMSE = 3 mph
  • MAPE = 3.03%
  • MAE = 2.2 mph

Figure 31 shows the performance of pure ML algorithms with introduction of probe data. In this case the observed flow and speed values and their variations are more accurately captured by the pure ML algorithms with probe data input.

Hybrid ML Approach

The accuracy of ML models depends heavily on the quality and quantity of training data. If training data are noisy or small, the performance of the ML algorithms may not be acceptable. The hybrid ML approach is intended to provide an alternative input data in cases where probe data are not available. In other words, this approach addresses the issue of the low performance of the pure ML models without probe data by introducing an alternative input dataset that can enable estimation in locations where there is no observed sensor and no observed probe data. The alternative input data are generated using the second-order macroscopic TFM. This model uses observed flow and speed at upstream and downstream stations, on-ramps, and off-ramps to estimate the traffic speed evolution on the target freeway section.

The hybrid ML approach then uses the observed sensor data, the second-order macroscopic TFM estimates, and observed probe data if they are available as training inputs into previously described ML algorithms.

The UDOT research team implemented the hybrid ML approach using the same dataset and parameters as what they used in the pure ML implementation. The results indicate that the pure ML implementation with probe data performs better than the hybrid ML approach. However, the performance of the two approaches is close, and the hybrid ML approach compared to the ground truth indicates that this approach could be used to accurately estimate flow and speed in the absence of probe data.

6.1.3.4 Quality Control

The accuracy of ML models depends heavily on the quality and quantity of training data (Figure 32). If training data are noisy or small, the performance of the ML algorithms may not be acceptable. This means that care must be taken when gathering and preparing training data to ensure that a sufficient volume of high-quality data are available.

However, even with the same volume and quality of data, various ML algorithms perform differently. The key is to evaluate outputs and select appropriate ML models and adjust them accordingly.

For example, in the UDOT use case implementation, each ML model’s performance, according to the statistical indicators, varies based on input data and can perform well for certain metrics in comparison to other models. Table 6 summarizes the performance of various ML models implemented in the UDOT use case. First, the relatively low values of TFM data’s RMSE, MAPE, and MAE show that TFM data could be a potential replacement of probe data when developing ML models for TSE. The yellow highlighted values in Table 6 show the performance of hybrid models, while the blue highlighted values show the performance of hybrid models with probe data indicating a generally better performance when probe data are used in the hybrid model.

The performance of models is also highly dependent on the location. In general, a well-performing model may only be applicable to nearby locations and may not be transferable to other freeway segments. This means that outputs must be verified, and the model adjusted accordingly as the model is implemented in various locations.

Finally, ML models may need to be retrained when the original data become dated. This means that, to maintain accuracy of estimates, the maintenance of these models must be a continuous effort.

6.1.4 Lessons Learned and More Information/Details

As was described in Chapter 4, not all fusion methods and use cases may yield results that warrant the investment. The Utah DOT example, when compared to the PennDOT use case

6.1.4.1 Example Implementation Code for the Continuous Model

6.1.4.2 Example Implementation Code for RF Model

6.2 CATT Lab User Delay Cost and Causes of Congestion Fusion Use Case

The second use case comes from the University of Maryland CATT Laboratory. Special thanks are extended to their research team for sharing their research papers and presentations in support of this publication.

6.2.1 Purpose

The CATT Laboratory, in collaboration with the Bureau of Transportation Statistics, TETC, and a nation-wide group of transportation engineers from 35 state DOTs, nine metropolitan planning organizations (MPOs), and two universities, developed a data fusion methodology whose initial purpose was to rebuild the 2004 national Congestion Pie Chart, shown in Figure 33. This original pie chart, while innovative at the time, was largely built off modeled data and was a single pie chart that represented the entire country (Cambridge Systematics and TTI 2004).

By 2019, it was realized that there were now real-world data that would allow the research team to develop a new congestion pie chart that would not need to rely on modeled data and would be able to compute the causes of congestion in every county of every state in the country—showing local causes. The result was a revised Causes of Congestion Pie Chart for the National Highway System that showed different results and provided new categories of congestion including

Source: Cambridge Systematics and TTI (2004)

Source: University of Maryland CATT Laboratory

Table 7. Causes of congestion data sources and size.

Some TMC segments may span across two or more defined volume link locations (if AADTs were provided by link segments instead of points), and vice versa (as shown in Figure 40). In order to obtain a single AADT measurement for TMCs that fall under this case, the AADT of the overlapped sensor locations must be weighted by the distance of the portion of the TMC that falls into the range of each link location.

Additional calculations are then made to break down the day of week ADTs into hourly profiles. The forementioned methodology for this follows closely the one defined by TTI in their 2021 Urban Mobility Report, Appendix A: Methodology (2021).

Event data from each source typically came with a latitude/longitude coordinate, road name, direction, start time, and an end time. Each event had to be map-matched with a specific TMC segment. At a high level, the methodology for incident and event map-matching (including for incidents derived from probe vehicles) is as follows:

1. Search for the TMC segments closest to the latitude/longitude coordinates of the event.
2. Of the returned results, identify TMC segments that have road names (or aliases) closely matching those associated with the event. For example, I-95.
3. Of the remaining results, identify the closest segment that is in the same direction of travel as the incident report.

6.2.3.3 Use Case-Specific Fusion Algorithm

The bulk of the point-sensor and probe data fusion for this use case occurs in steps one and two of the fusion framework. After these basic fusion techniques are employed, the remaining algorithms are largely computational in nature and are simply using the results of the prior fusion to make some decision (Figure 41).

While spatial data fusion has already occurred in Step 2, there is additional spatial and temporal data checking that occurs later in this methodology, described in subsequent sections.

From the paper titled Identifying and Quantifying the Causes of Congestion—A New Tool for the Nation’s States and Counties to Better Understand the Contributing Factors to Traffic by Franz, et al. (2022), the following methodology is described and reprinted with minor edits and clarifications with permission of the author.

The last step was to assign the total delay for each bottleneck to a causation category. As shown in Figure 33, this process began by looking at the temporal congestion pattern. If the delay pattern was similar to the historical pattern for the time of day and day of week of the target bottleneck, the bottleneck was categorized as a recurrent bottleneck. However, if the delay pattern was different, additional causes were considered. To identify recurrent patterns, the variations between the mean free flow travel time and the mean historical travel time were analyzed. In order to do this, you calculate the mean free flow travel time and the mean historical travel time across the TMC segments in the disruption queue for each minute of its duration. Once both values are calculated, the ratio of the free flow travel time to the historical travel time is taken. Similar to how a disruption is activated at 60% of the free flow speed, the ratio of the free flow travel time to the historical travel time is compared to a threshold of 0.60. Specifically, the recurrent nature of a disruption is classified based on the following conditions:

Source: I-95 Corridor Coalition (2019)

It is worth noting that other causes can be paired with recurrent congestion to exacerbate delays. To consider such scenarios when the recurrent pattern is worse than normal, the following rules were applied:

For all non-recurrent bottlenecks and recurrent bottlenecks plus additional cause(s), logic was developed to match the bottleneck location with a potential cause. The logic for each causation category was based on spatial and temporal boundaries as described below.

Incidents

The event data source is used to identify incident events to be matched to a disruption based on a mapping of event types to the incident category. The following spatial-temporal conditions are used to match the incidents (crashes), and events (disabled vehicles, debris, etc.).

• Spatial:
  • Condition 1: An incident/event TMC segment must be within the TMC segments in the queue of the disruption.
• Temporal:
  • Condition 2: The incident event created timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.
  • Condition 3: The incident event closed timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.

For an incident/event to be matched, spatial Condition 1 AND either temporal Condition 2 or Condition 3 must be met.

Weather

Weather identification integrates two data sources: a) Waze, CV, or ATMS events and b) NOAA radar that has been conflated to the roadway. The Waze, CV, or ATMS data source is used to identify weather events to be matched to a disruption based on a mapping of event types to the weather category. The NOAA data source is used to identify precipitation (snow, sleet, and rain) that occurs during the disruption.

A. The following spatial-temporal conditions are used to match the weather events.

• Spatial:
  • Condition 1: A weather event TMC segment must be within the TMC segments in the queue of the disruption.
• Temporal:
  • Condition 2: The weather event created timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.
  • Condition 3: The weather event closed timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.

For a Waze, CV, or ATMS weather event to be matched, spatial Condition 1 AND either temporal Condition 2 or Condition 3 must be met.

B. The following spatial-temporal conditions are used to match NOAA radar precipitation.

• Spatial:
  • Condition 1: The precipitation reading TMC segment must be within one of the TMC segments in the queue of the disruption.
• Temporal:
  • Condition 2: The precipitation reading timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.

For a precipitation reading to be matched, spatial Condition 1 and temporal Condition 2 must be met.

Work Zones

The Waze, WZDX, ATMS, and/or Lane closure permitting system agency data sources are used to identify work zone events to be matched to a disruption based on a mapping of agency event types to the Work Zones category. The following spatial-temporal conditions are used to match the work zone events.

• Spatial:
  • Condition 1: A work zone event TMC segment must be within the TMC segments in the queue of the disruption.
• Temporal:
  • Condition 2: The work zone event created timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.
  • Condition 3: The work zone event closed timestamp must be between the disruption start timestamp minus 5 minutes and the disruption closed timestamp.
  • Condition 4: The disruption start timestamp must be between the work zone event created and the work zone event closed timestamp.

For work zone events to be matched, spatial Condition 1 AND either temporal Condition 2 or Condition 3 or Condition 4 must be met.

Holiday Travel

A Holiday Travel Calendar is used to identify holiday travel days to be matched to a disruption. The following temporal condition is used to match the holiday travel days.

• Temporal:
  • Condition 1: The disruption start timestamp must be the same day as the holiday travel day.

Signals

The Open Street Map (OSM) Traffic Signal data source is used to identify signal locations that are matched to a disruption. The following spatial condition is used to match the OSM signals.

• Spatial:
  • Condition 1: The signal intersection TMC segment must be within the first two TMC segments in the queue of the disruption (i.e., the head of the disruption).

Multiple Causes

These are bottlenecks in which two or more congestion categories are present during the disruption. The delay associated with the multiple causes category was found to be significant in the preliminary exploratory analysis results, and it was determined that it should be further decomposed. Based on a sensitivity analysis in which CATT Lab sampled four months of data (February, May, August, and October) across seven states (AZ, CO, MD, MI, OH, VA, and PA),