Page 41

Suggested Citation: "Section 8 - Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2022. Evaluation of the Asphalt Binder Quality Tester. Washington, DC: The National Academies Press. doi: 10.17226/26603.

SECTION 8 Uncertainty

The analysis presented in this evaluation is a prospective assessment of the potential benefits of the ABQT if adopted as a screening tool to supplement the existing asphalt binder testing being conducted throughout the supply chain. As such, there is uncertainty around the magnitude of these potential net benefits. Throughout this evaluation report, the evaluation team has flagged several parameters that might have the highest level of uncertainty. These include several key input parameters in the probabilistic model that was used to calculate the annual benefits of using the ABQT device:

The ABQT device is 80% effective.
50% of samples collected by state DOTs are tested (the remaining 50% are not).
50% of tested-failed samples have no adverse effect on paved roads.
The use of an out-of-specification binder increased maintenance costs by 50%.

The following provides a sensitivity analysis for each of these key parameters indicating how changes affect Scenario 1 annual benefits. Note that in each of the tables presented in the following, the impacts in bold are the annual Scenario 1 benefits used throughout the evaluation report ($9,441,000 as shown in Table 6-3).

To reflect the uncertainty of the effectiveness of the ABQT in predicting full test failure, the evaluation team included results for 70%, 80%, and 90% ABQT effectiveness (see Table 8-1). The evaluation team’s base assumption of 80% effectiveness of the ABQT device provides a point estimate of $9.4 million per year in maintenance and removal cost reductions if all states were to adopt the ABQT as a screening tool. The 70% and 90% values provide a range of annual benefits from $8.7 to $10.2 million, which is approximately plus or minus 8%.

Increasing the percentage of samples received by DOTs that are tested reduces the potential benefits of using the ABQT as a screening tool. If 90% of samples are tested (as opposed to 50%), benefits are reduced by 18% (see Table 8-2).

As shown in Table 8-3, increasing the percentage of failed tests that lead to no increased maintenance costs reduces the annual benefits of using the ABQT as a screening tool. An increase or decrease of 20% results in about a 10% decrease or increase in annual benefits.

Varying the percentage increase in maintenance costs associated with out-of-specification binder has the most significant impact of the parameters investigated. As shown in Table 8-4, a plus or minus 20% change in increased maintenance costs leads to approximately a plus or minus 38% change in annual benefits.

Page 42

Table 8-1. Varying ABQT effectiveness: annual benefits for Scenario 1.

Parameter	Percentage Out-of-Specification Binder Placed	Annual Lane-Miles Paved with Subpar Binder	Maintenance Cost	Removal Cost	Economic Impact: Change from Business as Usual (BAU)
BAU (without ABQT):	0.430%	6,036	$23,998,000	$1,420,000
ABQT Effectiveness		DOT Adoption of ABQT
70%	0.288%	4,003	$15,770,000	$933,000	$8,715,000
80%	0.275%	3,829	$15,084,000	$892,000	$9,441,000
90%	0.263%	3,655	$14,399,000	$852,000	$10,167,000

Note: Cost and benefit numbers were rounded to the nearest $1,000 due to the precision of the evaluation team’s underlying assumptions. This may result in minor discrepancies due to rounding.

Table 8-2. Varying the percentage of samples tested: annual benefits for Scenario 1.

Percentage of Collected Samples Tested	Scenario	Percentage Out-of-Specification Binder Placed	Annual Lane-Miles Paved with Subpar Binder	Maintenance and Removal Cost	Economic Impact: Change from Business as Usual (BAU)
50%	BAU (without ABQT)	0.433%	6,036	$25,418,000	−
50%	DOT Adoption of ABQT	0.275%	3,829	$15,977,000	$9,441,000
70%	BAU (without ABQT)	0.409%	5,691	$23,965,000	−
70%	DOT Adoption of ABQT	0.265%	3,690	$15,396,000	$8,569,000
90%	BAU (without ABQT)	0.384%	5,346	$22,513,000	−
90%	DOT Adoption of ABQT	0.255%	3,551	$14,799,000	$7,714,000

Note: Cost and benefit numbers were rounded to the nearest $1,000 due to the precision of the evaluation team’s underlying assumptions. This may result in minor discrepancies due to rounding.

Table 8-3. Varying the percentage of failing samples with no adverse impact on pavement costs: annual benefits for Scenario 1.

Percentage of Failing Samples With No Adverse Impact on Pavement Costs	Scenario	Percentage Out-of-Specification Binder Placed	Annual Lane-Miles Paved with Subpar Binder	Maintenance and Removal Cost	Economic Impact: Change from Business as Usual (BAU)
30%	BAU (without ABQT)	0.508%	7,070	$29,775,000	−
30%	DOT Adoption of ABQT	0.365%	5,082	$21,206,000	$8,569,000
50%	BAU (without ABQT)	0.433%	6,036	$25,418,000	−
50%	DOT Adoption of ABQT	0.275%	3,829	$15,977,000	$9,441,000
70%	BAU (without ABQT)	0.359%	5,001	$21,060,000	−
70%	DOT Adoption of ABQT	0.185%	2,576	$10,748,000	$10,312,000

Note: Cost and benefit numbers were rounded to the nearest $1,000 due to the precision of the evaluation team’s underlying assumptions. This may result in minor discrepancies due to rounding.

Page 43

Table 8-4. Varying the percentage increase in maintenance costs: annual benefits for Scenario 1.

Percentage Increase in Maintenance Costs due to Subpar Binder	Scenario	Percentage Out-of-Specification Binder Placed	Annual Lane-Miles Paved with Subpar Binder	Maintenance and Removal Cost	Economic Impact: Change from Business as Usual (BAU)
30%	BAU (without ABQT)	0.433%	6,036	$15,819,000	−
30%	DOT Adoption of ABQT	0.275%	3,829	$9,943,000	$5,876,000
50%	BAU (without ABQT)	0.433%	6,036	$25,418,000	−
50%	DOT Adoption of ABQT	0.275%	3,829	$15,977,000	$9,441,000
70%	BAU (without ABQT)	0.433%	6,036	$35,017,000	−
70%	DOT Adoption of ABQT	0.275%	3,829	$22,011,000	$13,006,000

Note: Cost and benefit numbers were rounded to the nearest $1,000 due to the precision of the evaluation team’s underlying assumptions. This may result in minor discrepancies due to rounding.