5 Feedback on Proposed Methodology Components

5.1 Introduction

To obtain feedback on the proposed methodology components, the team conducted additional interviews and focus groups with owner and industry practitioners.

5.1.1 Owner Outreach

The research team invited 18 DOT representatives to participate in an owner focus group addressing project risks and strategies to promote competition. Five owner representatives were able to coordinate their schedules to participate in the roundtable discussion. All had significant APDM experience as shown in Table 5.1.

Table 5.1: Owner Representatives

In facilitating the roundtable discussion, the research team shared the methodology components discussed in Chapter 4, and asked each of the owner representatives to identify:

• What practices/tools they have used to align risk perception and promote competition.
• Any additional tools or practices that they have successfully used that were not identified by the team; and
• What content they would like to see in the Guide to be developed under this research.

Feedback and findings from this owner roundtable are summarized in Section 5.2 of this memorandum; notes from the focus group are included in Appendix H.

5.1.2 Industry Outreach

As discussed below, two forms of industry outreach took place:

• The team first conducted one-on-one interviews with various regional/local contractors to supplement its earlier outreach efforts, which focused on larger, national and international firms. (Note that these interviews took place in advance of the owner’s roundtable discussed in Section 5.2 to ensure that the team would be able to share a more complete view of industry perceptions with the owner participants.)
• Secondly, the team facilitated an industry roundtable, similar to that conducted with the owner practitioners, to seek feedback on the proposed methodology components.

Local / Regional Contractors

At the Interim Meeting held on February 27, 2024, the team agreed to conduct additional outreach to contractors who pursue more “bread and butter” DOT projects to ensure the Guide captures the conditions and concerns facing local and regional contractors as well as the larger, national and international contractors/developers whose perspectives were previously obtained as part of the Phase I research. Following the discussion at the Interim Meeting, the Panel provided the team with additional regional industry contacts. The team conducted one-on-one interviews with seven industry representatives recommended by members of the Panel. Five were contractors and two provided engineering / design services.

Findings from these interviews are discussed in Section 5.3.1.

Industry Roundtable

The team facilitated a roundtable discussion with the industry stakeholders. In addition to presenting the proposed methodology components, the team also posed the following questions to the group, which were designed to determine if their perceptions regarding risk allocation by owners had changed or evolved during the months that had elapsed since their original Phase 1 interviews, and if they continued to see progressive APD methods as the path forward, despite some concerns regarding the track record of such methods from owners and local/regional contractors:

• What risks do owners tend to misallocate?
• What tool(s) are most beneficial to helping identify/manage risk?
• What are your perspectives on the following aspects of progressive project delivery methods:
  • Attractiveness of progressive methods vs fixed price
  • Owners’ understanding of risk and risk allocation
  • Process of negotiating risks and construction price
• What are your perspectives on various risk “sharing” strategies (e.g., allowances, risk pools, deductible schemes, relief mechanisms)?
• Are owners learning anything through progressive approaches that can benefit other delivery methods?
• What influence does the surety industry have on contract size, risk allocation, choice of delivery method?

• What success factors would you suggest to owners to improve competition, reduce costs, and result in better project outcomes?
• What are potential showstoppers that would discourage you from pursuing a project?

Feedback and findings from this owner roundtable are summarized in Section 5.3.2; notes from the focus group are included in Appendix H.

5.2 Owner Outreach: Observations and Findings

As discussed in Section 5.1.1, the research team facilitated a roundtable discussion with five owner practitioners to discuss and validate the proposed methodology components and obtain feedback on the desired content for the associated Guide. A summary of key discussion items is provided below.

5.2.1 Methodology Components

For each stage in the project delivery lifecycle, the research team sought feedback regarding:

• Strategies and tools identified in the research that were most likely to mitigate risk and promote competition; and
• Any additional strategies and tools not directly identified in the research that the participants have successfully applied to improve alignment on project risks, clarify roles and responsibilities, and promote competition.

Table 5.2 summarizes the feedback provided by the owner practitioners regarding practices and tools that can be used during each stage of the project lifecycle.

Table 5.2: Practices and Tools

Throughout the discussion of the project stages, the owners also shared their thoughts regarding the use of progressive delivery methods such as PDB and CM/GC. Although they felt that such methods represented a valuable tool to help derisk large and complex projects, they had concerns regarding the lack of competitive tension and the ability to arrive at a fair construction price when using these methods. Observations from these discussions include the following:

• The owners generally perceived progressive methods as being less effective than fixed price DB at:
  • Incentivizing innovation (due to a lack of competitive tension; no competition of solutions) and
  • Minimizing the execution schedule duration (due to a perceived lack of urgency and potential for design churn)
• Owners continue to grapple with the negotiation of construction costs for CM/GC and PDB projects. There was a general recognition that DOT owners are often at a disadvantage when it comes to negotiating construction costs.
• Doubts were raised regarding the overall competence of the ICE industry (abilities vary widely). The general perception was that more standardization across agencies of the ICE process would be helpful.
• Concerns were also raised regarding the need to exercise off-ramps in PDB and CM/GC contracts, which could expose the owner to additional cost, schedule, and reputational risk.
• As an alternative to using progressive methods to improve competition on mega projects, packaging a large program into smaller projects of $150-250M could be a viable option.

5.2.2 Recommendations for Guidebook Content

As a final topic of discussion during the Owner forum, the team sought feedback on the desired content for the Guide to be developed under the research. As summarized below, the guidebook preferences primarily focused on guidance related to negotiating construction costs for CM/GC and PDB projects:

• Guidance on developing an opinion of the probable construction cost (for PDB & CM/GC)

• What supporting information to ask for
• How to review overhead and profit

• Guidance on the ICE process (for PDB & CM/GC)
  • What is an ICE (e.g., simple validation vs. bottom-up estimate)
  • Process (single blind? double-blind?)
• Sample RFP and contract language

5.2.3 Risk Allocation Strategies

As a post focus group activity, the research team asked the participants to identify how their agency typically allocates risks in fixed price contracts. Three participants provided feedback, as summarized in Table 5.3.

Table 5.3: Example DOT Risk Allocation Strategies

5.3 Industry Outreach: Observations and Findings

5.3.1 Interviews with Regional/Local Contractors

As discussed in Section 5.1.2, the team conducted interviews with seven regional/local contractors. Four interviewees represented firms operating in the Midwest and Mountain States: the remaining three represented firms working in the mid-Atlantic region. All had significant experience with fixed price DB projects. Some also had experience with CM/GC and PDB. Their so-called “sweet spot” for serving as a prime contractor on an APD method project are contracts valued between $100M to $250M. For projects greater than $500M, the interviewees generally have pursued joint venture arrangements with other firms.

Most interviewees noted that they have generally been more successful pursuing traditional, fixed price DB projects than progressive DB or CM/GC work. Their success with fixed price DB opportunities was attributed to having more intimate knowledge of the area and local market conditions, which allowed them to develop and offer more innovative and lower priced solutions than the larger, national players. They have generally been less successful pursuing progressive CM/GC and PDB work, for which larger, Tier 1 contractors can often demonstrate more experience and stronger qualifications.

The interviews provided the opportunity to compare the perspectives of these regional firms to the larger national players on issues such as the attractiveness of fixed price versus progressive APD methods, and how owners could improve risk allocation and increase competition on APD method projects.

Perspectives on Fixed Price vs. Progressive APDMs

In contrast to the larger firms, the regional contractors were far less enthusiastic of progressive PDMs. All questioned the necessity of strict qualifications criteria that require proponents to demonstrate at least five years of experience on comparable PDB or CM/GC projects, which few firms other than the larger Tier 1 contractors currently possess.

The interviewees generally viewed fixed price DB as providing owners with the highest likelihood of cost and schedule control. However, they also indicated that attracting and retaining staff on such projects is becoming more difficult, as the current workforce dislikes the fast-paced and often more stressful and adversarial nature of such projects.

PDB and CM/GC were seen as being good options for projects with high-risk profiles (e.g., due to underground utilities, railroads, permitting issues, etc.). However, these methods were viewed as providing less incentive for the integrated delivery team (owner, design, contractor) to find cost savings and minimize scope growth. They stressed that the owner team must have a strong project manager to exert the necessary discipline to minimize scope creep by stakeholders.

Risk Factors considered as part of the Pursuit Decision

As summarized in Table 5.4, factors considered as part of the firm’s pursuit decision closely mirrored those heard from larger firms, as discussed in Section 3.5.

Table 5.4: Factors Considered as part of the Pursuit Decision

Recommendations to Owners

Table 5.5 summarizes recommendations proposed by interviewees as to how owners could improve risk allocation and competition on fixed price APDM projects. These recommendations are similar to those shared by their larger counterparts, as discussed in Section 3.5.

Table 5.5: Recommendations to Owners

Key Takeaways from Regional Firms

The regional firms expressed far more support for the use of fixed-price DB than the larger national and international firms interviewed during Phase 1 of the research. Regional contractors and designers know their clients and the market and are confident in their ability to devise innovative solutions under fixed price DB optimized to meet local conditions. The regional contractors all viewed fixed-price DB as providing owners and taxpayers with the highest likelihood of cost and schedule control and perceived progressive methods as providing less incentive for the integrated delivery team (owner, designer, contractor) to pursue innovation and cost and schedule savings. They believed that national firms are advocating for progressive methods because they are less competitive and successful on fixed price procurements (as they are less familiar with local conditions that can drive cost savings). As owners increase project scale and complexity, they will continue to attract national or international players who may not have the same local knowledge and experience as the smaller regional firms.

5.3.2 National/International Contractors

The research team facilitated a roundtable discussion with five industry practitioners representing large, national and international firms.

In general, the perceptions of these practitioners regarding risk allocation remained unchanged from when the team first interviewed them during Phase 1 of the research. While noting that all delivery methods have a time and place, the participants continued to strongly favor increased use of progressive delivery methods as a means to de-risk large and complex projects and make them more attractive to pursue.

A summary of industry concerns along with key recommendations for owners regarding strategies and tools to align risk perceptions is provided below. Detailed workshop notes are provided in Appendix H.

Concerns

Particularly for large, fixed-price design-build contracts, the roundtable participants felt that owners in the past attempted to shift too much risk to contractors for things that they cannot control and cannot put a large enough contingency on to make the risk vs. reward tradeoff make sense. This risk imbalance has resulted in reduced competition and high contingency pricing, in addition to disputes and claims.

Although they believe that progressive PDMs provide an answer to this problem, they acknowledged that such methods introduce challenges of their own. For example,

• If project budgets are based on outdated estimates, it can lead to surprises and funding challenges when the PDB or CM/GC team provides an estimate that is far higher.
• The disparity in the level of experience between the owner’s pricing team and the industry team can be problematic when negotiating construction price.
  • Owners are often surprised and overwhelmed by the number of risks that contractors are considering, and the depth to which they are being assessed, when they develop their estimates and contingency calculations.
  • Although the risk workshops are helpful, the owner’s staff often “check out” after the first few hours of reviewing risks.
  • Owners have a hard time understanding the time and cost implications of risks.
• Qualifications of ICEs can vary widely.

Recommendations for Owners

To overcome such challenges, participants proposed the following recommendations:

• Retain the ICE early in the project development process. Instead of relying on old estimates, owners should retain the ICE early in the budgeting process (instead of waiting until the PDB or CM/GC team is engaged).