CHAPTER 7

Vendor Feedback on Technology Implementations

Overview

The purpose of this chapter is to summarize the perspective of technology vendors related to post-contract activities for deploying new technologies.

Overall Strategies for Technology Deployment

The technology vendors and implementation consultants who participated in interviews indicated three overall preferred approaches for DOTs to be aware of prior to a technology deployment effort.

• Adopt a demonstration mindset throughout deployment. First, technology vendors asked that a “demonstration mindset” be present across the entire deployment effort. For example, whatever the new technology happens to be, perform demonstrations of the technology right away and then throughout development and configuration efforts (e.g., show the technology being used or even put demonstration-level iterations in the hands of users). The purpose of this strategy is for the DOT to see their own data and/or use cases being addressed by the new technology as soon as possible, which will then make configuration reviews more impactful and relevant. When it comes to performing technology demonstrations, vendors suggested that DOTs provide a clear agenda for demos, emphasizing the technology’s capabilities. Using the new technology in nearly every deployment-related meeting over time is also a best practice from the standpoint of user familiarity and adoption.
• Have a solid communication plan. Technology vendors focused on the importance of regular and substantive communication at the project management level when overseeing a deployment. The most common feedback across vendors was to hold a bi-weekly project management meeting that covered (a) the last 2 weeks of accomplishments, (b) review of active work items, and (c) a clear lookahead schedule including who would be involved. Vendors also suggested a project portal to house all documentation in a central repository.
• Establish the DOT’s project management team. Vendors repeatedly highlighted the importance of the DOT properly establishing a core management team for the technology deployment. The role of the DOT’s team is to rapidly assist when the vendor has questions or needs information to be gathered from within the DOT. The DOT’s team also needs to be ready to make final decisions and keep the schedule moving. Vendors identified the lack of internal decision-making as a primary source of schedule delays.

Discovery Phase

The discovery phase is a critical initial step in the implementation of new technology, as it serves as the foundation for understanding, planning, and effectively executing the deployment effort. The discovery phase typically builds upon the information from the solicitation

and uses greater detail. Technology vendors wanted DOTs to understand several aspects of the discovery phase:

• Clarifications to requirements will be needed. Vendors noted that the initial requirements they are given will often lack important details, be too vague, or may even be changed by the DOT. For example, a requirement might state the need for “user-friendly interfaces” without specifying the number of users, types of user roles, or specific functionalities required. Therefore, the vendor’s team will want to confirm even basic elements of the scope, including topics such as the number and type of users, material and equipment requirements, intended integrations, and so forth—even if this information was already published in the RFP.
• Integration requirements are especially important. Vendors will ask to meet with end users and other stakeholders to discuss the needed integrations. Vendors noted that many client organizations will state in their RFP that they want to integrate System A with System B, but neglect to add further information. During discovery, vendors will want to expand upon the DOT’s request to understand what problem the DOT is trying to solve with each integration, what use cases are involved, and more.
• Discovery is a time to minimize risk. From a vendor perspective, perhaps the greatest risk when implementing a new technology is if the client requests a tool or desires functionality that is either not available or not possible to perform. This is especially a risk for COTS software technologies because the initial scope and procurement process places a constraint on the eventual outcomes that are achievable. In such an instance, the selected vendor must work within the bounds of what the selected technology can do. This becomes even more challenging in a new or advanced technology with which the client may have less familiarity. The vendor’s implementation team will attempt to catch any misalignment in expectations early in discovery to ensure no surprises later.
• Begin with the end in mind. An effective discovery process will take time to think about the end-of-implementation uses of the technology. Questions will be reviewed, such as “What data need to be produced from using the new technology?” and “What reports does the DOT need to inform?” and “What impact is expected on labor?” The selected technology vendor can typically design or implement the new technology based on the outcomes the DOT is trying to achieve, so long as these are defined upfront.
• Make client resources available for discovery. Vendors stressed that there will be a time commitment necessary from the DOT’s staff and resources. The DOT should expect to spend time addressing all the items noted earlier. Vendors also requested to see the DOT’s organizational chart to understand which groups are impacted by the new technology as well as a project-specific organizational chart of who will be involved during implementation.

While many of these items would have been started in the pre-solicitation phase, the extent to which the full details are captured in the exact manner needed by the selected vendor can vary. The selected vendor typically uses the discovery phase to build upon the information provided in the procurement process, including in the solicitation and SOW, by providing additional context, examples, and direct discussions between the client team and the selected vendor. The difference in detail is partially due to the client’s initial desire to keep the solicitation’s SOW clear, complete, and accurate while not overloading potential vendors with so much information as to be a distraction, which may discourage their decision to propose. Once the contract is awarded, the selected vendor will typically require additional information to proceed into full implementation activities.

Workflow Development and Technology Configuration Phase

After discovery, there is typically an iterative process of workflow development and technology configuration. This is often the lengthiest portion of the implementation and will vary based on technology type and client requirements. Most vendors stated the goal of this phase is to

deliver a technology solution that is more than 75% to 80% tailored to the DOT’s need to enable more detailed testing and final configuration.

Throughout this phase, the selected vendor will seek to understand the flow of the client’s activities before moving into detailed setup and configuration. This is an iterative process; vendors may initially start with base configurations or technology capabilities, which are then increasingly tailored to fit the DOT’s needs. Vendors will cyclically gather user input, build or set up the technology to meet the scenario, show the results to end users, and then tweak as necessary. Vendors will typically interact with DOT staff via configuration workshops to continually tailor the new technology for specific user scenarios.

For vendors, a risk in this phase can be the number, complexity, and size of the DOT’s workflows. The biggest challenge is when different user groups have slightly different requirements and vendors must accommodate multiple perspectives simultaneously.

Configuration activities need to be demonstrating the DOT’s future state environment extensively. Ideally, there will be shorter cycle times via vendors who use sprint methodology to iterate as much as possible. In such a methodology, the vendors will show or demonstrate the technology or application, gather feedback, and then rapidly make tweaks and updates for additional review and confirmation with the client’s user groups.

User Acceptance Testing

The UAT phase is intended to verify the final configuration of the new technology. A process is planned to validate the DOT’s requirements and user cases and then ensure user satisfaction with how the technology has complied with those requirements. Vendors indicated that they will typically develop test cases and train DOT staff on how to perform the tests. Most vendors acknowledged the UAT phase cannot be overly long or burdensome; rather, vendors said that a period of 3 to 4 weeks is the maximum amount of time for most technology deployments. Beyond 4 weeks, there is a decline in user willingness to test any further. Vendors should be expected to provide remediation immediately as the UAT phase identifies necessary changes or shortcomings.

Training and Adoption of New Technology

After UAT, the goal is to hand over the new technology in such a manner that the DOT can leverage the technology. The vendor will typically establish “administrators” or “power users” among the DOT staff (alongside the DOT’s IT office) and equip them to manage the application going forward. Training will be provided to end users such that they know how to perform their functions with the new technology.

In terms of training format, most vendors indicated that live and in-person training is necessary for administrators. This can be anywhere from a half-day to a full week, depending on the size and complexity of the new technology. For end users, conducting a “day in the life of” each user type is helpful to simulate their functions and workflows with the new technology. A key aspect of user training is to answer the questions, “What’s in it for me?” and “How does this impact me?”

Training is most effective when timed immediately before the “go live” deployment of the technology. This timing also minimizes the loss of knowledge that can occur if there is too large a gap in time from the training to when the users are fully utilizing the new technology as part of their normal work functions.