APPENDIX A

UAS Capability Maturity Model Assessment Tool

Note: A Word document version of Appendix A, which includes a version of the CMM assessment tool that can be edited and filled in with weights assigned to the different critical success factors, is included as part of the Implementation Memo for NCHRP 23-20, which can be found on the National Academies Press website (nap.nationalacademies.org) by searching for NCHRP Research Report 1147: Implementation of Unmanned Aircraft Systems Operational Capabilities: A Guide.

To use the unmanned aircraft systems (UAS) capability maturity model (CMM) assessment tool (Table A-1), first assign weights to different critical success factor categories and their associated components based on the importance of these success factors to a particular state DOT’s goals. By incorporating the assigned weights into the scoring system, agencies can measure their maturity level and progress in a more structured and consistent manner.

Each component within a critical success factor category is also scored by being ranked at one of three levels, adapted from Mallela et al. (2020).

• Level 1: The agency is in a relatively weak position to advance UAS operations with significant gaps in capability.
• Level 2: The agency is in a potentially tenable position to advance UAS operations but should address some gaps in capability that could pose risks to a successful implementation.
• Level 3: The agency is well-positioned to advance UAS operations.

The scoring across these three levels is split equally out of 100 percent, meaning each level is worth 33.3 percent. If an agency ranks at level 2, it would be given a score of 66.6 percent for that component, and if the agency reaches level 3, it would be given a score of 100 percent. The sum of the component scores for each category is then divided by the number of criteria in that specific critical success factor category. This number is then multiplied by the weighted percentage for that critical success factor category to provide a percentage score for the entire category. These final category scores are added up at the end for a total CMM assessment score.

Assessments based on the CMM can be conducted in a variety of ways, including self-assessment. In this approach, the agency’s staff evaluates its processes against the CMM criteria. This can be done through questionnaires, workshops, or collaborative discussions. While self-assessments may lack the objectivity of a formal outside review, they can still provide valuable insights and serve as a starting point for process improvement efforts. The other way is to use a third party to conduct a formal outside review; during the review process, the CMM Assessment Tool can be utilized.

Table A-2. CMM Assessment Scores and UAS Maturity Levels.

The completion of the assessment and scoring will result in a total score that correlates with a state DOT UAS Maturity Level (Table A-2).

The maturity level and associated go/no-go recommendation informs a state DOT on its present status and potential future steps to advance the maturity of its UAS practice. The levels have been adapted from Mallela et al. (2020).

1. No-Go: Decide not to advance the UAS practice at the time of this result because of:
   1. Insufficient interest.
   2. Insufficient capability.
   3. Insufficient resources.
   4. Inability to overcome the existing barriers.
   5. Inability to mitigate identified risks to an acceptable level.
   6. Some or all of the above.
2. Not-Now: Continue to monitor progress with UAS program development and application elsewhere, as well as all of the above factors that led to a “not-now” decision at this time. Discuss how these factors can be addressed and when it would be appropriate to revisit the decision. Include a discussion of the consequences and ramifications of not implementing UAS at this time.
3. Slow-Go: Decide to advance the maturity of the agency’s UAS program but for some combination of reasons, do so at an “evolutionary” pace by naturally incorporating the emerging practice into the agency’s other day-to-day operations as it becomes relatively mainstreamed. Include a discussion of the consequences and ramifications of a “slow-go” decision, alongside the conversation about how to address the outstanding gaps in maturity.
4. Go-Now: Decide to expeditiously advance UAS utilization into the agency’s various operations and practices, including an expedited testing and evaluation phase, potentially in collaboration with others interested in advancing it within the transportation sector.