2

Assessment of Existing Strategies and Roadmaps

This chapter assesses the effectiveness of the relevant strategies, plans, roadmaps, and related artifacts that shape the Department of the Air Force (DAF) posture for achieving digital transformation (DT) in the next decade. These will include guidance and strategy documents published in the open literature by the U.S. Air Force, the U.S. Space Force, the Office of the Secretary of Defense, and others that have the potential to influence the posture of the DAF.

Currently, although significant progress has been made toward the advancement of DT capabilities within each of these organizations, the DAF lacks an integrated strategy. Therefore, this chapter reviews one of the more pace-setting approaches within the DAF, led by the Air Force Materiel Command (AFMC) and considers the status of various strategies, roadmaps, and plans across the system life cycle.

THE AIR FORCE MATERIEL COMMAND’S DIGITAL MATERIEL MANAGEMENT

The AFMC mission is to “develop, deliver, support and sustain war-winning capabilities” for the Air Force, including major life-cycle activities in research and development, acquisition, test and evaluation, and logistics.¹ Additionally, AFMC has recently taken on responsibility as the servicing major command for the Space

___________________

¹ Air Force Materiel Command (AFMC), n.d., “We Are AFMC,” https://www.afmc.af.mil/About-Us, accessed January 14, 2025.

Force, which increases the complexity of its interests and concerns.² Given the complexity of its mission and the pacing threat by U.S. adversaries, AFMC and its Digital Transformation Office are adopting Digital Materiel Management (DMM) capabilities to accelerate fielding, sustainment, and modernization.³ This will lead to a reduction in life-cycle cost given that DMM leverages improved data management, standardization, automation, and open collaboration to effectively reduce maintenance and repair costs.

Among all of the organizations the committee has heard from, or are aware of, the AFMC and its DMM are the most cogent and resourced, and are well enabled by its four-star commander’s intent. This strategy is clearly expressed in material provided to this committee:

DMM is the process of integrating and employing digital methods across the entire life cycle—from invention to retirement—for both warfighting capabilities as well as installation and mission support capabilities.

The committee learned of five AFMC initiatives for realizing DMM. They are as follows:

1. Structure and secure AFMC data.
2. Facilitate digital training across AFMC.
3. Identify and facilitate access to tools.
4. Develop tactical digital strategies.
5. Establish a digital-first culture across AFMC.

___________________

² AFMC, 2021, “AFMC Takes on Role as Servicing Major Command for Space Force,” https://www.af.mil/News/Article-Display/Article/2718809/afmc-takes-on-role-as-servicing-major-command-for-space-force.

³ K. Hurst, S.A. Turek, C.M. Steipp, and D.Z. Richardson, 2023, Digital Materiel Management: An Accelerated Future State, Air Force Materiel Command, https://media.defense.gov/2023/Jun/12/2003239595/-1/-1/0/DMM%20-%20AN%20ACCELERATED%20FUTURE%20STATE_FINAL_compliant_17AUG23.PDF.

⁴ R.B. Fookes, 2024, “Digital Materiel Management (DMM) Strategy,” Paper presented to the committee, September 25, National Academies of Sciences, Engineering, and Medicine.

The committee was briefed about the status of each as they are currently, and concurrently, under way. Many digital enhancements and enterprise solutions directly rely upon each other. Part of the challenge is for users to understand and leverage these interdependencies to achieve “digital enterprise solutions.” This reliance is also generally true across most of the AFMC strategic plan lines of effort and its respective initiatives. Thus, a coordinated, integrated, and simultaneous maturation is critical to realizing the overarching AFMC vision for DMM.

One of three principles underpinning the various DMM initiatives centers on the AFMC workforce, with the goal of modernizing the workforce capability and culture to support the timely, data-driven decisions.⁵ The actions flowing from this principle are articulated as follows:

• Curate digital specific training opportunities to develop workforce skill and confidence, applying new technology to achieve decision advantage across program life cycles and acquisition functions.
• Equip the acquisition workforce with guidance, lessons learned, and best practices to support DMM implementation.
• Design a cultural transformation where the workforce is incentivized to apply digital methods to acquisition processes.
• Standardize data purchase, access, and delivery contract language for reuse across acquisition functions and product types.
• Prescribe contractor data delivery in model-based products in lieu of traditional document-based formats.

Modernizing, equipping, and motivating the workforce through DMM is absolutely essential to success of DT in the DAF.

LIFE-CYCLE PERSPECTIVES

This section assesses DAF’s implementation of the Department of Defense Strategy,⁶ with emphasis on the effectiveness of the individual DT strategies and roadmaps at the program level as they are employed in DAF organizations to support the U.S. Department of Defense (DoD) system life cycle. The life-cycle “lens” also illuminates the prospects for phased implementation of DT, and the value generated in a given stage by executing a strategy in a previous stage.

___________________

⁵ Ibid.

⁶ Office of the Deputy Assistant Secretary of Defense for Systems Engineering, 2018, Department of Defense Digital Engineering Strategy June 2018, https://ac.cto.mil/wp-content/uploads/2019/06/USA001603-18-DSD.pdf.

The DoD acquisition life cycle is laid out in DoD Instruction 5000.85⁷ and shown in Figures 2-1 and 2-2. As a program progresses through the life cycle, it must pass through a number of stages and milestones. The Materiel Solutions Analysis (MSA) phase provides for the development of requirements and an analysis of alternatives before the Milestone A decision is made. After Milestone A, the program enters the Technology Maturation and Risk Reduction (TMRR) phase, where requirements are refined and prototyping begins. The Milestone B decision allows a program to move into the Engineering and Manufacturing Development (EMD) phase, where the system design is completed and developmental testing occurs. The Milestone C decision leads to the program entering the Production and Deployment (P&D) phase. In this phase, the program typically sees low rate initial production, and operational testing occurs. After clearing the P&D phase, a program obtains its Initial Operational Capacity (IOC) and, eventually, its Full Operational Capacity (FOC). After FOC, the program moves into the Operations and Sustainment phase.

Failure to consider the full system life cycle early on may have adverse effects in terms of cost and performance long after fielding a solution. A primary example noted by the Government Accountability Office (GAO) is the F-35 Lightening II aircraft, where sustainment cost estimates ballooned over 44 percent—that is nearly $5 trillion—in just 5 years.⁸ The primary causes noted by the GAO include overestimating system reliability and usage and underestimating the required service life of the aircraft. A report by the Congressional Research Service found that the

___________________

⁷ Office of the Under Secretary of Defense for Acquisition and Sustainment (OUSD(A&S)), 2021, “Major Capability Acquisition,” DoDI 5000.85.

⁸ Government Accountability Office, 2024, “F-35 Sustainment: Costs Continue to Rise While Planned Use and Availability Have Decreased,” GAO-24-106703.

true cost of even just developing and acquiring the F-35 was significantly underestimated due to poor assumptions on production learning curves.⁹ This short example regarding the F-35 is used to emphasize the point that informed and accurate estimates regarding downstream life-cycle phases are crucial to fielding—and

___________________

⁹ Congressional Research Service, 2024, “F-35 Lightning II: Background and Issues for Congress.”

sustaining—systems that are critical to U.S. national security. DT is intended to provide that integrated, full life-cycle perspective.

Materiel Solutions Analysis Phase

The MSA phase provides the first opportunity to influence the supportability and affordability of weapon systems through the development of requirements to meet identified capability gaps and the development of analysis of alternatives to address those gaps.¹⁰ The requirements-setting process drives the development of numerous models and data artifacts that are reference points for the life of the system. In this phase, hardware and software models of conceptual designs are exercised in simulations and proof-of-concept prototypes are evaluated to inform the analysis of alternatives, design concepts, and eventually the Milestone A decision.¹¹

DoD has demonstrated an increased focus on mission engineering, with the October 2023 release of the Department of Defense Mission Engineering Guide Version 2.0. Mission engineering is a process that helps DoD to better understand and assess impacts to mission outcomes based on changes to systems, threats, operational concepts, environments, and mission architectures. Its “results inform decisions on military requirements, acquisition, research, and development.”¹² “[Digital engineering (DE)] principles support a continuum that enables consistency and reuse of models and data when applied in the mission engineering process.”¹³

The committee was briefed on existing capabilities, at the enterprise level developed by the Office of the Undersecretary of Defense for Research and Engineering (OUSD(R&E)), that start with a mission problem to establish high-level digital models that can assess and compare competing system architectures against identified metrics.¹⁴ Figure 2-3 shows the relationship between mission models from operational data and a simulation of the operational environment. These models and resulting artifacts would constitute the initial DE foundations for programs and for the DAF more broadly. Practitioners should compile a library of models and data sets that are developed and used throughout the life cycle of the system.

The MSA phase should result in the following three mission engineering artifacts: “1) synthesis and documentation of mission impacts and outcomes obtained

___________________

¹⁰ OUSD(A&S), 2021, “Major Capability Acquisition,” DoDI 5000.85 Section 3.6, pp. 11–12, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500085p.pdf.

¹¹ Defense Acquisition University (DAU), n.d., “Materiel Solutions Analysis (MSA) Phase,” Adaptive Acquisition Framework, https://aaf.dau.edu/aaf/mca/msa, accessed April 28, 2025.

¹² Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)) Mission Capabilities, 2023, Department of Defense Mission Engineering Guide Version 2.0, https://ac.cto.mil/wp-content/uploads/2023/11/MEG_2_Oct2023.pdf.

¹³ Ibid.

¹⁴ Ibid.

from the mission engineering analysis, 2) the capture and presentation of recommended mission architectures, and 3) the curation of mission engineering artifacts for future use.”¹⁵

Data standards must also be applied to ensure that information is searchable, accessible, and retrievable in formats that will—for the life of the system—support diverse analyses including use of machine learning (ML) and artificial intelligence (AI) capabilities.

Technology Maturity and Risk Reduction Phase

In the TMRR phase, activities are guided by the draft Capabilities Description Document that is developed in MSA as part of the requirement-setting process. Design and requirements trades are conducted using advanced digital tools and methodologies to ensure that the final product meets mission needs, is affordable, and is supported by an executable development and production program. Digital Materiel Management (DMM) plays a crucial role in optimizing these processes by preserving the resulting models, design, and data artifacts in an integrated product structure (IPS) repository. This IPS resides in a collaborative digital environment and represents full suite design and models that will evolve through TMRR. This will serve as the authoritative grounding for multiple organizations’ design and development teams throughout the life cycle of the system. Additionally, this ensures all modifications and updates are traceable, preventing information loss and maintaining design consistency across the life cycle.

Engineering data such as hardware, software, and firmware models are stored and managed within this IPS, providing a single source of truth that enhances collaboration and decision making. DMM employs specific tools that enable detailed

___________________

¹⁵ Ibid.

requirement validation and efficient design trade-off analysis, ensuring semantic consistency among models and engineering artifacts. All of the supporting analyses, test plans, build records, test results, peer reviews, program major reviews, minor reviews, meeting documentation, etc., will be linked in a standard format within the single IPS.¹⁶ Exploiting these concepts during the TMRR phase helps identify critical design flaws early, reducing rework and associated costs.

Accessible digital collaborative environments are recognized as extensive. It is an important aspiration for the DAF to fully achieve the DT vision with full deployment of DE capabilities. But it is also important that incremental steps in the direction of DE should yield immediate benefits, thus creating impetus to strongly support the process. In this phase, hardware and software models of preliminary designs will be exercised in performance simulations, and early prototypes will be tested to inform design modifications and eventually the Milestone B.¹⁷ In summary, the TMRR phase leverages a highly integrated digital environment to not only mature technologies but also mitigate risks by ensuring efficient resource utilization, rigorous validation, and proactive maintenance planning.

Engineering and Manufacturing Development Phase

In the EMD phase, the core activities to develop, build, test, and evaluate a materiel solution are conducted to ensure that all operational and implied requirements, including those for security, have been met, and to support production, deployment, and sustainment decisions.¹⁸ This is the most intense phase for building the digital thread and the most critical to ensure support for the entire life-cycle program. All hardware- and software-detailed designs that result from extensive modeling and simulation will be developed and must be preserved in the previously established IPS within the collaborative digital environment.¹⁹ As a real-time authoritative source of truth that is accessed by all primary and support organizations, inconsistencies that were previously the result of translating models and data from one format to another, or from lags in updating records in multiple repositories, must be resolved in this phase.

As hardware and software designs are refined and completed, the associated models will be exercised in advanced performance simulations, while components

___________________

¹⁶ OUSD(A&S), 2021, “Major Capability Acquisition,” DoDI 5000.85 Section 3.8, pp. 13–14, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500085p.pdf.

¹⁷ DAU, n.d., “Technology Maturity and Risk Reduction (TMRR) Phase,” Adaptive Acquisition Framework, https://aaf.dau.edu/aaf/mca/tmrr, accessed April 28, 2025.

¹⁸ OUSD(A&S), 2021, “Major Capability Acquisition,” DoDI 5000.85 Section 3.11, pp. 15–16, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500085p.pdf.

¹⁹ DAU, n.d., “Engineering and Manufacturing Development (EMD) Phase,” Adaptive Acquisition Framework, https://aaf.dau.edu/aaf/mca/emd-phase, accessed April 28, 2025.

and full-scale builds will be subjected to full environmental and development testing. The results of these performance simulations and testing will form the basis for qualification and certification of the design. As production facilities develop manufacturing operations and assemble hardware for qualification, the associated manufacturing instructions, assembly records, production models, production equipment model numbers and settings, test data, and so on must be incorporated into the digital thread to support production readiness review and operational test readiness review. They will also provide the authoritative source of truth for the first digital twins that can be compared to design definitions to ensure that the design intent has been preserved through production and any discrepancies can be assessed against validated performance models.

Production and Deployment Phase

In the P&D phase, the product is produced and fielded or deployed for use by operational units. A number of key events are conducted: low-rate initial production, personnel training, completion of developmental test and evaluation (if required), initial operational test and evaluation, and the full-rate production²⁰ or full-deployment decision. In this phase, all system sustainment and support activities are established, and the appropriate operational authority will declare IOC.²¹

The focus of the digital thread now shifts to production, the evaluation of digital threads and digital twins, and assessment of production data to identify opportunities to increase first pass yields and streamline operations. Informed by increased deployment and test data, and to address production needs, design definitions are modified. It is crucial in this phase that as design changes are implemented, the associated models are updated to reflect the production definition. Digital data generated in this phase will often be referenced in the future phases to assess the effects of system aging and as it is exposed to the operational environments. Preservation of these records will eliminate the need to reverse engineer components in the Operations and Support Phase.

The vast majority of the production portion of P&D is conducted by and through industry. A thorough investigation of the extent of DT adoption by industry is beyond the scope of this study. However, reports to the committee suggest that the employment of digital tools and methods in production by defense suppliers exceeds the levels seen in the early life-cycle phases where the DAF itself played the more central role.

___________________

²⁰ DAU, n.d., “Full Rate Production (FRP) Decision,” Adaptive Acquisition Framework, https://aaf.dau.edu/aaf/mca/production-decision, accessed February 3, 2025.

²¹ OUSD(A&S), 2021, “Major Capability Acquisition,” DoDI 5000.85 Section 3.13, p. 17, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500085p.pdf.

The deployment portion presents a different picture as, in the majority of cases, it is DAF personnel (uniformed and civilian) who operate the systems that are deployed and execute sustainment functions. While private contractors are certainly part of some deployment and sustainment activities, the primary processes and assets used are governmental.

The committee found exemplar activities happening in the DT space through the Air Force Sustainment Center (AFSC), discussed in Chapter 5. However, outside of that example, and through the assessments the committee heard from many experts, there remains very little evidence of meaningful connection of P&D digital processes and tools (and most importantly, information and learning) via feedback to earlier life-cycle phases. Additionally, “disconnect” appears to be prominent, embodied in the fact that the industry-side digital innovations are not shared with or seemingly informed by the DAF digital efforts, and vice versa. The T-7A Redhawk case study presented in Chapter 4 illustrates this situation.

Operations and Support Phase

The Operations and Support phase satisfies materiel readiness and operational support performance requirements and includes major efforts such as sustainment, operation of the system, and disposal. In modern practice, this phase also involves a continuous stream of technical refreshment and capability insertion through upgrade and modernization efforts. The program will measure, assess, and report system readiness using sustainment metrics and implement corrective actions for trends diverging from the required performance outcomes.²²

The tools established initially in the MSA phase to support mission engineering, and the accumulation of artifacts that subsequently reside in the digital thread that are accessible through the digital collaborative environment, enables the reuse of products to facilitate updates and changes as needed.

The system digital data will continue to grow with continued maintenance of the digital thread and the growth of digital twins. Operational and further test data will be assessed against performance predictions to ensure that the system meets requirements as it ages and external factors evolve. These evaluations will inform necessary actions to sustain the system through its operational life to eventual retirement and dismantlement.²³

In this phase, engineering is now largely focused on the sustainment of fielded systems that were either developed prior to modern engineering capabilities, or for

___________________

²² OUSD(A&S), 2021, “Major Capability Acquisition,” DoDI 5000.85 Section 3.15, p. 18, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500085p.pdf.

²³ DAU, n.d., “Operations and Support (O&S) Phase,” Adaptive Acquisition Framework, https://aaf.dau.edu/aaf/mca/os-phase, accessed April 28, 2025.

which the DAF does not have ready access to models and model-based designs that are now being reverse engineered so that aging parts can be replaced. AFSC has been effective in establishing a DMM ecosystem that integrates maintenance, supply chain, and mission enablers. AFSC’s integrated tool set has supported the reverse engineering and manufacture of key system components (as described in Chapter 4). These systems were developed prior to the emergence of DE, so there was no other mechanism besides reverse engineering to allow service of critical aircraft (e.g., the A-10). This places emphasis on how to express what is learned in the reverse engineering and to validate it, so it becomes useful in an emerging digital thread. Developing techniques to support this kind of “reverse engineering” approach is important to address legacy challenges. However, the focus on DE should also be extended to acquisition, so that the capabilities and artifacts will be available across all life-cycle stages, including the Operations and Support phase, to prevent any future need for reverse engineering.

CONCLUSION

The challenge for the DAF going forward is to establish a comprehensive strategy and architecture that will result in full digital threads and the realization of full capabilities while in parallel continuing to make progress within each command to meet the immediate needs of its respective responsibilities. Efficient establishment of the digital thread based on validated models (ideally an authoritative source of truth) must begin in the earliest system life-cycle phases, and so this assessment follows that progression. However, the DAF faces significant challenges in its DT efforts, primarily due to the lack of a comprehensive strategy. Despite the presence of promising initiatives and “pockets of excellence,” the absence of a unified, enterprise-wide DT strategy limits overall progress and cohesion.

The absence of a clearly defined DT strategy is evident by substantial gaps in the establishment of a digital enterprise, particularly in integrating a seamless digital thread across the system life cycle. This deficiency is exacerbated by the lack of integration, illustrating the urgent need for a standardized, high-level digital strategy that encompasses all life-cycle operations. Furthermore, the DAF lacks a named leader with the necessary authority and resources to effectively implement such a strategy. The lack of a designated officer or senior leadership position specifically accountable for DT results in a diffusion of responsibility, with no single entity empowered to direct acquisition executives (e.g., program executive officers) or enforce the adoption of enterprise capabilities. This leadership vacuum leads to inconsistent application of digital tools and methodologies, limits innovation, and hinders the ability to establish and maintain digital continuity across the DAF.

Allowing the current grassroots efforts to proliferate in the absence of a coherent strategy and authoritative leadership will likely result in suboptimal islands of

limited transformation. These isolated pockets of digital advancement, while beneficial at a local level, cannot collectively achieve the broader objectives required for a successful digital enterprise. At best, these efforts may bring about incremental improvements; at worst, they could culminate in failed and expensive experimentation campaigns, wasting valuable resources and time.

To mitigate these risks, the DAF must prioritize the development of a comprehensive DT strategy and appoint a senior leader with the clear mandate, authority, and resources to drive this initiative. By doing so, the DAF can ensure cohesive, enterprise-wide DT efforts that enhance operational efficiency, adaptability, and long-term success in an increasingly complex and challenging operational environment. A starting point for articulating this strategy could be an intentional design of a phased approach that provides the initial connection points among DT efforts across the life cycle.

Overarching Conclusion: Although the Department of the Air Force has established numerous elements that support a digital enterprise with pockets of excellence in some centers, the efforts under way are not on a trajectory to achieve a unified capability that serves the U.S. Air Force and the U.S. Space Force with a seamless digital thread across the system life cycle and commands, due to lack of a common high-level leadership, strategy, architecture, and consistent funding.