E

Additional Information on Common Elements to Aid Digital Transformation

For each of the committee’s identified common elements for digital transformation (DT) success, the committee generated initial consideration questions, presented below. These questions are far from all-encompassing. Instead, they offer a sampling of the types of questions that a DT leadership team needs to be able to answer if they wish to stack the odds of DT success in their favor.

With the common elements and associated consideration questions as background, the committee considered a cursory thought experiment where ideal conditions lead to exceptional success for a digital engineering (DE) effort—Project Nightlight—within a broader DT initiative. In this fictional world, visionary leadership, strategic resource allocation, robust secure information technology, standardized software and data protocols, and meaningful metrics seamlessly integrate. The result is an agile, systems-thinking approach to DE, leveraging artificial intelligence (AI)-powered solutions and a resilient end-to-end life-cycle framework that empowers innovation, efficiency, and collaboration across all organizational levels and stakeholders. This hypothetical use case can be found below.

SAMPLE CONSIDERATION QUESTIONS FOR LEADERSHIP AND ADOPTION CULTURE

1. Leadership
   • Who is in charge of driving change?
   • What authority does the leader have, and what is the span of control?
   • How is the need communicated, and what are the consequences?

1. • Is the DT initiative dependent upon the leader, and will it endure when the current “DT champion” leaves?
2. Workforce
   • Do leaders know what skill sets the organization needs?
   • Is there an assessment of the current workforce skill set?
   • What is the ratio of hiring to the required skill set relative to training the current workforce to the required skill set?
   • What is the accessibility to the tools?
   • Is there a funded training program, either in-house or outsourced?
   • Is there a “sandbox” environment where the workforce can learn the tools?
3. Adoption
   • Are there incentives or consequences for not using the tools and processes?
   • What is the assessment of the DT adoption, and is it widely used?
   • Is there an effective “lessons learned” or “best practices” process at the organization?
   • Is there a sense of urgency?
   • What actions are taken for units within an organization who do not adopt DT efforts?
4. Organizational strategy and governance
   • How is the organization’s DT vision aligned with its broader mission and goals?
   • Does the organization have a centralized governance structure, such as a Digital Transformation Office, to unify and enforce DT policies?
   • Are there clear, measurable objectives and maturity metrics to evaluate progress?
   • How are resources allocated across programs to ensure balanced and equitable support for DT initiatives?
   • Does the organization allow for flexibility in implementation to accommodate diverse programs and needs while maintaining overall alignment?
5. Change management and communication
   • What structured change management strategies are in place to address resistance to DT?
   • How are risk-averse individuals or groups engaged and encouraged to adopt a “digital first” mindset?
   • Are there frequent and transparent communication efforts, such as town halls or newsletters, to update stakeholders on DT progress and successes?

1. • Are real-world success stories and proof-of-concept examples showcased to build trust and momentum for DT initiatives?
   • How do leaders address workforce fears of obsolescence or job loss from DT efforts?

SAMPLE CONSIDERATION QUESTIONS FOR FUNDING AND INCENTIVES

1. Evaluating the return on investment (ROI) for DT efforts
   • How will leaders show that a dollar saved via DT is “capability earned” downstream?
   • How will leaders create a standard quantification of “capability earned”?
   • How do leaders define and measure ROI for DT tools and initiatives across different stages of a program’s life cycle?
   • What mechanisms evaluate and guide funding to programs or tools with the highest demonstrated impact?
   • How can cost savings or efficiencies achieved through DT initiatives be reinvested into broader DT goals?
2. Incentivizing mechanisms
   • What formal mechanisms for incentives and funding do government and military bureaucracies have available for use?
   • How are those incentives applied to existing programs (not related to DT)?
   • How do leaders achieve sustained funding for a DT effort?
   • How might additional incentives be created (e.g., request National Defense Authorization Act language)?
3. Incentivize feedback for continuous learning and accountability
   • How will leaders ensure that digital models evolve during the system life cycle?
   • How will leaders measure continuous learning in terms of updating a model through its life cycle?
   • What are the examples of continuous learning in government acquisition programs, beyond the acquisition phase?
   • How can centralized funding mechanisms incorporate feedback from decentralized programs to ensure adaptability?
   • How can leaders develop penalties if a stakeholder deviates from providing feedback?
4. Understand the simultaneous impacts from centralized and decentralized authority funding
   • What processes make decentralized program managers feel incentivized rather than constrained by centralized funding approaches?

1. • Can leaders measure diminishing returns on a purely directive, centralized model?
   • How can funding strategies balance the need for standardized tools with the flexibility required by unique program demands?
   • What processes exist to review and approve program-specific exceptions to centralized funding rules?
   • How are program managers, financial officers, and centralized funding bodies collaborating to align funding priorities with operational needs?
   • How can external partnerships (e.g., with industry or academia) be leveraged to supplement funding and incentivize innovation?

SAMPLE CONSIDERATION QUESTIONS FOR SOFTWARE AND DATA STANDARDS

1. System artifacts—software and hardware within the system, including associated data
   • How will leaders manage shared architecture development to achieve stakeholder buy-in?
     • Technical architecture—software, hardware, interfaces/application programming interfaces, buses?
     • Collaborative multi-stakeholder open-architecture practices: Modular Open Systems Approach, Associate Contractor Agreements, etc.?
     • Bias to adopt industry standard/normative architectural practices/elements?
   • Develop best-practice resources within the Services, while also maintaining awareness that there is no substitute for intelligent, capable people
   • Collaboratively resolve measurement challenges?
   • How will leaders model supply chain and sourcing?
     • Components?
     • Services?
   • How can leaders perform configuration and consistency management of engineering data and the associated software and hardware?
     • Resulting in government owning its data and controlling access?
     • Curation and meta-data for system artifacts, with appropriate access support?
2. Engineering artifacts—tools and models, including associated data, intellectual property (IP), and practices
   • How will leaders separate their evidence from the tools that produce it and reduce “vendor lock”?

2. • • Use engineering-data focus to provide low barrier to entry to new tools and tool sources?
   • How will leaders enable a maximum range of models, analyses, tests, and metrics?
     • Not just SysMLv2, but also model-based systems engineering models, security, and performance?
     • Enable use of AI models as they evolve?
   • How will leaders achieve componentized and composable model ontologies?
     • Avoid grand single unified, unmaintainable
     • Evidence that models are aligned with engineering artifacts
   • How will leaders integrate with system test, evaluation, verification, and validation (TEVV) tools, practices, evidence?
3. Life cycle–sustained consistency management of system and engineering artifacts that serve as authoritative sources of truth, with access management
   • What are DT process concepts to get benefit both across the life cycle and within its stages?
     • Concept, requirements, development, evolution, sustainment, and continuous integration/continuous delivery?
   • How will leaders achieve more fully digital sharing and handoffs among stakeholders?
   • How will leaders digitally enable contract management and program management office engagement?
   • How will leaders integrate modeling and simulation continuously throughout life cycle?
     • Including concept development, early validation, and training?
   • How will leaders integrate continuous TEVV practice?
     • Including coordination of development and operational test and evaluation?
4. Data governance and ownership
   • How can organizations define clear policies for data ownership and access across stakeholders?
   • What mechanisms ensure proper tagging, curation, and metadata management to support FAIR (findability, accessibility, interoperability, and reuse) principles?
   • How can secure data-sharing practices be established to balance transparency and protection of IP?
   • What strategies can address challenges in managing proprietary data while maintaining interoperability?

5. Continuous improvement and scalability
   • How will our organization incrementally adopt digital tools to minimize disruption while scaling capabilities over time?
   • What are our strategies to keep models and tools relevant and scalable as technology evolves?
   • How will our programs maintain alignment between evolving program requirements and existing digital standards?

SAMPLE CONSIDERATION QUESTIONS FOR HARDWARE AND SECURE INFORMATION TECHNOLOGY (IT)

1. Operating architecture for hardware and secure IT including data and software
   • How will leaders create a target architecture and transition plan?
   • How does our infrastructure support scalability for evolving program demands and technological advancements?
   • What mechanisms are in place to adapt to new technologies without disrupting ongoing operations?
   • How will leaders deal with issues such as security/classification in building an architecture?
   • When working with commercial tools do leaders have any contractual constructs that can be shared to foster long-term compatibility?
   • Does the architecture of the digital environment align with the architecture of the operational environment?
2. Modernization of existing/legacy infrastructure
   • What are our current infrastructural capabilities and who operates them?
   • Do leaders have a prioritized list of needs and gaps of our current capabilities?
   • What are the impediments to modernization?
3. Operational aspects of infrastructure
   • What is the plan to move from today to envisioned future (who is going to lead this)?
   • What talent/recruitment/funding systems are in place to support the plan?
   • How (in terms of resources) are leaders going to sustain the infrastructure, tool base and artifacts being developed?
4. Risk management and cybersecurity
   • How will leaders provide cybersecurity, particularly for interconnected systems with multiple access points?

4. • What role do periodic cybersecurity assessments (e.g., black hat/white hat exercises) play in maintaining our system’s resilience?
   • How can leaders minimize risks associated with “vendor lock” and promote long-term sustainability and compatibility of digital systems?
5. Collaboration and stakeholder integration
   • How will leaders make secure IT support seamless collaboration between government, industry, academia, and other stakeholders?
   • What strategies are in place to align infrastructure capabilities with the operational needs of diverse participants?
   • How will leaders balance transparency and IP protection when sharing infrastructure resources and data?
   • Is there a framework for sharing digital standards and tools across organizations to foster interoperability and joint investment?
   • Are there mechanisms to include smaller businesses and support their participation in the digital ecosystem?

SAMPLE CONSIDERATION QUESTIONS FOR METRICS

1. Conversion metrics: The DT plan should describe a set of major releases, tasks, or phases that increasingly add more functionality and data into the digital enterprise. Schedule metrics are based on budgeted work versus accomplished work (or something equivalent).
   • Percent (release/task/phase) completed, in-work, or blocked, e.g.:
     • Framework definition
     • Number of components under contract
     • Number of components assigned to development team
     • Number of days (or weeks) of critical path slip or ahead of schedule
     • Days of schedule margin
     • Budget (current spent, planned spent, margin)
     • Percent of framework completed
     • Percent of framework testing completed
     • Percent of transition project completed
     • Number of open risk items
     • Number of closed risk items
     • Percent of planned data converted
     • Percent of planned services and apps converted
2. Time horizon: DT initiatives (e.g., projects, programs) have finite time horizons that are determined by cost and other constraints. The key questions in this regard are:
   • What is the duration of the DT development and deployment project or program?
   • How was the time horizon picked (i.e., rationale)?

2. • Is the time horizon metric compatible with project/plan duration?
3. Measurability: All metrics should either be measurable, predictable, and/or computable (i.e., become measurable when an implementation becomes available). All measures of effectiveness (MOEs) and measures of performance (MOPs) can be expected to be relevant to some stakeholder or concern. However, keeping track of MOEs and MOPs costs time and money. Therefore, all MOEs and MOPs must contribute to diagnosticity for a related concern, and the Department of the Air Force (DAF) should not keep track of those that are not needed.
   • Can leaders measure this directly or do leaders need a proxy measurement?
   • Are there different parameter thresholds associated with different stages of the project?
4. Relevance: Metrics need to be relevant to the enterprise DT strategy.
   • Is the metric directly related to DT system goals?
   • Is it compatible with project funding constraints?
   • Does the relevance of the metric change with the project phase?
   • Is the metric relevant to the sustainment phase and life-cycle affordability?
5. Diagnosticity: This metric pertains to the insights provided by its measurement.
   • Does the metric help compare time-to-milestone speed relative to the status quo or any other baseline?
   • Does the metric help track reductions in defects and improvements in test outcomes?
   • Does the metric help track sustainment affordability?
6. Feedback-driven adjustability: Some MOEs or MOPs might cease to be relevant at some point and new ones may be added as new stakeholders and/or concerns surface. For example, leaders might want to include a MOP related for user errors once the system is deployed. Leaders will not need that MOP until there is enough of the framework and transitions complete.
   • User error rate
   • Number of users trained
   • Time to complete tasks
   • User complaint rate (bugs, misunderstandings, missing capabilities, etc.)
   • System availability
   • System interoperability
   • System update rate (how often are updates made to fix something)
   • System resource utilization:
     • Are processors swamped or hardly used?
     • Is there enough data?
     • What is the rate at which the framework runs out of resources?, etc.

Key questions relative to feedback-driven adjustability are:

• How can feedback loops be designed to update metrics to reflect emerging challenges and opportunities?
• What mechanisms are in place to capture and analyze real-time data to enhance dynamic decision making?
• How can metrics be adapted over time to remain relevant to the evolving goals of the DT initiative?
• Are there means in place to collect stakeholder feedback on the relevance and accuracy of metrics?

HYPOTHETICAL USE CASE EXAMPLE: PROJECT NIGHTLIGHT

With the common elements and associated consideration questions as background, the committee considered a cursory thought experiment where ideal conditions lead to exceptional success for a DE effort within a broader DT initiative. In this fictional world, visionary leadership, strategic resource allocation, robust secure IT, standardized software and data protocols, and meaningful metrics seamlessly integrate. The result is an agile, systems-thinking approach to DE, leveraging AI-powered solutions and a resilient end-to-end life-cycle framework that empowers innovation, efficiency, and collaboration across all organizational levels and stakeholders. For the purposes of this thought experiment, resourcing constraints such as timing and funding have been mostly minimized … truly a fictional thought, if ever there was one.

Leadership and Adoption Culture

In this best-case scenario, DAF leaders foster a culture of innovation and adaptability. Project Nightlight program leaders, supported by a centralized Digital Transformation Office (DTO), establish a clear vision for DE that is aligned with mission-critical objectives. They prioritize transparency and communicate frequently with stakeholders through town halls, workshops, and digital platforms, showcasing tangible examples of DE’s transformative impact. This builds trust and enthusiasm across the organization.

Specialized career pathways for architects, engineers, operators, and testers, ensure that the appropriate Airmen and Guardians workforce are equipped—by way of hiring or training—for the life cycle demands of DE.

Agility underpins DAF’s success. DAF leaders have largely empowered their DTO experts to maintain flexibility in policies, promoting an 80/20 model where 80 percent of tools are standardized across the enterprise, while 20 percent remain customizable for unique program needs. This balance minimizes friction while maximizing efficiency. Collaboration with industry, academia, and other govern-

ment entities further enhances capabilities by fostering joint investments in training and technology development. Real-world experimentation in digital sandboxes accelerates innovation, ensuring that Project Nightlight and other acquisition programs with the U.S. Air Force and the U.S. Space Force remains resilient and adaptive to future challenges.

Funding and Incentives

The DAF program objective memorandum fundings are strategically designed to support both foundational DT efforts and program-specific needs. Centralized funding mechanisms provide scalable resources for enterprise-wide tools and platforms, while decentralized program managers at Project Nightlight and other programs retain the flexibility to address unique requirements. Project Nightlight and other early adopters of end-to-end life-cycle DE are incentivized with additional funding to mitigate initial technical barriers and to serve as proof-of-concept that will aid broader enterprise adoption.

DAF leaders track the savings achieved through DE initiatives, such as reduced life-cycle costs and improved sustainment efficiency, and reinvest them into DT efforts. Leaders reward teams that demonstrate exceptional results, fostering a culture of accountability that supports excellence. Agile funding strategies enable rapid response to emerging requirements, ensuring that DE initiatives remain aligned with long-term objectives. Additionally, proactive budgeting for emerging tools with AI helps the DAF become future-ready, with resources allocated for continuous improvement.

Software and Data Standards

The Air Force and Space Force empower their DTOs—through additional personnel, top cover, and DT championing from a four-star general officer—to establish a robust framework for software and data standards that prioritizes openness, flexibility, and interoperability. A unified digital thread connects every stage of a new program’s life cycle, from concept to sustainment, providing stakeholders with an authoritative source of truth. Legacy programs are integrated with the DE that leads to cost efficiencies and benefits. All tools and data are standardized using open formats, supporting seamless integration across diverse systems and minimizing “vendor lock.”

AI-driven solutions also play a role in advancing DE practices. AI-powered metamodels enhance modeling and simulation by improving fidelity and reducing uncertainty. Automated data validation and translation tools streamline workflows, enabling faster decision making and reducing errors. Modular open system archi-

tectures allow for third-party adaptability, while clear application programming interface definitions to keep tools compatible with future technologies.

Project Nightlight, other acquisition programs, and the Air Force and Space Force leverage advanced knowledge management systems to capture lessons learned and disseminate best practices. FAIR principles are applied rigorously, ensuring that digital assets are easily searchable, accessible, and reusable. Collaborative platforms facilitate real-time data exchange among stakeholders, accelerating program cycles, and fostering innovation. Standards-compliant frameworks, provide the consistency needed for scalability while allowing flexibility for evolving requirements.

Hardware and Secure Information Technology

A resilient and secure IT infrastructure underpins the organizations’ DE success. An open architecture enables interoperability and collaboration between integrators, vendors, and other stakeholders. The infrastructure is designed with both near-term operational needs and long-term scalability in mind, supporting a flexible and adaptive ecosystem.

Cybersecurity is a top focus. Proactive strategies, such as continuous monitoring, black hat/white hat testing, and automated threat detection protect against vulnerabilities in connected systems. High-speed, secure IT capabilities also support real-time collaboration while reducing delays and improving decision making within the teams. Modular and plug-and-play designs enhance reusability, creating efficiencies that accelerate development timelines and reduce costs.

DTO leaders conduct proactive planning activities to anticipate future needs and comprehensive market research to identify cutting-edge tools that will meet those needs. Infrastructure investments align the DE systems to remain operationally effective and adaptable to changes necessitated by emerging technologies.

Metrics

DTO leaders, in conjunction with program leaders for Project Nightlight and other programs as well as Air Force and Space Force leaders, create and integrate a comprehensive framework of metrics to evaluate and guide DT initiatives. These metrics provide both quantitative and qualitative insights, enabling stakeholders to measure progress, identify areas for improvement, and assess return on investment. Key performance indicators track cycle times, defect rates, and milestone completions, demonstrating the efficiency and effectiveness of DE processes.

Predictive metrics support scenario planning and procurement decisions allowing leaders to anticipate challenges and optimize outcomes. Continuous performance tracking provides real-time feedback, ensuring that programs stay aligned

with organizational objectives. Metrics are also used to evaluate workforce training effectiveness and other skill recruitment and retention efforts.

By establishing a constant feedback loop, DAF programs and enterprise leaders are properly informed to make dynamic adjustments to their DE strategies. Metrics are aligned near-term focus with long-term outcomes to measure progress, provide accountability and transparency, and support success of DT efforts.

Concluding Thoughts

In this hypothetical future, DAF success in DE and DT is driven by the seamless integration to promote visionary leadership, strategic funding, secure IT infrastructure, standardized software and data protocols, and actionable metrics. As Airmen and Guardians internalize an agile mindset, systems thinking, and digital solutions, the organizations create a resilient and adaptive framework that not only meets current challenges but also positions their enterprises for enduring innovation and excellence. This best-case scenario demonstrates how a unified approach to DE and DT can transform organizational capabilities and achieve mission success.