4

Incentives

IMPORTANCE OF INCENTIVES

U.S. commercial technology providers, in part because of the highly competitive nature of their operating environment, have produced world-leading and world-changing capabilities that are both agile and assured. Mature competitive markets that foster such innovation are characterized by

• Large-scale supply and demand with many and diverse buyers and sellers;¹
• Common, available, scalable, “cheap” infrastructure, such as cloud computing and networking (removing barriers to entry);² and
• Free movement of capital and talent.³

A rich market for product offerings with similar core functionality and unique differentiating features results. Furthermore, these properties influence the granularity and frequency of purchasing decisions, which in turn influence the pace of market evolution. If purchasing decisions are infrequent and locked in for long periods of time, then the pace of evolution of desired characteristics is slow. If purchasing decisions are rapid and frequent, then the pace of evolution is fast.⁴

___________________

¹ Course Sidekick, n.d., “Perfect Competition | Boundless Economics,” Course Sidekick, https://www.coursesidekick.com/economics/study-guides/boundless-economics/perfect-competition.

² Ibid.

³ A. Hayes, 2024, “Perfect Competition: Examples and How It Works,” Investopedia, https://www.investopedia.com/terms/p/perfectcompetition.asp.

⁴ Ibid.

In mature competitive markets, agility and assurance have become “table stakes.” Industry successes in developing agile and assured systems come from companies for which lack of agility or resilience is a significant, often existential, business risk. This risk creates a forcing function. Driven by this pressure to survive, commercial vendors continuously seek out most effective ways of achieving competitive levels of agility and assurance. They learn from academia and from each other, experiment, and, through trial and error, find approaches that work for them and their specific technology stack, market constraints, and corporate culture. Product and service availability and security must be high, and the ability to incorporate new capabilities and/or support frictionless second- and third-party introduction of capability is essential to maintain and advance in their markets.

Incentive alignment is key to competing successfully in such environments, as the key means of motivating organizational and individual behaviors. These motivators include the following:

• Clear outcome goals with articulated and objective measures of success and measures of effectiveness,⁵
• Incentives for performance improvements,⁶ and
• Rewards for innovation.⁷

Although it is clear that incentives for more assured products are not uniform in commercial industry, they are acutely felt by dominant market players in some of the most critical industry segments. Although hard data on this subject are scarce, the experience shared by committee members from commercial industry, as well as that shared during multiple briefings from experts, consistently demonstrated that assurance is among the top business concerns with direct impact on the market performance of companies, garnering regular attention from the C-suite. In contrast, the experience of committee members from the defense industry and briefings from experts indicated that security vulnerabilities would have no perceptible impact on the market position of defense contractors. Moreover, due to the high capital expenditure nature of many defense systems, software development costs in general make up only a small portion of the overall cost. As a result, efforts to improve agility and reduce associated costs receive little priority.

Much of the Department of Defense (DoD) market (e.g., for supporting functions such as personnel and strategic logistics) can and should be satisfied via competitive contracting with commercially aligned incentives.

___________________

⁵ J. Phillips, 2004, PMP Project Management Professional Study Guide.

⁶ C.M. Christensen, 2015, The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail, Harvard Business Review Press.

⁷ Ibid.

For those aspects of the defense acquisition market providing unique, “exquisite” capabilities, DoD can and should distill the lessons and incentives inherent in competitive markets as a means to attain more and better assurance and agility. In particular, incentives for performance improvement and innovation should be emphasized and significantly rewarded within DoD acquisition processes. For example, supplier revenue should be tied not only to provisioning of capability but also to platform assurance and agility, as well as market “penetration” and uptake. Questions to consider include the following: Is this platform the preferred platform for potential second- and third-party add-on capability providers? Second- and third-party acquirers? Can the platform provide capabilities outside of its original scope, quickly and correctly, through providers not associated with the original program? Can the platform leverage emerging technology without significant refactoring and rework? Affirmative answers to these questions would indicate that a platform has been thoughtfully architected to be both agile and assured.

A wholesale restructuring of the defense acquisition system is well beyond the study statement of task, but it is the committee’s hope that future studies will explore the role of market incentives in shaping outcomes and organizational culture. The focus of this chapter is to identify aspects of commercial practices contributing to better alignment of incentives with desired outcomes that can, and should, be applied to DoD complex systems development. It is unlikely that the market will deliver what it is not incentivized to deliver, and it is virtually certain that it will not deliver what it is actively disincentivized to deliver. That is why addressing incentive alignment is a prerequisite to achieving greater agility and assurance in defense systems.

WHAT WORKS WELL TODAY AND WHAT DOES NOT

As discussed elsewhere in this report, DoD has had only limited experience in transitioning software acquisition programs to the software acquisition pathway, although there has been some experience with the adoption of modern development practices. Although several programs reported success with those practices, their adoption can present challenges.

To take one example, the Army and Marine Corp the Advanced Field Artillery Tactical Data System (AFATDS)⁸ has been in service since the mid-1980s and is the underpinning of U.S. and Allied Joint Fires for Artillery, Missiles, Counter Battery radars and Fire planning. It has more than 8 million lines of code and has gone through more than 17 upgrades by Raytheon (RTX). The Army’s attempts to recode AFATDS from Ada to Java and use

___________________

⁸ S. Lindecamp, E. Keele, and D. LaFontaine, 2017, “Give More, Get More,” Army ATL Magazine, August 30, https://asc.army.mil/web/news-alt-ond17-give-more-get-more.

DevSecOps and a modular architecture have been problematic. Contractors were selected based on technical skill sets without sufficient consideration of (tactical fires) domain expertise, and the modernization effort has cost more than $100 million with few tangible results. Additional funding is planned over the next 5 years to attempt to mitigate problems and modify the legacy code. The AFATDS experience demonstrates the need for both software technical skills and deep “domain expertise” in implementing modern software approaches such as the software acquisition pathway. Deep technical skills alone will not succeed in rebuilding or building a large-scale software program.

An additional challenge to large-scale DoD acquisitions is that experimental units can bear little resemblance to initial production units, resulting in little to no experience for the platform provider with the fundamental challenges to production and operations that must be addressed in the architecting and early design processes. An example of this can be identified in the U.S. Air Force F-35 program. Consider the Helmet-Mounted Display (HMD) that incorporated emerging display technology and a completely redefined user experience in a form factor heretofore unavailable as a fundamental part of the program baseline. The HMD represented one “small” area of the inherent technical risk to the program and, as might be expected, failed to resolve in a timeline and capability that comported with the program baseline. Too much science, culture, and manufacturing risk was incorporated into this small but vital component, resulting in program overruns and ever increasing pressure on the program.⁹

The Intelligence Community (IC) recognized situations such as these and added a separate phase—and process (the equivalent of a DoD pathway)—for acquiring experimental units, called an Experimental Research Demonstration (ERD), as articulated in Intelligence Community Standard 801-02¹⁰:

___________________

⁹ Government Accountability Office (GAO), 2024. “F-35 Joint Strike Fighter: Program Continues to Encounter Production Issues and Modernization Delays,” GAO-24-106909, https://www.gao.gov/products/gao-24-106909.

¹⁰ Office of the Director of National Intelligence, 2013, “Experimental Research Demonstration,” Intelligence Community Standard ICS 801-02, June 18 (available in the public access file for this study [Email: paro@nas. edu]).

An ERD requires a specific experimental development plan that satisfies formal program oversight requirements while also demanding the tailoring of the DoD 5000 series regulations to acknowledge the high and inherent risks of audacious technical undertakings, providing programs with the needed opportunities to experiment, learn, and redesign while setting realistic expectations for oversight and governance. Several highly classified IC programs have implemented ERDs for initial units, resulting in innovative mission capabilities and high praise from oversight organizations and suppliers alike. Better understanding of operations and mission use have resulted in formal acquisition of additional units at reduced non-recurring engineering costs, highly stable and predictable program baselines, and fast operational evaluation and employment.

Another challenge to complex DoD systems acquisition is that often the elapsed time between identification of the mission need and availability of the first production unit is measured in multiple years. The U.S. Navy AEGIS¹¹ program has been, for years, exemplary in accepting and leveraging this reality. For example, the program managers and their oversight have been quite comfortable projecting computation requirements for hard-real-time command and control systems into the future and then, based on Moore’s law, designing to those specifications in anticipation of technology closing any existing gaps in performance.

Digital twins—virtual models of physical assets that allow for real-time data analysis and simulation—that are continuously maintained provide an alternative to long time-lines in terms of system evaluation and capability assessment. Security models incorporated in the digital twin can be updated, incorporating new environmental attack vectors and flagging points of potential vulnerability that may require redesign. However, application of digital twin technology, other than in training environments, is nascent. Beyond some limited experimental attempts,¹² the use of digital twins has not been broadly embraced by DoD. Although there are risks that digital twins may not achieve perfect fidelity, they, like other model-based engineering approaches, offer significant potential for avoiding surprises late in a development program, and they have seen successful adoption by commercial concerns such as the automotive industry (see below).

___________________

¹¹ R. Frew, 2024, “Perspective on Software Assurance and Nimbleness from the Defense Industry,” Presentation to the committee, September 27; R. Jandrian and C. Toney, 2024, “United States Navy Perspective on the Navy Software Programs,” Presentation to the committee, November 22.

¹² J. Bennet, 2023, “USAF Awards $100M Digital Transformation & Sustainment Contract to WSU Aviation Research Institute,” ExecutiveGov, https://executivegov.com/2023/08/usaf-awards-100m-digital-transformation-contract-to-wsu-niar; U.S. Navy, 2023, “Electronic Attack Systems Program Using Digital Twins to Impact Naval Aviation,” Naval Aviation Enterprise Communications, https://www.navy.mil/Press-Office/News-Stories/Article/3518626/electronic-attack-systems-program-using-digital-twins-to-impact-naval-aviation; K. Pettaway-Clarke, 2 022, “CCAD Leverages Digital Twin Scan Data from 8360 Scanbox to Automate Measurement,” Defense Visual Information Distrobution Service, https://www.dvidshub.net/news/424971/ccad-leverages-digital-twin-scan-data-8360scanbox-automate-measurement.

“Insanity” is often defined as “doing the same things over and over, hoping for a different outcome.”¹³ If DoD truly wants to drive agile, assured platforms for their application, proven incentives and their resultant models should be embraced and adapted for complex system development and delivery. The sections that follow address commercial examples and then discuss key aspects of the DoD acquisition process and recommended steps to address the incentive gaps identified herein.

SOME COMMERCIAL EXAMPLES

Apple and Android’s mobile device platforms and associated app stores are well-known examples of well-architected, assured, and agile instances of a platform that requires no centralized system integrator to yield a rich market of solutions competing with each other on functionality, agility, and assurance. Apple’s App Review Guidelines¹⁴ include requirements for functionality, design, content, privacy, safety, and legal compliance, ensuring the app is well built, provides value to users, and adheres to Apple’s standards for user experience and data handling. The guidelines address technical quality, user experience, content appropriateness, privacy, legal compliance, guidelines adherence (e.g., functionality and suitability), accessibility, and listing quality.

Apple relies in part on revenue from app sales and subscriptions to sustain the platform and associated applications infrastructure, while the app providers reap the benefit of highly usable, fit-to-purpose capabilities that leverage the underlying platform services. Such co-dependent ecosystems incentivize innovation and concurrently inform platform capability evolution and innovation.

Another example of a highly successful commercial platform–based approach that enjoys both agility and high assurance is the Amazon Web Services (AWS) microservices architecture.¹⁵ A set of composable services, including compute, networking, storage, data, and monitoring, provides a complete development and uniform run-time environment through well-defined, well-behaved invocation interfaces. Many of the architectural “plumbing” decisions are fixed in such a way as to mitigate typical assurance risks and increase overall application agility. AWS market data suggests that, as the incumbent

___________________

¹³ Wikipedia contributors, 2025, “Rita Mae Brown,” Wikipedia, The Free Encyclopedia, https://en.wikiquote.org/wiki/Rita_Mae_Brown.

¹⁴ Apple, n.d., “App Review Guidelines,” Apple, https://developer.apple.com/app-store/review/guidelines, accessed November 19, 2024.

¹⁵ Amazon Web Services, n.d., “What Are Microservices?” https://aws.amazon.com/microservices, accessed June 11, 2025.

market leader, Amazon has successfully architected a platform approach that satisfies major swathes of enterprise needs across most market verticals.¹⁶

Part of Amazon’s ability to capture and take an early lead in its market evolved around early definition and provisioning with service-level agreements (SLAs).¹⁷ Patterned on telecommunications industry practices, SLAs provide contractual agreements with customers that specify minimum performance of their services. These can include such things as uptime, data transfer speeds, and response times. Failure to adhere to these SLAs can result in significant monetary penalties as well as brand damage. Customers can choose to buy more or less assurance, with escalating costs for more, while Amazon is strongly incented to ensure that its performance is consistent with or better than these SLAs.

For hard-real-time command-and-control platform applications, automotive manufacturers and their use of digital twins offer another highly successful commercial platform–based approach. In the automotive industry, 75 percent of companies have adopted digital twin technologies, according to McKinsey.¹⁸ Toyota,¹⁹ Tesla,²⁰ BMW,²¹ and Audi²² are heavily invested. Digital twins can help automakers design and develop new cars; mix, fix, and build cars; create sustainable products; predict wear and tear; improve safety features; refine performance metrics; and optimize energy usage. Even better, these digital twins allow critical suppliers to understand their contribution’s role and functionality in production and support widespread “innovation from the bottom,” which is so characteristic of the automotive industry.²³

Imagine if DoD weapons systems required no formal systems integrator and different sensors, capabilities, and components could be plugged-and-played over the life of the system with high confidence in the assured behavior of the underlying software platform and the weapons system as a whole. Certain capabilities might require additional incentives for production, but overall, such an approach could revolutionize the DoD acquisition governance and procurement processes. It would likewise require different

___________________

¹⁶ Synergy Research Group, 2024, “Cloud Market Growth Surge Continues in Q3 – Growth Rate Increases for the Fourth Consecutive Quarter,” https://www.srgresearch.com/articles/cloud-market-growth-surge-continues-in-q3-growth-rate-increases-for-the-fourth-consecutive-quarter.

¹⁷ Amazon Web Services, n.d., “AWS Service Legal Agreements (SLA),” https://aws.amazon.com/de/legal/service-level-agreements, accessed June 11, 2025.

¹⁸ Ascentt, 2024, “How Digital Twins Accelerate Automotive Innovation,” https://www.ascentt.com/how-digital-twins-accelerate-automotive-innovation.

¹⁹ Toyota Europe, 2023, “Our Manufacturing Companies Have Digital Twins,” https://www.toyota-europe.com/news/2023/our-manufacturing-companies-have-digital-twins.

²⁰ B. Marr, 2022, “The Best Examples of Digital Twins Everyone Should Know About,” https://www.forbes.com/sites/bernardmarr/2022/06/20/the-best-examples-of-digital-twins-everyone-should-know-about.

²¹ BMW Group, 2023, “This Is How Digital the BMW iFactory Is,” https://www.bmwgroup.com/en/news/general/2022/bmw-ifactory-digital.html.

²² Audi Media Center, 2022, “Smart Production: How Audi Is Designing the Production of the Future,” https://www.audi-mediacenter.com/en/press-releases/smart-production-how-audi-is-designing-the-production-of-the-future-14786.

²³ Toyota Management System, 2023. “Toyota’s Digital Transformation in Manufacturing,” https://www.in-eak.com/toyotas-digital-transformation-in-manufacturing/#google_vignette.

incentives for platform providers, such as platform lifetime, platform consumption, fit-to-purpose, and overall performance parameters. Incentives for “app” providers could be based on characteristics such as service consumption, fit-to-purpose, and SWAP (size, weight, and power) consumption.

Clearly, a transition approach for acquisition governance and procurement, combined with an agreed and robust platform definition, would be essential. This report’s recommendations propose that DARPA explore each of these as part of its response to this study. By marrying tools, processes, and outcomes to existing and new DARPA initiatives, DARPA could “dogfood” these approaches—demonstrating capability and driving the diversity of innovation and supply chains while mitigating risk for widespread and successful DoD adoption.

FINDINGS AND RECOMMENDATIONS

Program Definition

DoD acquisition methodologies incent only timely delivery and performance of the prime contractor and its subcontractors. There are no measurable, meaningful incentives that reflect “ilities” such as agility, assurance, reusability, or maintainability. Competitive pressures do not exist post-contract award. DoD unique systems’ requirements often demand development, test, and evaluation infrastructures that are purpose-built and highly secure. Most of these infrastructures are obsolete or aged (e.g., shipyards, radio frequency test ranges, wind tunnels) with limited commercial need or application. Capital requirements are specified at program establishment (e.g., Major Defense Acquisition Program designation) and, while these can be modified during program execution, modifications may require significant program restructure and justification if pre-established growth limits are exceeded. Many of the engineering tasks involve classified material, necessitating cleared personnel for the work to be done. That, in turn, both limits the talent pool in absolute numbers, and further restricts the flow of talent in and out of a given program, choking off the experience exchange that characterizes the private sector. As a result of regular military rotation cycles, the program manager tasked to deliver the capability originally is likely multiple assignments away from the final delivery date when accountability is demanded.

Taken together, these operational constraints in the DoD acquisition environment present significant challenges that undermine efficiency, flexibility, and long-term system sustainability. These constraints manifest in several ways, from outdated metrics of success to fragmented approaches for integrating third-party components and transitioning systems to more modern architectures. Addressing these constraints requires a systemic

reevaluation of how systems are architected, contracts are structured, and updates are funded and executed.

Finding 4-1: Commercial information technology incentives drive agility and assurance, largely manifested as resilient platforms that underpin rich ecosystems of independent third-party innovation. Both incentives and platforms (as industry defines them) are largely absent in DoD software development. DoD culture does not sufficiently prioritize supplier behavior with aligned, substantial incentives.

Recommendation 4-1: The Office of the Under Secretary of Defense for Acquisition and Sustainment should modify Department of Defense Instruction 5000.02 to require and incent that software-intensive systems be based on extensible platforms and architectures that support iterative extension of capabilities by a broad set of suppliers (beyond the original platform vendor) with clear delineation of long-term maintenance and currency responsibilities.

Contracts and Acquisitions

The traditional DoD approach to developing systems is to contract with industry (defense contractors) to deliver a set of capabilities laid out in a specification by performing the tasks laid out in a statement of work (SoW). The contract may be fixed-price, in which case the acquiring organization has very strong motivation to avoid any changes to the specification or SoW, or it may be a form of cost-reimbursement contract, which provides more flexibility and adaptability to unexpected developments, but also more financial (and, typically, schedule) risk. From the description of the software acquisition pathway, it is clear that fixed-price contracting is inconsistent with a pathway approach.

The Other Transaction Authority (OTA) experimental acquisition process offers a flexible and innovative contracting mechanism that is particularly well suited to agile software development across DoD. Unlike traditional fixed-price or cost-reimbursement contracts, OTA agreements enable rapid prototyping and iterative development by allowing nontraditional defense contractors to engage in defense projects without the burden of extensive regulations that are required in traditional DoD contracts. This approach has proven effective in attracting innovative companies to defense programs, fostering competition, and accelerating technology transitions.²⁴ However, while OTAs provide much-needed agility, their effectiveness hinges on clear accountability structures and robust oversight mechanisms. For example, a requirement for an assurance case, as

___________________

²⁴ See, for example, Vertx Partners, 2023, “5 OTA Success Stories Highlighting How OTAs Can Benefit You,” May 9, https://vertxpartners.org/5-ota-success-stories-highlighting-how-otas-can-benefit-you.

discussed in Chapter 3, might be one such accountability structure. Without these, OTAs risk producing outputs that fail to meet mission needs, leading to mismatches between delivered capabilities and operational requirements.

Thoughtful and informed pairing of contract methodology with development environment and outcomes is essential to delivery success. For example, if a well-defined architecture exists and the developer is delivering an application programming interface (API), then a fixed-price contract should work well. If the developer is delivering an application and user feedback is essential to be incorporated into the development process, then cost plus could make more sense. In either case, government management should be equipped, technically and programmatically, to manage overall system delivery.

Recommendation 4-2: The Office of the Under Secretary of Defense for Acquisition and Sustainment should identify revenue-sharing methods and metrics between platform developers and application or component developers (who would be referred to as third-party developers in the commercial off-the-shelf world—the platform vendor is the “first party” and the end customer the “second”), as exemplified by the commercial sector. For example, a percentage of third-party provider payments or award fees could accrue to an extensible, resilient platform provider. This modification, if properly structured to protect the equities of competing contractors, would incentivize the delivery of high-quality software platforms that could be applied broadly and would provide the Department of Defense with more flexible and adaptable systems.

Platform Architecture and Third-Party-Driven Innovation

The integration of third-party plug-ins is another area of concern, as many systems are not designed to accommodate external components seamlessly. This lack of foresight often results in bespoke and costly integration efforts that inhibit flexibility and stifle innovation. The DoD should require that systems be designed with open interfaces and modular architectures from the outset, enabling the incorporation of third-party plug-ins and reducing the dependency on proprietary solutions. Leveraging Government Reference Architectures (GRAs) and open standards, such as Open Mission Systems (OMS),²⁵ can further facilitate third-party integration, fostering a competitive ecosystem of suppliers and enabling faster deployment of new capabilities.

High-fidelity digital twins may be another mechanism for enabling the effective use of robust platforms in the face of complex, multi-vendor supply chains. Even better,

___________________

²⁵ Visual Distributed Laboratory, 2024, “Open Mission Systems (OMS),” June 10, https://www.vdl.afrl.af.mil/programs/oam/oms.php.

it permits every level of supplier to examine their interactions for opportunities to correct or improve their component experience.

Recommendation 4-3: The Defense Advanced Research Projects Agency (DARPA), in partnership with the Office of the Under Secretary of Defense for Acquisition and Sustainment (OUSD(A&S)) should develop robust platform definitions and attributes, to include incorporation of already evaluated code and/or platforms, and systematically pilot, measure, and assess these in its own development initiatives. Such attributes should also address DevSecOps environments associated with those platforms. Attributes that DARPA validates should, likewise, be continuously incorporated into OUSD(A&S) standard acquisition practices. Implementation of this recommendation would enable program offices to require and contractors to deliver more flexible and assured software systems.

Measures of Success

Based on the committee’s collective experience and meetings with numerous experts, it found that a persistent challenge in DoD acquisitions is the lack of accountability for mission-aligned outcomes, particularly in large, complex programs. Too often, as a result of the operation of the DoD contracting system, the focus remains on meeting contractual obligations rather than ensuring that the system delivers on its intended mission. This misalignment results in systems that technically meet specifications but fail to address real-world operational challenges. For example, the absence of clear mission-based metrics for success and mechanisms to adapt to shifting requirements leaves the government unable to hold contractors accountable for delivering meaningful capabilities. Aligning acquisition incentives with mission assurance—such as rewarding contractors for mission-focused innovation and penalizing failures to address operational needs—would significantly improve outcomes.

Recommendation 4-4: The Office of the Under Secretary of Defense for Acquisition and Sustainment, in partnership with the Defense Acquisition University, should develop and pilot life-cycle metrics to evaluate secure, extensible platforms and establish a Department of Defense–wide acquisition framework for them.

Recommendation 4-5: The Office of the Under Secretary of Defense for Acquisition and Sustainment, in partnership with the Defense Acquisition University, should conduct a longitudinal study of the longest-serving and most successful acquired capabilities, identifying their attributes and

candidate metrics, both historical and current. Lessons learned and success metrics should be incorporated into those frameworks and toolsets identified in each of the identified areas with regular revisit and update to ensure continuous process improvement and associated culture change.

Recommendation 4-6: The Office of the Under Secretary of Defense for Acquisition and Sustainment should continuously evaluate ongoing Department of Defense acquisitions and establish a regular assessment/reassessment framework for acquisition process improvement, drawing in the Defense Advanced Research Projects Agency for those areas that require breakthrough systems and complex capability pilots.

Program Scope

Prototyping

Too often, DoD program leaders are required to provide both prototypes and deliverables in the same Major Defense Acquisition Program (MDAP), even though the risk of those phases is significantly different. For example, significant refactoring of mission capabilities is incorporated into next-generation replacement acquisitions, increasing overall program risk and incenting basic capability delivery at the expense of assurance and infrastructure agility. Separating out low technology readiness level (TRL)²⁶ and manufacturing readiness level (MRL)²⁷ early development from large-scale production and acquisition contracting, as articulated in Intelligence Community Standard 801-02, has worked well for the IC in addressing architectural trades early on, resulting in more highly assured and agile systems and composite software architecting.

Sustainment

On the opposite end of a platform life cycle, DoD system sustainment is too often incented as a continuing revenue stream long after the platform has been delivered. Commercially, cloud and other software as a service providers incur additional revenue streams only when new services are provided. Fixing and distributing fixes for flaws are acceptable only for new threats to the operating environment. Discovering bugs that could negatively impact service-level agreements can cost the provider significant penalties—as well as brand recognition—in a highly competitive market (e.g.,

___________________

²⁶ NASA, 2023, “Technology Readiness Levels,” September 27, https://www.nasa.gov/directorates/somd/space-communications-navigation-program/technology-readiness-levels.

²⁷ Wikipedia, n.d., “Manufacturing Readiness Level,” https://en.wikipedia.org/wiki/Manufacturing_readiness_level, accessed June 11, 2025.

CloudStrike²⁸). Commercial providers track individual team and team member contributions to source and create leaderboards and financial constructs that penalize consistently “buggy” delivery and reward “bug free” delivery. Verified code reuse, discussed elsewhere in this report, is an excellent baseline for reducing bug counts and improving maintainability and assurance.²⁹

Technology Stack Refresh

Annual refreshes of the technology stack within the first few years of a program are often deferred due to budget cuts, leaving systems outdated and increasingly difficult to maintain. Modern systems and their digital twins require regular updates to remain compatible with emerging technologies and to address evolving threats. Technology stack updates should start with cleanly defined modules, as identified in the assurance and agility chapters. Failure to refresh technology stacks exacerbates technical debt and reduces the agility of defense systems. A solution is to mandate periodic updates within the contract structure, ensuring that funding for these updates is non-discretionary and prioritized within program budgets. This approach would allow DoD to keep systems technologically current, improving their long-term viability and adaptability.

Finding 4-2: Successful commercial program leadership clearly delineates the boundaries between prototype and production, and production and operational sustainment capabilities. Often, DoD program leaders are required to both prototype and deliver manufacturing end items in the same budget and schedule profile. Conversely, DoD program leaders are not incented to address long-term sustainability of expensive platforms generally delivered in a rush to the “finish line.”

Recommendation 4-7: The Office of the Under Secretary of Defense for Acquisition and Sustainment should modify the software acquisition pathway to require that contracts for software stipulate operational support periods for the expected lifetime of the system, including service-level agreements for remediation of discovered vulnerabilities.

Recommendation 4-8: Using the Intelligence Community’s Experimental Research Demonstration as one input, the Office of the Under Secretary of Defense for Acquisition and Sustainment and the Defense Acquisition

___________________

²⁸ A.R. Sorkin, R. Mattu, B. Warner, S. Kessler, M. Merced, L. Hirsch, E. Livni, and A. Gaffney, 2024, “Counting the Costs of a Global IT Outage,” New York Times, July 19, https://www.nytimes.com/2024/07/19/business/deal-book/tech-outage-crowdstrike-microsoft.html.

²⁹ D. Feitosa, A. Ampatzoglou, A. Gkortzis, S. Bibi, and A. Chatzigeorgiou, 2020, “CODE Reuse in Practice: Benefiting or Harming Technical Debt,” Journal of Systems and Software 167:110618, https://doi.org/10.1016/j.jss.2020.110618.

University should develop and pilot criteria that explore and incent experimental revolutionary innovation for platforms, as opposed to current Department of Defense Instruction 5000–based evolutionary innovation.

Recommendation 4-9: The Defense Advanced Research Projects Agency (DARPA), in partnership with the Office of the Under Secretary of Defense for Acquisition and Sustainment, should develop and pilot service-level agreements (SLAs) for classes of software-intensive systems and assess these in its own development initiatives. SLAs that DARPA validates should be used to inform and establish a Department of Defense–wide acquisition framework for them. Such SLAs could be a valuable input to development of relevant and cost-effective incentives for contractors.

Program Security

Evaluating Operational Fit

The previous sections have discussed the development of assured software. One factor that several people who briefed the committee cited as impacting the timeliness and assurance of software development is the security environment in which development occurs. In some cases, developers may not be permitted access to real-world data for testing or may be restricted from conducting tests on an integrated system that includes all subsystems or components. Instead, the developers complete their parts of the system as best they can and then hand them off to a separate organization that integrates them for system-level testing, and then returns test results. The returned results may even be sanitized of specifics of the data that caused test failures.

Agile development requires thorough testing early in the development process and high-fidelity feedback from users. To the extent that security restrictions block the satisfaction of these requirements, they can have significant negative impacts on program success. DoD should carefully consider the potential impacts on program success of program security measures with the aim of providing mechanisms for rapid and complete feedback to developers.

Recommendation 4-10: The Office of the Under Secretary of Defense for Acquisition and Sustainment and the Office of the Under Secretary of Defense for Intelligence & Security should jointly review and revise the Department of Defense’s security and clearance policies to minimize impediments to agile software development and testing.

Vulnerability Management

Compounding previously discussed development and test practices is the treatment of vulnerability remediation as an additional cost. This approach often leads to delays in addressing critical security issues and incentivizes contractors to deliver minimally compliant systems without regard for long-term security. Moreover, end-of-life (EoL) support for legacy systems is frequently handled through costly stopgap measures, placing an undue burden on government budgets. Contracts must shift to include ongoing vulnerability management and resolution as a fundamental, non-negotiable requirement. By embedding these costs into initial contracts and holding contractors accountable for timely fixes, DoD can ensure that systems remain secure and operational without incurring excessive additional expenses.

Finding 4-3: Commercial providers have intimate knowledge of use cases critical to user success that also incorporate the realities of continuously evolving threat environments. Too often, DoD obfuscates critical use cases and detailed threat descriptions to the detriment of systems in the acquisition pipeline, rendering them ineffective in the end state operating environment.

Recommendation 4-11: The Office of the Under Secretary of Defense for Acquisition and Sustainment should require that the continuous Authorization to Operate implementation mandated by the software acquisition pathway rely principally on automatically generated evidence that is updated with every delivery milestone and maintained over the system’s life cycle. Regularized, independent red team analysis and remediation of identified system’s interface (e.g., systems’ seams) vulnerabilities should be identified and evaluated for system-to-system induced vulnerabilities.

Recommendation 4-12: The Defense Advanced Research Projects Agency, in coordination with the Intelligence Advanced Research Projects Activity, should develop and pilot a cybersecurity information-sharing framework that protects highly sensitive equities while continuously revealing most current and detailed cyber threat information. Working with the Office of the Under Secretary of Defense for Acquisition and Sustainment, this framework should be made securely available to the Department of Defense acquisition community and continuously maintained as current.

Program Talent

DoD software factories, such as Kessel Run, are good proof points for the software acquisition pathway. The challenge associated with following the pathway for system development seems to be with assuring an experienced workforce rather than the pathway itself. For example, Kessel Run³⁰ and the Navy’s submarine software programs³¹ operate by hiring contractors whose deliverable is staff hours of development effort and then assigning contractor personnel to development teams as needed. The government has great flexibility in terms of what the contractors will do, but also great responsibility for choosing the right tasks and monitoring the development to detect problems early. This is a very different model from traditional government contract work, but it appears to have been successful in the programs that were described to the committee.

While DoD has focused initiatives on software-intensive systems leadership and development cadre activities, the Kessel Run and Navy Submarine program experiences still represent the exception and not the rule.³² Government-led program teams must combine deep understanding of the mission environment and needs with deep, current technical experience and with excellent leadership and people skills. Too often, government program leads³³ are steeped in mission and leadership expertise, but their “technical” expertise is focused on successfully navigating the programmatic processes associated with establishing and assuring continued funding of their program. Any real technical expertise they have was obtained very early in their careers and is significantly outdated. Furthermore, this experience-based expertise will likely not encompass the various technical disciplines that must be knitted together for the successful execution of their current program.

For example, program managers and architects with long-term deliverables should incorporate expected gains in underlying component performance (e.g., cpus, gpus) into their requirements satisfaction projections and plans. That is, the current system SWAP will change dramatically if the development cycle is multiple years, providing additional capacity for new capabilities. However, if all of their time is spent defending the program rather than tracking current technology and expected delivery and adoption

___________________

³⁰ R. Lopez and R. Malur, 2024, “Software Development and Acquisition,” Presentation to the committee, July 19.

³¹ R. Frew and G. Miller, 2024, “Perspective on Software Assurance and Nimbleness from the Defense Industry,” Presentation to the committee, September 27; R. Jandrian and C. Toney, 2024, “United States Navy Perspective on the Navy Software Programs,” Presentation to the committee, November 22.

³² GAO, 2023, “Additional Actions Needed to Help DoD Implement Future Modernization Efforts,” GAO-23-105611; J.M. McQuade, R.M. Murray, G. Louie, M. Medin, J. Pahlka, and T. Stephens, 2019, “Software Is Never Done: Refactoring the Acquisition Code for Competitive Advantage,” Defense Innovation Board, https://media.defense.gov/2019/May/01/2002126690/-1/-1/0/SWAP%20EXECUTIVE%20SUMMARY.PDF; Defense Science Board, 2018, “Design and Acquisition of Software for Defense Systems,” Department of Defense Office of Publication and Safety Review, https://apps.dtic.mil/sti/pdfs/AD1048883.pdf.

³³ D. Gehlhaus, J. Ryseff, and J. Corrigan, 2023, The Race for U.S. Technical Talent: Can the DoD and DIB Compete? Center for Security and Emerging Technology.

curves, combined with the ability to incorporate such expectations into their program baselines, they are trapped in “delivering tomorrow’s technology with today’s tools.”

Contractor-based development teams must likewise combine intimate mission understanding with deep and current technical skills. Cycle times on most platforms and associated toolsets occur frequently. For example, Apple updates MacOS in a major way annually with regular patches applied throughout the year.³⁴ AWS updates its microservices tens of thousands of times per day. To successfully create and maintain such a battle rhythm, state-of-the art capabilities, including release validation and distribution tools, are essential. These must be developed and maintained at the same rate as the capabilities they are designed to support. Furthermore, the currency of toolsets is often a prioritized discriminant in attracting top talent in a highly competitive talent acquisition environment. As the exceptions, Kessel Run and Aegis teams have prioritized current tools and skills in their combined teams as an essential requirement and not a “nice-to-have.”

Recommendation 4-13: The Office of the Under Secretary of Defense for Acquisition and Sustainment should review and consider changing the Department of Defense’s (DoD’s) policies for reimbursement of contractors’ true development costs, such as development infrastructure hardware and software and developer training, to bring the support standards for developers on DoD programs closer to those for commercial off-the-shelf vendors.

Recommendation 4-14: The Office of the Under Secretary of Defense for Acquisition and Sustainment should sponsor the development, adoption, or adaptation of tools to enhance system agility and assurance and integrate and incent their use within the Department of Defense acquisition processes. These tools must evolve with advancements in technology, threats, and scientific complexity. This tooling, by definition, would change over time as technology, threats, and underlying scientific understanding increases.

Recommendation 4-15: In partnership with the Office of the Under Secretary of Defense for Acquisition and Sustainment (OUSD(A&S)), the Software Engineering Institute should pilot and publish how best to anticipate and incorporate commercial standards and practices fluidly into ongoing development and sustainment activities. OUSD(A&S) should incorporate these techniques into guidance for implementing the software acquisition pathway.

___________________

³⁴ Endoflife.date, n.d., “MacOS,” Github (blog), https://endoflife.date/macos, accessed June 11, 2025.

Recommendation 4-16: The Defense Advanced Research Projects Agency should develop advanced tooling to rapidly aid transition to commercial, current software baselines where they exist and companion advanced tooling to simplify timely system certification.

Recommendation 4-17: As is already mandated for suppliers, Department of Defense acquisition leadership teams should ensure that all aspects of complex systems acquisition leadership are represented and equally heard and valued in day-to-day and strategic operations. The Defense Advanced Research Projects Agency, in partnership with the Office of the Under Secretary of Defense for Acquisition and Sustainment (OUSD(A&S)), should develop and pilot tools that can be pragmatically applied to identify strengths and weaknesses in acquisition leadership teams. OUSD(A&S) should mandate “composed” team profiles to increase acquisition success.

Jumping the S-Curve: Moving from Vertical Integration to Platforms

The transition from tight, vertically integrated systems to modular platform architectures remains one of the most challenging operational constraints. Funding for such transitions is often inadequate, resulting in programs that fail to address the dual demands of maintaining legacy systems while developing new, platform-based capabilities. This leads to stagnation and missed opportunities for innovation. DoD must allocate overlapping funding to simultaneously support legacy systems and new platform development. This phased approach allows for incremental modernization, ensuring continuity of operations while building the foundation for more agile and extensible systems. DARPA and the Defense Innovation Unit (DIU) can play pivotal roles in demonstrating how to navigate this transition through pilot programs and experimental platforms that showcase the value of modular architectures.

Finding 4-4: Current DoD MDAPs are largely building tight, vertically integrated systems. Achieving assurance over the life cycle of these systems is undermined by current acquisition practices. Agility of these systems is likely not achievable without migrating to a modular, platform-based approach.

Recommendation 4-18: The Defense Advanced Research Projects Agency (DARPA) Information Innovation Office should partner with the DARPA Tactical Technology Office or another DARPA office focused on weapons system technology to demonstrate the application of the recommendations in this report to a major weapons system prototyping program. The collaborating

offices should jointly prepare a report for other elements of the Department of Defense and the Defense Industrial Base on lessons learned from the effort.

Closing Thoughts

DoD should encourage acquisition and development innovation on a local and experimental basis and actively enlist DARPA experiments to pilot and experiment side by side with these innovators, rewarding those who differ and truly study their successes or failures. This approach is what has happened at AWS with formal methods. Overwhelmingly, most systems at AWS are not developed with formal methods, but some have been, and those successes have motivated further activities. This is the most likely path for DoD success.

DoD and its contractors need strong technical people to carry out advanced work. These people need to be motivated, mission driven, well trained, treated well, and well paid. The committee heard repeatedly in the briefings it received that DoD is losing good people because these requirements are not met. This problem relates to incentives generally, as mentioned above.

Ideally, a competitive market in which the buyers are valuing agility and assurance would produce suppliers that deliver those properties, up to the limit set by the state of art. Unfortunately, the defense acquisition system lacks essential characteristics of competitive markets, such as low barrier to entry and diversity of suppliers and consumers. Accepting that these are fundamental constraints of the defense market and that a wholesale restructuring of the defense acquisition system is well beyond the realm of viability for the purposes of this study, this report focuses on specific proposals to better align incentive structure with desired outcomes.