Summary

Almost every system used by the Department of Defense (DoD) depends on software. This dependence is not a new development, but the criticality of software has grown markedly over the past few decades. DoD has historically found it challenging to acquire or develop software that is on schedule and meets mission requirements, whose security and functionality are assured, and that can be easily and rapidly adapted to new requirements without discarding those assurances.

The Defense Advanced Research Projects Agency (DARPA) asked the National Academies of Sciences, Engineering, and Medicine to conduct a study of ways to enhance the assurance and nimbleness of large-scale, integrated software-based systems. The statement of task for the study is shown in Appendix A. The National Academies assembled a study committee drawn from academia, the research and defense communities, and commercial software vendors.

The task facing the Committee on Enhancing the Assurance and Nimbleness of Large-Scale Integrated Software-Based Systems does not represent a new challenge. Studies of ways to improve the software that DoD uses go back at least 30 years.¹ Historically, DoD has also sought input on the software acquisition programs.² Program offices have been constrained by rigid processes, and contractors have been constrained in ways that are sometimes inconsistent with the production of useful, adaptable, or performant software.

___________________

¹ Appendix C provides a table summarizing 37 studies and their results, reprinted from Defense Innovation Board, 2019, “Software Is Never Done: Refactoring the Acquisition Code for Competitive Advantage,” May 3, https://media.defense.gov/2019/Apr/30/2002124828/-1/-1/0/softwareisneverdone_refactoringtheacquisitioncodeforcompetitiveadvantage_final.swap.report.pdf.

² Government Accountability Office (GAO), 2023, “Defense Software Acquisitions: Changes to Requirements, Oversight, and Tools Needed for Weapons Programs,” GAO-23-105867, July 20, https://www.gao.gov/assets/gao-23-105867.pdf.

For this study, the committee focused on the ways that DoD is acquiring software, including some promising new acquisition approaches, and on commercial trends and recent research results in software assurance. The committee recognized that, in many ways, commercial off-the-shelf (COTS) software vendors might do a better job than government programs at delivering software with acceptable assurance and might be better able to adapt that software to changing requirements. The committee sought to identify lessons learned that DoD might apply.

This report is organized around three topics derived from the statement of task:

• Agility, the ability of software development organizations to create software that is on schedule, fully functional, and adaptable to changing requirements including new threats.³
• Assurance, the confidence that software will function as intended, and do so securely in the face of malicious attacks. The statement of task and this report focus primarily on security assurance.
• Incentives, the factors that motivate and enable organizations to deliver software with agility and assurance.

Additionally, machine learning (ML) and artificial intelligence (AI) are rapidly maturing technologies that have great potential to enable more effective classes of software (e.g., computer vision) and to enable more agile and better assured development. Dually, new threats that leverage and exploit AI will undoubtedly emerge and require new approaches for assurance. Given the growing importance and rapid changes around AI, this report includes a discussion highlighting key issues as understood today, but a thorough understanding could easily be the subject of a separate report.

Through its work, the committee identified crosscutting findings that underpin its recommendations across the three key topics of this report.

Finding 1: DoD established a “software pathway” for acquisition intended to streamline and expedite development and delivery of software capabilities through iterative development, agile principles, and streamlined requirements.⁴ The Secretary of Defense’s memorandum mandating the software pathway⁵ highlights the criticality of enhancing

___________________

³ Although the statement of task for this study (see Appendix A) uses the term “nimbleness” to be consistent with software community practice, this report refers to agility rather than nimbleness.

⁴ Department of Defense (DoD), 2020, “Operation of the Software Acquisition Pathway,” DoD Instruction 5000.87, October 2, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500087p.PDF.

⁵ P. Hegseth, 2025, “Directing Modern Software Acquisition to Maximize Lethality,” https://media.defense.gov/2025/Mar/07/2003662943/-1/-1/1/directing-modern-software-acquisition-to-maximize-lethality.pdf.

software assurance and agility—DoD cannot afford to fail in this endeavor. Successful adoption of software pathway will require the DoD to make changes to its programs and practices. By adapting proven industry practices for achieving agility and combining industry practices with the results of DARPA-sponsored research in assurance, DoD has an opportunity to make radical improvements in software that is critical to the warfighter.

Finding 2: The experience of DoD programs that have adopted agile development practices based on industry models, such as the Air Force Kessel Run, justifies confidence that wide adoption of the software pathway will lead to significant improvements in the agility of DoD software systems. Adherence to the software pathway will require flexibility in budgeting and contracting well beyond traditional DoD acquisition practice. Flexible contracting mechanisms, and in particular the Other Transaction Authority (OTA), can be critical to enabling agility in DoD software programs. The use and benefits of OTA are discussed in Chapter 4, “Incentives,” in the subsection “Contracts and Acquisitions.”

Finding 3: Software agility and assurance are complementary, and many practices contribute to both. Both rely on a well-designed architecture that supports change and evolution. Both require that one integrate and move testing and assessment as far “left” in the development cycle as possible to catch errors and flaws at the point where it is easiest to fix them. And both are most effective when the development pipeline leverages automation.

Finding 4: Formal verification (also known as formal methods—the application of rigorous mathematical techniques to verify the correctness of software) has matured to the point where it is not only viable for (at least) critical software components to provide the highest levels of assurance, but also that commercial software developers have found that when done well, formal verification can make software development more agile.

Finding 5: DoD systems often face challenges well beyond what commercial providers must address. Assurance needs are much more stringent, not only because of the life- and-death context, or the real-time constraints of flight controls and weapons systems, but because adversaries will actively seek to break and disrupt these systems. Unlike COTS software, which is exposed to hostile attack from the Internet on a daily basis, DoD systems may operate for years before being deployed in combat and exposed to adversary attack. Consequently, DoD software may continue to have “shallow” bugs that, in the absence of better development practices, are only uncovered in the heat of battle.

The rest of this summary provides a brief overview of the committee’s observations and key recommendations, organized around agility, assurance, and incentives.

AGILITY

Global conflicts, such as the wars in Ukraine and Gaza, have provided a striking demonstration of the importance of agility in the weapons and information systems that support modern warfare. Rapid adaptation of the software underlying weapons systems—such as drones; command, control, and intelligence systems; and logistics systems—have proven critical to battlefield success. Today, such systems rely heavily on software. A critical facet of agility is that the end user or organization, rather than the development organization, determines whether the software is sufficiently agile, satisfactory, and fit for purpose.

System agility depends on software agility and agile development practices, which are characterized by

• User engagement—the end user and requirements organizations are continuously involved and empowered by the development organization in determining whether the development project is succeeding and whether software changes are required to meet evolving requirements or accommodate technical challenges.
• Software architecture—a successful software architecture enables agility by clearly defining interfaces, bounding the scope of changes, enabling the reuse of software components in multiple programs, and enabling multiple development organizations to contribute software components that plug into the system.
• Continuous integration and testing—software developers and development teams are most successful when they receive almost immediate feedback (i.e., in minutes or hours) on the functioning and assurance of newly developed software components.

A useful mechanism for agile development is digital model-based engineering (MBE),⁶ in which digital models are constructed of hardware and software components, allowing for early design iteration, testing, and verification. Paired with a test harness,⁷

___________________

⁶ NASA, 2017, “Digital Model-Based Engineering: Expectations, Prerequisites, and Challenges of Infusion,” NASA/TM-2017-219633, https://ntrs.nasa.gov/api/citations/20170006995/downloads/20170006995.pdf.

⁷ Wikipedia, n.d., “Test Harness,” https://en.wikipedia.org/wiki/Test_harness, accessed June 11, 2025.

MBE is extremely promising for supporting both agility and assurance but demands significant investment to develop and maintain models, particularly for rich cyberphysical systems.

The software pathway appears to be a significant step toward the achievement of agility. It is not clear how widely programs are following the pathway at the time of this report (particularly for weapons systems⁸), but there are encouraging signs both from programs that are formally following the pathway and programs that have implemented similar processes. A March 2025 Secretary of Defense memorandum⁹ directs essentially universal adoption of the software pathway.

Security measures introduced to protect classified or sensitive program information can be an obstacle to agility. To the extent that programs are compartmentalized, and that security precludes developers from understanding requirements or from testing with real-world (or realistic) data, agility will be more difficult to achieve, and the benefits of the pathway are likely to be diminished.

Key Recommendations

Recommendation 2-2: The Office of the Under Secretary of Defense for Research and Engineering, the Defense Advanced Research Projects Agency, and the Defense Acquisition University should collaborate to develop policy, guidance, training, and organizational and technical frameworks in support of agility, architectures, and adoption of the software pathway.

Recommendation 2-4: The Office of the Under Secretary of Defense for Acquisition and Sustainment and the Services should initiate the development of common software repositories with required maintenance that can host software components across programs and enable programs to reuse components that they did not develop. Both Department of Defense (DoD) contractors and DoD personnel should be incentivized with appropriate rewards for contributions to the repositories.

ASSURANCE

Assurance of software functionality has traditionally resulted from system testing. In a world of agile development, early exposure to users and real-world conditions provides

___________________

⁸ GAO, 2023, “Defense Software Acquisitions.”

⁹ P. Hegseth, 2025, “Directing Modern Software Acquisition to Maximize Lethality.”

higher and more rapid confidence in functionality. Assurance of software security is a more challenging problem—neither testing nor assessments of process or organizational maturity are sufficient. Ideally, software is accompanied with an “assurance case” in which the development process creates specified kinds of evidence, including evidence produced by successful tests, by the application of tools (including, for example, the use of a memory safe language or a framework that rules out classes of errors), and by the execution of verification processes. That evidence is updated as the software is changed and is reviewed automatically and frequently to provide users and security personnel with confidence that software is sufficiently assured.

COTS vendors integrate security into development, routinely applying practices that align with the National Institute of Standards and Technology Secure Software Development Framework (SSDF) and relying on integrated teams to identify and test critical components. Although vulnerabilities remain, and COTS vendors are far from achieving perfect security, vendors who apply these practices have achieved significant progress in meeting customers’ demands for improved assurance. DoD programs have adopted static analysis and continuous authorization to operate (ATO) to enhance the security of software development, but it is not clear that their maturity is comparable to the practices of the best COTS vendors. There does not appear to be any requirement that DoD programs follow the SSDF.

Beyond the practices explicitly identified in the SSDF, Amazon, Google, Microsoft, and other software producers have substantially moved away from developing new software in C and C++, preferring instead to use mainstream memory-safe languages, which make it easier to construct secure systems from individual components, and for which there is far less risk from a large class of devastating, memory-related software vulnerabilities.

Going further, these companies have also applied formal methods with great success to critical components that have been deployed. These successes were built on technology developed by numerous government-sponsored research programs (DARPA, the National Science Foundation, and the Office of Naval Research, among others) in formal methods, but DoD has not leveraged this technology as much as it could.

Although DARPA programs have demonstrated significant achievements in applying formal verification, the application of formal methods to DoD systems was not visible to the committee. The committee’s perception is that although these methods may be used in some limited contexts (i.e., classified settings), there are many more settings where they could be profitably applied.

Key Recommendations

Recommendation 3-1: The Office of the Under Secretary of Defense for Acquisition and Sustainment and the Office of the Under Secretary of Defense for Intelligence and Security should update Department of Defense Instruction 5000.87 to clarify that software security assurance should be based on adherence to the National Institute of Standards and Technology Secure Software Development Framework (SSDF) and should update the requirements for authorization to operate to remove any compliance mandates that would conflict with or duplicate the requirements of the SSDF.

Recommendation 3-2: The Defense Advanced Research Projects Agency should sponsor the research, development, and prototyping of a process for creating and maintaining assurance cases and should use its emerging programs aimed at integrating formal methods into Department of Defense programs as pilots for application of the assurance case concept.

Recommendation 3-3: The Office of the Under Secretary of Defense for Acquisition and Sustainment should issue guidance that strongly encourages software research and development programs to adopt type-safe, memory-safe programming languages (e.g., Rust, Go, SPARK) or environments (C#, Java) rather than unsafe languages (e.g., C, C++). They should (among other things) leverage these languages’ ability to enforce component separation, with clear interfaces, to enable higher assurance.

Recommendation 3-4: The Department of Defense should strongly incentivize developers to use formal methods to build high-criticality software components. The Defense Advanced Research Projects Agency (DARPA) should collaborate with the Office of the Under Secretary of Defense for Acquisition and Sustainment to identify a small number of software programs that might benefit from enhanced assurance and should deploy a cadre of DARPA-funded “formal methods engineers” to each of those programs with the mission of identifying components where the application of formal methods would benefit system assurance. The formal methods engineers should take direct responsibility for developing assured components and, if their performance and assurance are acceptable, integrating them into the target systems in collaboration with the systems’ development contractors.

INCENTIVES

Financial and policy incentives have an enormous impact on the ways that DoD program offices and software development contractors set priorities, organize their development processes, and deal with changes and challenges. The differences between the software development practices of COTS vendors and DoD programs are striking and are largely a result of differences in incentives and the organizations’ reactions to them.

A key difference between COTS and DoD programs is the time required to receive feedback. COTS vendors get immediate feedback on success or failure in meeting user requirements and achieving assurance. Failure to meet requirements can challenge a vendor’s existence—the paradigm is literally “succeed or die.” In contrast, user feedback on DoD programs is often long delayed, and DoD systems may operate for years without being operationally challenged, so success or failure is often assessed based on artificial or unrealistic measures.

Commercial vendors are faced with constant competition, and there is relatively little friction to impede customers from switching suppliers. Commercial product architectures enable multiple vendors to contribute to the success of an underlying system (referred to as a platform) but also facilitate customers’ decisions to switch suppliers if they are not satisfied with products’ functionality and assurance. The Apple app store is an example of an extensible platform that enables multiple development organizations to meet customer requirements for functionality, agility, and assurance, and customers can choose among multiple vendors for most applications (e.g., email, web browsing). Similarly, the Amazon Web Services (AWS) microservices architecture is an example of well-defined interfaces implemented by easily replaceable components to support more agile development of functionality and assurance. In contrast, DoD development contractors have historically been penalized for or forbidden from proposing designs that respond to potential future requirements that are not yet part of the program of record. The result is often short-term cost savings at the expense of delays or failure of longer-term modification programs.

COTS vendors compete for and prioritize talented software developers. They prioritize the provision of state-of-the-art development systems and software tools to make their developers productive. They encourage sharing of ideas, feedback, and code to maximize productivity. In contrast, contractors on DoD software programs are often constrained by tight budgets, contractual constraints, and security regulations that limit developers’ productivity and ability to work efficiently and effectively. Talented

developers are often more motivated by the ability to work effectively on interesting problems than by financial compensation, and while DoD programs offer exciting technical challenges, working conditions may prove to be an obstacle to attracting talented developers.

Key Recommendations

Recommendation 4-1: The Office of the Under Secretary of Defense for Acquisition and Sustainment should modify Department of Defense Instruction 5000.02 to require and incent that software-intensive systems be based on extensible platforms and architectures that support iterative extension of capabilities by a broad set of suppliers (beyond the original platform vendor) with clear delineation of long-term maintenance and currency responsibilities.

Recommendation 4-7: The Office of the Under Secretary of Defense for Acquisition and Sustainment should modify the software acquisition pathway to require that contracts for software stipulate operational support periods for the expected lifetime of the system, including service-level agreements for remediation of discovered vulnerabilities.

Recommendation 4-8: Using the Intelligence Community’s Experimental Research Demonstration as one input, the Office of the Under Secretary of Defense for Acquisition and Sustainment and the Defense Acquisition University should develop and pilot criteria that explore and incent experimental revolutionary innovation requirements for platforms, as opposed to current Department of Defense Instruction 5000–based evolutionary innovation.

Recommendation 4-10: The Office of the Under Secretary of Defense for Acquisition and Sustainment and the Office of the Under Secretary of Defense for Intelligence and Security should jointly review and revise the Department of Defense’s security and clearance policies to minimize impediments to agile software development and testing.

Recommendation 4-13: The Office of the Under Secretary of Defense for Acquisition and Sustainment should review and consider changing the Department of Defense’s (DoD’s) policies for reimbursement of contractors’ true development costs, such as development infrastructure hardware and

software and developer training, to bring the support standards for developers on DoD programs closer to those for commercial off-the-shelf vendors.

Recommendation 4-18: The Defense Advanced Research Projects Agency (DARPA) Information Innovation Office should partner with the DARPA Tactical Technology Office or another DARPA office focused on weapons system technology to demonstrate the application of the recommendations in this report to a major weapons system prototyping program. The collaborating offices should jointly prepare a report for other elements of the Department of Defense and the Defense Industrial Base on lessons learned from the effort.

In summary, enhancing DoD software for a contested future requires concurrent progress in agility, assurance, and incentives. The committee’s recommendations call for cultural and procedural shifts that enable rapid, iterative software updates (agility), rigorously verified and secure software (assurance), and alignment of institutional incentives to support both.