Rigorous operational testing (OT) of weapon systems procured by the U.S. Department of Defense (DoD) is fundamental to ensuring that these sophisticated systems not only meet their stated requirements, but also perform under realistic operational conditions when faced by determined adversaries employing their own highly capable offensive and defensive weaponry. Without adequate OT, operational commanders would be unable to make the most effective use of their capability and warfighters would lack confidence in the weapons they bring to the fight or, worse, may inadvertently put themselves in harm’s way because they do not have a fundamental understanding of their weapons’ capabilities and limitations. DoD’s test and training range enterprise provides the geography, infrastructure, technology, expertise, processes, and management that make safe, secure, and comprehensive OT possible. However, the range enterprise, along with the talented and committed range workforce that makes the system function, is under great stress. Unless prompt action is taken to address both longstanding and emerging challenges, including test capacity, modernization, digital infrastructure, encroachment, and resources, DoD’s ranges will be unable to support timely or adequate OT in the future.
The challenges facing the nation’s range infrastructure are both increasing and accelerating. Limited test capacity in physical resources and workforce, the age of test infrastructure, the capability to test advanced technologies, and encroachment impact the ability to inform system performance, integrated system performance, and the overall pace of testing. Investments in the U.S. test infrastructure and changes in test
and evaluation (T&E) methodologies and handling of data are necessary to inform the delivery of lethal, survivable, reliable, and affordable weapon systems to the field at a speed that is relevant to the operational need. This study draws on testimony from senior military officers and officials from operational, acquisition, and test backgrounds as well as on test and training experts, leading technologists, leaders from relevant commercial enterprises, and individuals with deep experience in DoD and congressional budget processes. The study committee conducted virtual and physical site visits to a representative sample of test ranges; collected test range inputs on modernization, sustainment, operations, and resource challenges; and reviewed prior studies and reports from the office of the Director of Operational Test and Evaluation (DOT&E), the military service test organizations, and the Test Resource Management Center (TRMC). This report makes a set of interdependent recommendations that the committee believes will put the DoD range enterprise on a modernization trajectory to meet the needs of OT in the years ahead.
The report emphasizes the following three fundamental themes:
deploying new generations of weapons designed to negate U.S. warfighting advantages. At the same time, new weapon systems are employing never-fielded technologies, which are also evolving at a pace enabled by Moore’s Law. Usable weapon systems are fielded promptly, but there is a need for continuous testing and assessment.
To address the challenges tied to these themes, the committee developed conclusions and recommendations that fall into the following five broad categories:
Make model-based engineering, the unbroken digital thread, and continuous integration/continuous delivery software practices the foundation for range agility, rapid test evolution, and speed-to-field. Ultra-high-bandwidth information flows must become frictionless, on-demand, and secure. [Recommendations 4-2, 4-3]
Table ES.1 maps the report recommendations and key conclusions to the themes laid out above.
TABLE ES.1 Report Recommendations and Key Conclusions Mapped to Themes
| Theme | Recommendations and Key Conclusions |
|---|---|
| Develop the “range of the future” to test complete kill chains in JADO environments | Conclusion 3-1: The lack of a Department of Defense or joint publication set of definitions for multi-domain operations and cyber-physical systems can result in different operational use cases. Conclusion 3-2: Testing ranges are not optimized for testing end-to-end kill chains; they were not designed for collaborations with other ranges, and they lack the framework and infrastructure to test concurrent and connected kill chains. Recommendation 3-1: To enable a range of the future that is capable of testing kill chains and multi-domain operations that can integrate effects across National Defense Strategy modernization areas, the Secretary of Defense should address the need to enable Department of Defense ranges to provide regular venues to “test as we fight” for acquisition and prototyping programs in a joint multi-domain battlespace of integrated systems. |
| Theme | Recommendations and Key Conclusions |
|---|---|
| Restructure the range capability requirements process for continuous modernization and sustainment | Conclusion 3-3: Encroachment leads to the inability to demonstrate mission capability and identify deficiencies due to lack of access to the physical and electromagnetic spectrum space with which to conduct test and evaluation. This creates operational risk as DoD will have to field weapon systems that have not been tested against certain threats. Recommendation 3-2: To ensure the ability to validate the survivability of Department of Defense (DoD) weapon systems against a realistic operational threat environment across air, sea, land, space, and cyberspace domains, DoD should identify and prioritize bands that cover U.S. military operational and test requirements which should be protected from sell-off to preserve these capabilities. Recommendation 3-3: The Test Resource Management Center (TRMC) should assess current and projected commercial radio frequency communications technologies and spectrum allocations for secure, agile, high-bandwidth operational test needs. In addition, TRMC should determine the feasibility of developing new large-scale enclosed testing facilities combined with expanded modeling and simulation to support electromagnetic spectrum activities not suitable for open-air testing. Recommendation 3-4: The Department of Defense should broaden the authority of the Test Resource Management Center to address issues of internal encroachment by reviewing internal range policies and actions to ensure that the test groups retain adequate mission space and prevent the placement of equipment or infrastructure that could potentially interfere with test operations. The Director of Defense Research and Engineering for Advanced Capabilities should be granted the authority to mitigate disputes arising over internal encroachment concerns and provided additional funding to manage internal encroachment. Recommendation 3-5: The Test Resource Management Center should develop a strategy that assesses the use of and potential investment in suitable allied resources for open-air testing. This strategy should include criteria for the usage of allied resources and areas of potential investment to include range space available, data collection, security risks, and support facilities. |
| Theme | Recommendations and Key Conclusions |
|---|---|
| Bootstrap a new range operating system for ubiquitous M&S throughout the weapon system development and test life cycle | Recommendation 4-1: A Department of Defense joint program office should establish a shared, accessible, and secure modeling and simulation (M&S) and data ecosystem to drive development and testing across the life cycles of multiple supporting programs. M&S should be planned from early concept development to support the entire life cycle of the system, from requirements generation, through design development, integration and test, and sustainment. Uncertainty quantification should be employed to identify the primary sources of uncertainty in the understanding of the system being developed and to define an integrated testing and simulation activity to reduce those uncertainties to an acceptable level. |
| Create the “TestDevOps” digital infrastructure for future operational test and seamless range enterprise interoperability | Recommendation 4-2: A Department of Defense joint program office should adopt and promulgate modern approaches for standardization, architectural design, and security efforts to address data interoperability, sharing, and transmission challenges posed by the complexity of next-generation systems. The joint mission office should determine how to develop and maintain a protected data analysis tool and model repository for testing, increase the interconnectivity of test ranges, and ensure the development of data protocols for the real-time transfer of data at multiple classification levels. Recommendation 4-3: The Test Resource Management Center should continue monitoring and supporting the Assured Development and Operation of Autonomous Systems (ADAS) Project, and prioritize efforts to develop a common set of standards, measurement approaches, and operational scenarios from which to evaluate the performance of artificial intelligence (AI) and autonomous systems, while recognizing that testing approaches may differ between AI and autonomous systems. |
| Theme | Recommendations and Key Conclusions |
|---|---|
| Reinvent the range enterprise funding model for responsiveness, effectiveness, and flexibility | Recommendation 5-1: The Joint Requirements Oversight Council (JROC) should consult regularly with the Director of Operational Test and Evaluation (who is an advisor to the JROC) about the test requirements for systems considered by the JROC. This consultation should include an evaluation of current testing capabilities, facilities shortfalls, and plans to address these shortfalls. Recommendation 5-2: The Office of the Secretary of Defense should either allow an exemption or set shallower expenditure benchmarks for the first 2 years of test modernization programs. This will reflect realistic expense curves for the technologies and projects needed to test next-generation programs and complex integration. Conclusion 5-1: New mechanisms and funding limits for applying minor military construction are necessary for responsive test and evaluation activities. Conclusion 5-2: There exists a need for the Department of Defense to pilot new process and authorities for funding ranges and infrastructure to make them simpler, more responsive, and more effective. |