2022 Assessment of the DEVCOM Army Research Laboratory (2024)
Chapter: 5 Competency Conclusions and Recommendations and Crosscutting Conclusions and Recommendations
5
Competency Conclusions and Recommendations and Crosscutting Conclusions and Recommendations
HUMANS IN COMPLEX SYSTEMS CONCLUSIONS AND RECOMMENDATIONS
Conclusion: A state-of-the-art digital infrastructure is an important component for the success of the humans in complex systems competency. Computing resources and technology platforms will need investment to ensure their digital infrastructure is supported and strengthened. Currently, the Information for Mixed Squads (INFORMS) Laboratory does not have a convenient or uncomplicated means of updating the software in their large bank of computers and servers due to the local computer security restrictions. This problem poses a significant hindrance to the programmers and research scientists working in the laboratory. In particular, the use of industry-standard and cutting-edge technologies requires continual upgrading and software maintenance (e.g., Windows operating system updates), often over the open internet, which is challenging to undertake in the secure network environment that Army Research Laboratory (ARL) requires.
Recommendation: The DEVCOM Army Research Laboratory should prioritize the maintenance of the Information for Mixed Squads (INFORMS) Laboratory and other computing resources and platforms necessary to the work of the humans in complex systems competency. These systems should be adequately resourced with appropriately trained staff, either through new staff hires or reallocation of existing staff. For the INFORMS Laboratory, this would require a combination of information technology management and game development expertise to overcome the challenges faced in maintaining a laboratory of this type in a secure network environment.
Conclusion: The humans in complex systems competency needs to prioritize the incorporation of data scientists to work on statistical analysis of data sets. For the human system team interactions core competency it was noted that some statistical tests used by ARL presenters were not appropriate for the categories of data and the analysis would have benefitted from consultation with a statistician. For the estimating and predicting humans in complex systems core competency, most if not all projects in the core involve collecting, processing, evaluating, and managing vast amounts of data, in some cases in real time. Given this fact, to keep up with industry and leading academic institutions, it is recommended to invest in bolstering the data science expertise within this core competency. More scientists and engineers who work in the field of big data will be needed, including statisticians trained in cutting edge techniques (e.g., nonlinear and Bayesian), computer scientists, and applied mathematicians who can develop and apply leading edge machine learning (ML) algorithms and tools and computing infrastructure that enables rapid, parallel computation. Review of the work in the neuroscience and neurotechnologies core competency noted concerns about the balance of research and experimental design in terms of secondary data analysis versus autonomy over designing experiments. ARL research teams could have more autonomy in terms of designing experiments and data collection. One suggestion could be the creation of a small team, center, or laboratory that is dedicated to providing data science support to the ARL teams for improving reproducibility of experiments. More expertise could also be developed locally by
expanding the educational and training opportunities focused on statistical and data analysis for current staff or by working more closely under the mentorship of extramural partners adept in these areas (or through the mentorship of internal statistical and data analysis experts if they are hired into ARL as part of a team, center, or laboratory). Incorporating experts that have cross-training in data and statistical science and biologic and behavioral science may also be helpful.
Recommendation: The DEVCOM Army Research Laboratory (ARL) should consider recruiting more statistical and data science expertise to support the humans in complex systems competency. The recruitment of more experts in advanced statistical approaches may facilitate the development state of the art machine learning tools and computing infrastructure that enables rapid, parallel computation. The uniqueness of ARL’s research and development efforts may serve as a strong selling point to attract these experts. One idea may be the development of a small team, center, or laboratory dedicated to providing data science and statistical support to the humans in complex systems core competencies. Working more closely under the tutelage of extramural partners who are skilled in these areas or expanding the educational and training opportunities focused on statistical analysis and data analysis for current staff could also help to grow this expertise locally.
Conclusion: For the estimating and predicting humans in complex systems core competency, nearly all of the projects in the core use experimental frameworks that try to recreate real-world scenarios in a controlled laboratory or virtual environment. While this is an important and logical first step, it will eventually stifle progress. It is therefore recommended that this competency rapidly accelerate toward acquiring data and studying behaviors in the field in ecologically relevant contexts. Operationally this means more investment in lean, wearable sensor technology that can be used in real-world settings. Additionally, in order to ensure the ecological validity of the research, ARL needs to consider where research in other core competencies (e.g., neuroscience) may be transitioned to real-world settings and real-world systems earlier.
Recommendation: The humans in complex systems core competency leadership should consider conducting a review of relevant work within the humans in complex systems core competencies to determine which projects may be ready for a transition into real-world settings and real-world systems. Additionally, when appropriate to the project needs, the Army Research Laboratory may consider investing in wearable sensor technology designed for real-world settings.
Conclusion: The antidisciplinary approach for the hybrid human–technology intelligence core competency is promising, and critical for ideating solutions for future and unknown problems. The work does not currently appear to engage with contemporary work in the ethics of artificial intelligence (AI)—an extremely important engagement given the importance to hybrid human–machine thinking of matters of trust in decision-making and the soundness of decision-making. The complexity of this work requires engaging with a multitude of expertise areas and reviewing ethical AI principles.
Recommendation: The humans in complex systems competency leadership should actively reach out to multidisciplinary, transdisciplinary, and antidisciplinary research communities and disciplines relevant to hybrid human–machine interaction, including game design, game development, computational creativity, and engage in contemporary work in ethical artificial intelligence.
Conclusion: The humans in complex systems competency would be well served by improving staff resources in game development, especially for art assets development. High-fidelity models are important for user experience in game-based research in order to create a naturalistic environment; many projects
had high-quality, high-fidelity art assets (e.g., the INFORMS Laboratory demonstrations), while others seemed not to have access to skilled artists and designers.
Recommendation: The DEVCOM Army Research Laboratory should consider improving staff resources in game development and, in particular, art assets development, which includes access to skilled artists and designers.
Conclusion: A makerspace could be designed to support the antidisciplinary, experimental, and speculative design research that the hybrid human–technology intelligence core competency team aims to undertake. Such a makerspace could have technologies such as three-dimensional (3D) printing, laser cutting, and textile prototyping. Human decision-making is inherently tangible and embodied; to support this aspect of the research, access to tools that can help the research team prototype new interfaces and technologies will be key to its success. For the human-guided system adaptation core competency, a makerspace with manufacturing expertise in embedded systems and hardware, that is more streamlined to use would encourage closing the hardware loop earlier. A makerspace could facilitate open sharing of resources across core competency areas and encourage more intramural and extramural research.
Recommendation: The DEVCOM Army Research Laboratory should consider the development of a makerspace with technologies such as three-dimensional printing, laser cutting, textile prototyping, and manufacturing experts in embedded systems and hardware.
TERMINAL EFFECTS CONCLUSIONS AND RECOMMENDATIONS
Conclusion: Junior scientists and engineers at the DEVCOM ARL would benefit from a formal mentoring program, especially given the unique science and engineering crucial to supporting army mission needs, including many that are classified, and as these skills are not taught within university science and technology (S&T) curriculum.
Recommendation: The DEVCOM Army Research Laboratory (ARL) should implement mentoring perhaps by taking advantage of established intramural and extramural investigators, who can act as mentors. Several successful mentoring models exist in academia that may be parlayed to ARL uses.
Conclusion: The impact of COVID-19, which increased the amount of remote work in the terminal effects competency drastically reduced human-to-human interactions onsite at ARL. Such extensive teleworking, as is ongoing at ARL, is a strong impediment of knowledge transfer, productivity, and mentoring. Given the unique and often classified nature of ARL’s science and engineering terminal effects programs, direct staff-to-staff interactions and contact time onsite is critical to knowledge transfer and mentoring in the S&T underpinning terminal effects—especially in light of looming retirements of senior staff within the terminal effects branch.
Recommendation: DEVCOM Army Research Laboratory (ARL) management should assess the balance of onsite versus teleworking to ARL’s long-term scientific talent development, knowledge transmission, and staff retention in support of Army classified mission objectives. Establishment of a face-to-face, onsite, dedicated mentoring and knowledge retention program appears to be especially crucial for ARL to pursue for the classified focus areas within the terminal effects competency that cannot be conducted remotely.
WEAPONS SCIENCES CONCLUSIONS AND RECOMMENDATIONS
Conclusion: There are opportunities within the aerodynamics and control core competency to include both scientific ML and data science in computational and experimental research. Computational model reduction techniques can potentially speed up the design process. Data-driven modeling with ML has increased the insight from experimental data.
Recommendation: The DEVCOM Army Research Laboratory should consider integrating machine learning and data science into their computational and experimental research. Computational model reduction techniques can potentially speed up the design process. Data-driven modeling has magnified the insight from experimental data. The latest developments in reduced order modeling could also be considered.
Conclusion: For the discovery, synthesis, and formulation of energetic materials core competency, a database capturing available molecules, prior successful formulation pathways, and resulting performance, sensitivity, and stability properties would serve as a tool for new researchers to capture the experience of senior researchers so that it is not lost and allow any researcher to work beyond their own experience. This database could serve as the starting point for application of ML techniques to the synthesis and formulation process, with the end goal to be able to develop a capability (ML or otherwise) that would suggest molecules and formulation paths based on desired parameters.
Recommendation: The DEVCOM Army Research Laboratory should consider developing a database that captures what molecules are available, what prior formulation pathways have been performed with such molecules and their result; and the resulting performance, sensitivity, and stability properties of each attempt.
Conclusion: The modeling, simulation, and experimental characterization of energetics competency relies heavily on the concept of bridging scales. Measurements and calculations at smaller scales are used to inform subscale models or effects in larger scales. Any errors at the smallest scales may thus propagate though the increasing cascade of scales and influence major findings at system level calculations. It may thus prove valuable to both estimate uncertainty and, where possible, validate results at each modeling and experiential scale.
Recommendation: The DEVCOM Army Research Laboratory should consider defining rigorous uncertainty quantification methodologies when using scaled modeling approaches and linking these efforts to experimental validation measurements at each scale.
Conclusions: Further work is needed for the study of vortex–shock interactions, unsteadiness effects, and shock and boundary layer interactions as well as effects associated with asymmetric platforms. For many applications, these unsteady phenomena may be more important than transition. Hypersonic flow studies at higher Mach numbers with associated coupling of new physics are recommended and ARL has opportunity for leadership here.
More thorough examination of optimal choice for unmanned aerial vehicle (UAV) power and propulsion systems is suggested with attention to the strong influence of the desired endurance, range, and thrust. For short endurance, the engine or power weight becomes a handicap; while at longer endurance, the weight of fuel dominates and efficiency becomes more important in the optimization process. Concerns were also expressed about several challenges for the tethered UAV studies and more attention needs to be given to potential problems with snags and tangles in the UAV tethers. Optical fibers might be considered in the UAV tether design because of lightweight, high flexibility, and good power and data transmissions.
Additionally, more ARL attention needs to be given to research on rotorcrafts at NASA’s Ames Research Center and working connections need to be formed. Furthermore, large eddy simulation using high-order numerical methods, as exemplified by the work of Z.J. Wang (University of Kansas) could be incorporated for rotor dynamics.
Additionally, the connection between the core competency team and the ARO researchers could be better strengthened, and connections with other researchers, including some within the Department of Defense (DoD), can also be given more attention.
Recommendation: The DEVCOM Army Research Laboratory should consider focusing on true enthalpy in addition to cold flow environments and on pushing analysis to higher-altitude and Mach number conditions.
Recommendation: The DEVCOM Army Research Laboratory should examine the potential problems with snags and tangles in the unmanned aerial vehicle (UAV) tethers. Optical fibers might be considered in the UAV tether design because of their lightweight, high flexibility, and good power and data transmission capabilities.
Recommendation: The DEVCOM Army Research Laboratory should consider stronger connections with the researchers studying rotorcraft at NASA’s Ames Research Center and associated computational model development at Ames Research Center and Stanford University.
Conclusion: In the guidance and navigation of weapons systems core competency the support for extramural research is leading to high-quality innovations specifically in the areas of data-driven autonomy, robust atomic clocks, and optical time transfer. ARL’s internal technical strength can be further enhanced through its extramural collaborations.
Recommendation: DEVCOM Army Research Laboratory (ARL) should build on the well-established extramural collaborations to further grow the technical strength of its internal teams. This could be done through a variety of mechanisms including integration of externally developed techniques and software, visiting appointments for extramural researchers, technical exchange programs for current DEVCOM ARL staff, and future staff recruitment.
CROSSCUTTING CONCLUSIONS AND RECOMMENDATIONS
The following crosscutting conclusions and recommendations are based on analysis of the projects and programs presented within the humans in complex systems, weapons sciences, and terminal effects competencies. They were developed through the identification of similar themes that emerged across these competencies. The purpose of this chapter is to provide ARL with a broader picture of important opportunities that have been identified across their laboratory.
Conclusion: ARL staff exhibited professional competence and an awareness of what is being done in their respective research fields. Nevertheless, ARL would benefit broadly from formulating and incentivizing formal staff development, early-career mentoring, and ensuring continual knowledge transfer between generations of staff. This may include enhanced and more explicit research collaborations with scientists and engineers from academia and other parts of government.
Crosscutting Recommendation: DEVCOM Army Research Laboratory should consider mechanisms and incentives for lifelong personnel development that will enhance
recruitment, advancement, and retention of staff. Explicit emphasis on early-career development and approaches for knowledge capture and transfer should also be prioritized.
Conclusion: The DEVCOM ARL has a tremendous amount of valuable legacy data (both open and classified) of unique and irreplaceable nature to DoD. Generating new data can be expensive and time consuming, which calls for planning, archiving, and curating data beyond just a data-driven ML or AI approach. Documentation of the information required to duplicate all experimental and computational programs or results needs to be archived and curated.
While there exists no single database that can serve as the model for the Army’s efforts (especially considering the security challenges), there are some good examples to start with—if ARL is not currently using them, such as the Materials Research Society Artificial Intelligence Staging Taskforce meetings. This voluntary group (academia, industry, and government) is exploring how the materials community uses, stores, and shares data to better serve the emerging needs of materials researchers. ARL should consider actively joining and participating in this group.
Crosscutting Recommendation: The DEVCOM Army Research Laboratory (ARL) should establish a formally funded ongoing commitment to archiving and curating both new and legacy data (i.e., experimental, computational, and statistical), which is crucial to supporting current and future design and performance capabilities supporting the breath of Army programs. ARL should develop incentives to facilitate researchers to constructively archive their data and access this database to both capture knowledge and minimize duplication of prior investigations.
Conclusion: Availability of large data sets, new computational tools for statistical analysis and data analytics, and the potential afforded by emerging approaches in ML and AI have great potential for accelerating the pace of understanding for technology development in the competency areas reviewed in this study. Failure to incorporate these techniques presents a significant risk to maintaining international leadership. While previous reviews have identified these areas for increased attention, the urgency continues to accelerate as more successful applications emerge.
Crosscutting Recommendation: DEVCOM Army Research Laboratory should ensure that each competency contains sufficient data science, statistics, and machine learning expertise to take advantage of experimental and simulation data, and to help develop new models for design and development.
