1

Introduction

At the request of the U.S. Army, on August 7-9, 2018, the National Academies of Sciences, Engineering, and Medicine’s Panel on Review of Extramural Basic Research at the Army Research Laboratory (ARL) met to review the programs of the Information Sciences Directorate (ISD) of the Army Research Office (ARO), which is an organizational unit within the ARL. The meeting was held at the ARO headquarters in Durham, North Carolina.

The panel’s review was guided by the following statement of task provided by the National Academies of Sciences, Engineering, and Medicine:

The current report summarizes the 2018 findings of the Panel on Review of Extramural Basic Research at the Army Research Laboratory, which reviewed the programs at the ARO’s ISD. Over the 2019-2020 period the National Academies is scheduled to conduct reviews of the ARO’s Physical Sciences Directorate’s programs in physics, chemistry, and life sciences and its Engineering Sciences Directorate’s programs in mechanical sciences, electronics, materials science, and earth sciences.

PROGRAMS WITHIN THE INFORMATION SCIENCES DIRECTORATE

The Army Research Laboratory’s ARO describes its mission as

Research programs in the ISD are focused on discovering, understanding, and exploiting the mathematical, computational, and algorithmic foundations that are expected to create revolutionary capabilities for the future Army. Discoveries in this area are expected to lead to capabilities in materials, the information domain, and soldier performance augmentation, well beyond the limits facing today’s Army.² The ISD’s programs are organized by three divisions: Computing Science, Network Science, and Mathematical Sciences. Across the three divisions, ISD currently funds 580 projects with a budget of $108 million—$22.3 million core funding and $85.7 million leveraged funding from sources that include the Multidisciplinary University Research Initiatives program, the Presidential Early Career Awards for Scientists and Engineers, the Defense University Research Instrumentation Program, the Minority Institutions Program, the Small Business Innovation Research program, the Small Business Technology Transfer program, and other Army and Department of Defense (DoD) sources.

APPROACH TO THE ASSESSMENT

The panel consisted of 18 leading scientists and engineers whose expertise matched the programs at the ARO’s ISD that were reviewed. All panel members were volunteers who participated without compensation. The entire panel attended overview presentations by and held discussions with the directors of the ARL, the ARO, and the ISD. The panel members then divided into three teams of 6 members each; the teams separately attended presentations by and discussions with the managers of selected programs in the three ISD divisions (Computing Science, Network Science, and Mathematical Sciences). The presentations and discussions occurred over a 2-day period. On the third day of the meeting, the panel assembled to share findings from the team reviews, develop impressions common across the team reviews, and form an outline for the panel’s report draft. On the afternoon of the third day, the panel met with ARO staff to seek clarification of factual and contextual understandings.

After the meeting, the panel members prepared written summaries of their findings, conclusions, and recommendations, which were iteratively reviewed by the panel and formed the basis for the draft report that was subsequently developed under the guidance of the National Academies’ Army Research Laboratory Technical Assessment Board (ARLTAB), which focused particularly on the panel’s approach to the review and the report’s conclusions and recommendations. The ARLTAB consists of the chairs of the panels that review the scientific and technical work of all ARL directorates, including those at the ARO.

Once the panel addressed the comments offered by the ARLTAB, the report was submitted to the National Academies’ Report Review Committee (RRC). The RRC appointed a team of reviewers to examine the report, considering such factors as the scope of the panel’s task, the reasonableness of the panel’s conclusions and recommendations, and the clarity of the panel’s expression. Once the RRC reviewers’ comments were adequately addressed, the report was released for delivery to the Army and for public posting on the National Academies Press website (www.nap.edu).

The panel applied a largely qualitative rather than quantitative approach to the assessment. The approach of the panel relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the core technical competency areas in which ARL and ARO activities are conducted. The panel reviewed selected examples of the scientific and technological research programs at the ARO’s ISD; it was not possible to review all ISD programs and projects

___________________

¹ Army Research Laboratory, “Army Research Office,” http://www.arl.army.mil/www/default.cfm?page=29, accessed October 10, 2018.

² Army Research Laboratory, “Army Research Office: Information Sciences,” http://www.arl.army.mil/www/default.cfm?page=3227, accessed October 10, 2018.

exhaustively. ARO selected the programs and projects that were presented for review. Given the necessarily nonexhaustive nature of the review process, the omission of mention of any particular program or project should not be interpreted as a negative reflection on the omitted program or project.

The panel’s goal was to portray an overall impression of the ARO programs in information sciences while preserving useful mention of suggestions specific to programs that the panel considered to be of special note within the set of those examined. Therefore, the panel strove to identify and report salient examples that supported discussion of accomplishments and opportunities for further improvement with respect to the ISD’s programs.

ASSESSMENT CRITERIA

The panel was asked to consider the following criteria during the review:

1. Overall Scientific Quality and Degree of Innovation: Was there a clear and cogent strategy regarding how each of the program managers’ major objectives are likely to make substantial and unique progress in advancing scientific frontiers of their discipline? Is the research novel, leading the field in an important area, and does it have the appropriate level of risk and payoff? Was related research being sponsored by other major players adequately summarized in terms of approach and goals? Were there areas of duplication?
2. Scientific Opportunity: Is there some reasonable basis (e.g., incipient breakthrough, new understanding, novel theory, etc.) to believe that the scientific objectives might be met? Have the highest priority objectives been selected?
3. Significant Accomplishments: Did the accomplishments represent significant scientific advances? If not, what is the potential that the accomplishments will lead to significant scientific advances? How do the accomplishments map to the stated program goals? Do the accomplishments reflect productivity and ingenuity on the part of the performers?
4. Relevance/Transitions: Is the potential, long-term Army application of the research significant? Were there appropriate examples of significant transitions, or anticipated transitions of research, to follow-on applied research or exploratory development either within industry or within an Army or DoD laboratory?
5. Additional Considerations:
   1. What were the particular strengths in the program, and what were the weaknesses, if any?
   2. If there were notable weaknesses, what are suggestions for improvements in these areas?
   3. Are there any high-priority missed opportunities/areas?
   4. If so, what lower priority area(s) should be reduced or eliminated to accommodate the new area?

The panel was instructed that the following items are outside the scope of the panel’s charge and that these items should not be considered in the assessment:

1. Other divisions or offices within the ARO and ARL: The panel is charged only to assess the Information Sciences Directorate of the ARO.
2. Organizational changes: The ARO organizational structure is not subject to the assessment.
3. Employee morale or motivation: The assessment panel does not conduct scientific surveys nor analyze the data required to assess morale, and is not asked to do so.

4. Funding: The panel is not asked to assess or recommend the amount or sources of ARO funding.

REPORT CONTENT

This chapter discusses the process used to conduct the assessment and report the resulting findings, conclusions, and recommendations. Chapters 2 through 4 provide assessments of the programs within each of the ISD divisions (Computing Science, Network Science, and Mathematical Sciences). Chapter 5 presents findings common across two or more of the divisions.