1
Introduction
STATEMENT OF TASK
Starting in 1959, the National Academies of Sciences, Engineering, and Medicine have annually assembled panels of experts—from academia, industry, medicine, and other scientific and engineering communities of practice—to assess the quality and effectiveness of the six National Institute of Standards and Technology (NIST) measurements and standards laboratories,1 as well as the adequacy of the laboratories’ resources. These reviews are conducted under contract at the request of NIST.
For fiscal year (FY) 2024, NIST looks forward to the panel’s review of the Information Technology Laboratory (ITL). The assessment of ITL will address the following factors at the request of the NIST Director:
- Assess the organization’s technical programs.
- How does the quality of the research compare to similar world-class research in the technical program areas?
- ls the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
- Assess the portfolio of scientific expertise within the organization.
- Does the organization have world-class scientific expertise in the areas of the organization’s mission and program objectives? If not, in what areas should it be improved?
- How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
- Assess the adequacy of the organization’s facilities, equipment, and human resources.
- How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
- Assess the effectiveness by which the organization disseminates its program outputs.
- How well are the organization’s research programs driven by stakeholder needs?
- How effective are the dissemination methods and technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
- How well is this organization monitoring stakeholder use and impact of program outputs? How could this be improved?
CONDUCT OF THE ASSESSMENT
The overall structure of ITL, its goals, and the associated programs are described in Chapter 2. This report assesses three of ITL’s divisions:
___________________
1 The six NIST laboratories are the Center for Neutron Research, Communications Technology Laboratory, Engineering Laboratory, Information Technology Laboratory, Material Measurement Laboratory, and Physical Measurement Laboratory.
- Applied Cybersecurity Division (ACD)
- Applied and Computational Mathematics Division (ACMD)
- Computer Security Division (CSD)
The panel held a meeting and site visit on June 4–6, 2024, in Gaithersburg, Maryland. At this meeting, the panel as a whole received an introductory overview of NIST and ITL. The panel then broke into three separate subpanels that met independently and in parallel with the ITL staff aligned with the divisions assessed in this report. These subpanel meetings included structured presentations, discussions, and tours. The panel as a whole also had a working lunch with early career ITL staff and postdoctoral researchers.
STRUCTURE OF THIS REPORT
This report opens with this introductory chapter, followed by an overview of ITL. Each ITL division assessed in this report is then presented in its own chapter. The structure within each of these chapters is aligned with the statement of task presented above to aid the reader in understanding the panel’s assessment. Following the assessment chapters is a chapter that presents selected recommendations from the FY 2018 and FY 2021 ITL assessment reports—the last time these divisions were assessed—and ITL’s responses to those recommendations. The final chapter presents the recommendations from the current report in one place for ease of reference. The structure of this report is laid out thus:
- Chapter 1: Introduction
- Chapter 2: Overview of the Information Technology Laboratory
- Chapter 3: Applied Cybersecurity Division
- Chapter 4: Applied and Computational Mathematics Division
- Chapter 5: Computer Security Division
- Chapter 6: Information Technology Laboratory’s Responses to the Recommendations of Previous Assessment Reports
- Chapter 7: Overarching Themes, Key Recommendations, and Chapter Recommendations
To draft this report, the panel reviewed the material provided by ITL before and during the review meeting. ITL chose what information to provide to the panel. The panel applied a largely qualitative approach to the assessment, using the members’ professional experience, expertise, and judgment to conduct the assessment. The panel was quantitative where possible, but much of this assessment is, by its nature, subjective, and the panel’s opinions are based on the facts presented to it.
Because this assessment depends on the information presented by ITL, it is not exhaustive. Similarly, there are natural variations among the assessment chapters (Chapters 3–5) in terms of length, level of detail, and approach. These convey no message about the quality of work being performed by ITL or the information provided to the panel. Each assessment chapter was drafted by one of the subpanels and reflects the content that the ITL staff chose to present to each subpanel and the level of detail provided to that subpanel. The assessment chapters are also not a comprehensive presentation of the entirety of the information provided to the subpanels. Rather, each subpanel selected what stood out to its members in fulfillment of its statement of task and drafted the chapter around those items. Thus, the omission in this report of any particular ITL project is not a negative reflection of the omitted project.
Last, the statement of task asks in some places if the work of ITL is “world-class” or how it compares with work at other international institutes. This is always a subjective assessment based on the totality of the panelists’ knowledge and experience. No comprehensive picture of work around the world
was compiled; there was no time for it. Also, in many instances, NIST’s work is unique in the world. This, itself, makes much of the work world-class or world-leading.
USE OF THE 2023 NIST CAPITAL FACILITY NEEDS REPORT IN THE PANEL’S WORK
This report adopts the full description of the problems identified in Technical Assessment of the Capital Facility Needs of the National Institute of Standards and Technology (NASEM 2023). Box 1-1 summarizes that report and its findings and recommendations.
BOX 1-1
Technical Assessment of the Capital Facility Needs of the National Institute of Standards and Technology
In February 2023, the National Academies of Sciences, Engineering, and Medicine released the report Technical Assessment of the Capital Facility Needs of the National Institute of Standards and Technology (NASEM 2023; hereafter, the Capital Facility Needs report). The committee that authored this report was tasked with assessing the National Institute of Standards and Technology’s (NIST’s) facilities and utility infrastructure, and reviewing and assessing plans and projects to reinvigorate NIST’s facilities and utility infrastructure, the cost estimates for doing so, and the factors that NIST should consider in developing a comprehensive capital strategy for the facilities and utility infrastructure at NIST’s campuses in Boulder, Colorado, and Gaithersburg, Maryland. The committee engaged with the Department of the Interior, the National Institutes of Health, the U.S. Army Engineer Research and Development Center, and the Johns Hopkins University Applied Physics Laboratory to learn about their methods and metrics for assessing facility conditions and maintaining their facilities.
The condition of NIST’s facilities and utility infrastructure has been a concern since 2002, when the Visiting Committee on Advanced Technology (VCAT) issued a report calling NIST’s facilities condition and the related funding situation “alarming” and “critical.” Over the following 20 years, the VCAT returned consistently to this theme with increasingly dire language. Eventually, the conference report accompanying the Consolidated Appropriations Act of 2021 (P.L. 116-260) requested that NIST “contract with an independent entity to develop a report that assesses the comprehensive capital needs of NIST’s campuses.” In response, NIST’s Office of Facilities and Property Management approached the National Academies to conduct a study based on a successful study and report completed for the National Institutes of Health in 2019 (NASEM 2019). The result was the Capital Facility Needs report (NASEM 2023).
The committee that authored the Capital Facility Needs report visited both the Boulder, Colorado, and the Gaithersburg, Maryland, campuses. It discovered that many NIST facilities are inadequate to support the world-leading research that is NIST’s mission. Both the quality and the reliability of power can be problematic, resulting in slowed work, lost work, and unnecessary time spent recalibrating sensitive instruments. Inadequacies in basic environmental controls can result in laboratories that are too hot or cold, too humid, or not humid enough, and lack proper vibration insulation. In one 1950s-era Boulder laboratory, the gaps between the windows and frames allow dust to blow straight into the laboratory. Roof leaks have destroyed multimillion-dollar pieces of equipment, such as tunneling electron microscopes in both Boulder and Gaithersburg. A water leak in Gaithersburg resulted in permanent damage to the world-leading Kibble balance that tied the standard kilogram to the speed of light. There are many more instances and stories. In all, the committee found that the NIST research staff loses 10–40 percent of its working time fighting against facility inadequacies, also consuming research money to do so. Conditions have reached the point where NIST researchers will not be able to continue their world-class research no matter their efforts. This is already impacting the ability to recruit and retain staff and the willingness of foreign researchers to do work at NIST. At risk also is NIST’s international credibility and influence and its ability to support national security, U.S. international competitiveness, medical therapeutics, and a wide range of other activities on which users in the U.S. government, industry, and academia rely.
TABLE 1-1-1 Overview of NIST Facility and Infrastructure Funding Needs
| Funding Component | Amount Needed Annually |
|---|---|
| Construction and major renovations (CMR) | $300 million to $400 million |
| Safety, capacity, maintenance, and major repairs (SCMMR) | $120 million to $150 million |
| Total needed for construction of research facilities (CRF) | $420 million to $550 million |
NOTE: CRF funding is the sum of CMR and SCMMR funding.
SOURCE: NIST (2022).
In the course of its work, the committee found that NIST’s internal facility and property management policies are not responsible for this situation. Rather, the cause is more than 2 decades of erratic, unpredictable, and inadequate funding for NIST’s construction of research facilities budget, which includes facility sustainment, restoration, modernization, and expansion. Exacerbating this problem is congressionally directed pass-through funding for items such as building laboratories on university campuses that are not used by NIST. This pass-through funding is not revenue-neutral to NIST, costing staff time and money to administer, draining even more much-needed money from NIST’s facilities coffers.
In short, the committee found that the situation requires serious and sustained attention, particularly from leadership levels above NIST. The committee also endorsed the coordinated recovery plan drafted by NIST’s Office of Facilities and Property Management and recommended its continued refinement and shortening its timeline to completion in 12 years. Critically, the committee identified the need for significant and sustained funding to address NIST’s facilities and utility shortcomings and bring them to the standard necessary for modern metrology. This funding is the critical piece of the recovery plan. The committee recommends $420 million–$550 million per year in funding for NIST’s construction of research facilities budget over at least 12 years. As shown in Table 1-1-1, this includes $120 million–$150 million per year for safety, capacity, maintenance, and major repairs funding to address the more than $800 million deferred maintenance backlog and to bring existing facilities to an acceptable condition and keep them there. It also includes $300 million–$400 million per year over at least 12 years for the construction and major renovations budget to upgrade, renovate, and build the new laboratories with the new capabilities needed to conduct modern metrology research.
The picture is not unremittingly bleak. NIST has already begun to modernize laboratories as its current budget allows. These new laboratories are state of the art and enable the cutting-edge world-leading research that is NIST’s mission. As an example, one NIST research group—after waiting 18 months to be relocated into a new, modern laboratory—won the 2021 Physics World Breakthrough of the Year award for a previously unprecedented demonstration of the quantum entanglement of microresonators. NIST’s staff is world-class and capable of producing amazing results, results that will serve the nation and inspire the next generations of researchers, provided they are given the facilities and tools needed to do their work.
SOURCE: NASEM (2023).
REFERENCES
NASEM (National Academies of Sciences, Engineering, and Medicine). 2019. Managing the NIH Bethesda Campus Capital Assets for Success in a Highly Competitive Global Biomedical Research Environment. The National Academies Press. https://doi.org/10.17226/25483.
NASEM. 2023. Technical Assessment of the Capital Facility Needs of the National Institute of Standards and Technology. The National Academies Press. https://doi.org/10.17226/26684.
NIST (National Institute of Standards and Technology). 2022. “NIST Facilities Summary for Representative Trone.” Point Paper. June. Office of Facilities and Property Management.
