1

The Charge to the Panel and the Assessment Process

At the request of NIST, the National Academies of Sciences, Engineering, and Medicine has, since 1959, annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now six,¹ as well as the adequacy of the laboratories’ resources.

At the request of the Director of NIST, in 2020 the National Academies formed the Panel on Assessment of Selected Divisions of the Physical Measurement Laboratory at the National Institute of Standards and Technology and established the following statement of task for the panel:

The NIST Director requested that the panel focus its assessment on the following factors:

1. Assess the organization’s technical programs.
   • How does the quality of the research compare to similar world class research in the technical program areas?
   • Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
2. Assess the portfolio of scientific expertise within the organization.
   • Does the organization have world-class scientific expertise in the areas of the organization’s mission and program objectives? If not, what areas should be improved?
   • How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
3. Assess the adequacy of the organization’s facilities, equipment, and human resources.
   • How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
4. Assess the effectiveness by which the organization disseminates its program outputs.

___________________

¹ The six NIST laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology Laboratory, and the NIST Center for Neutron Research.

• How well are the organization’s research programs driven by stakeholder needs?
• How effective are the technology transfer mechanisms used by the organization? Are these sufficiently comprehensive?
• How well is the organization monitoring stakeholder use and impact of program outputs? How this be improved?

The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct research to anticipate future metrology and standards needs, enable new scientific and technological advances, and improve and refine existing measurement methods and services.

NIST specified that 5 of the 10 divisions of the PML would be reviewed—Quantum Measurement Division, Radiation Physics Division, Microsystems and Nanotechnology Division, Nanoscale Device Characterization Division, and Sensor Science Division. The following PML divisions were not reviewed because they had previously been reviewed (NIST may request their review in the future): Applied Physics Division, Quantum Electronics Division, Quantum Physics Division, Time and Frequency Division, and Weights and Measures Division. In order to accomplish the assessment, the National Academies assembled a panel of 24 volunteers whose expertise matches that of the work performed by PML staff.²

On May 17-20, 2021, the panel conducted a virtual review (via Internet media). During a plenary session, the panel received overview presentations by the acting NIST Director and the director of the PML. Subsequently, the panel spent approximately 1.5 days receiving presentations from and engaging in discussions with the staff at the five divisions reviewed. On the third day, the panel met in a closed session to deliberate on its findings and define the contents of this assessment report. The panel met with NIST management on the fourth day to clarify open questions.

The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. The panel reviewed selected examples of the technical research performed at PML; because of time constraints, it was not possible to review PML programs and projects exhaustively. The examples reviewed by the panel were selected by PML. The panel’s goal was to identify and report salient examples of accomplishments, challenges, and opportunities for improvement with respect to the factors suggested above by the NIST Director. These examples are intended collectively to portray an overall impression of the laboratory, while providing useful suggestions specific to the projects and programs that the panel examined. The panel applied a largely qualitative rather than quantitative approach to the assessment.

Given the necessarily broad and non-exhaustive nature of the review, omission in this report of any particular PML program or project should not be interpreted as implying any negative reflection on the omitted program or project. Because the divisions reviewed by the separate teams address separate specific missions and tasks, the type of information provided by each division was inconsistent with respect to some metrics. The lack of discussion of a metric, such as publications, in one division but not another should not be taken as an indication that there was a difference in performance on the given metric between the divisions.

___________________

² See on the NIST Physical Measurement Laboratory homepage for information on organization and programs at https://www.nist.gov/pml accessed May 17, 2921.