Summary

The Information Technology Laboratory (ITL) of the National Institute of Standards and Technology (NIST) has been assessed by the Panel on Information Technology, appointed by the National Research Council (NRC). The panel of experts visited the six divisions of the laboratory and reviewed their activities. The scope of the assessment included the following criteria, provided by the Director of NIST in his charge to the NRC: (1) the degree to which laboratory programs in measurement science, standards, and services achieve their stated objectives and fulfill the mission of the operating unit (laboratory); (2) the technical merits and scientific caliber of the current laboratory programs relative to comparable programs worldwide; and (3) the alignment between laboratory research and development (R&D) efforts and those services and other mission-critical deliverables for which the laboratory is responsible. On the basis of its assessment using these criteria, the panel formed the observations and recommendations presented below, among others discussed in this report.

OBSERVATIONS

Observations 1 through 3 below pertain directly to how the ITL is performing with respect to the three assessment criteria from the Director of NIST. Observations 4 through 6 address changes that have taken place since the 2009 assessment performed by the NRC panel appointed for that review.¹ Observations 7 through 10 focus on areas of concern.

1. The programs of the Information Technology Laboratory are focused on research and development that advance measurement science, standards, and technology. As an example, the Virtual Measurement Systems Program has identified the importance of understanding virtual measurements and uncertainties in advancing industry’s increasing reliance on software modeling and simulation in, for example, the design of new, advanced products. Similarly, the Cloud Computing Program is working on a U.S. government cloud computing technology roadmap that is focused on the highest-priority national cloud computing interoperability, portability, and security requirements. The Health Information Technology Program is working to improve standards for health technologies. These programs and the others reviewed are all making substantial progress toward meeting their objectives and are well aligned with the ITL mission and responsibilities.
2. The technical merits and scientific caliber of the current ITL programs are very high relative to comparable programs worldwide as measured by publications and especially by outstanding products such as the Digital Library of Mathematical Functions (DLMF) and the NIST Special Publication 800* series. The DLMF is without peer in the broader

_____________________

¹ National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2009. Washington, D.C.: The National Academies Press, 2009.

community, and the NIST Special Publication 800* series is renowned for providing technically sound, unbiased, relevant guidelines that are frequently adopted voluntarily in private-sector procurements and practices and often mandated by the Office of Management and Budget for use by the federal government.

3. The ITL R&D efforts appear to be carefully aligned with the mission-critical deliverables for which the ITL is responsible. Programs in cloud computing, health information technology, identity management, cybersecurity education, trusted identities, and voting standards are all addressing national priorities in information technology. National priorities with critical information technology aspects are being addressed by projects in biosciences and bioimaging, cyber physical systems, forensics, greenhouse gas measurement, optical medical imaging, public safety communications, quantum information, smart grid, and trusted networking (Internet Protocol Version 6 [IPv6], Domain Name System Security Extensions [DNSSEC]).

4. The Software and Systems Division (SSD) has made great strides since the previous assessment panel registered concerns in its 2009 report.² The most prominent concern was “the lack of strong scientific and administrative leadership within the SSD and also, in some cases, at the programmatic level.”³ Today those concerns are being aggressively addressed, and the SSD has become more focused and better able to respond to its current challenges.

5. The ITL leadership has done an excellent job in filling two critical management positions: division chief for the Computer Security Division (CSD) and division chief for the Software and Systems Division. The ITL management is still faced with finding a permanent chief for the Advanced Network Technologies Division (ANTD).

6. The ITL has struggled with how crosscutting programs—those that involve work in a collaborative fashion across divisions—would be managed, since they do not fit neatly into the divisional structure. The ITL answer has been to use a matrix management structure (a structure in which an individual reports to two supervisors, one functional and one operational). In 2007,⁴ less so in 2009, the panel was aware of considerable angst on the part of management and staff as to how that would work. This year there were no signs of that distress. It appears that the ITL has done an excellent job of working out the kinks and implementing matrix management.

7. The Statistical Engineering Division (SED) is continuing on an even keel with strong leadership and technical expertise. However, as observed in the 2009 assessment report, the division workload is growing but the division is not. The SED is seriously understaffed, and this problem needs to be addressed with some urgency.⁵

_____________________

² Ibid.

³ Ibid., p. 15.

⁴ National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2007. Washington, D.C.: The National Academies Press, 2007.

⁵ National Research Council, An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2009. Washington, D.C.: The National Academies Press, 2009, pp. 2, 9, 14.

8. The Computer Security Division is also understaffed, although neither performance nor morale has as yet been affected.

9. The work of the Applied and Computational Mathematics Division (ACMD) continues to be excellent. However, the scientific culture of the division may not be sufficiently focused on collaboration to address the problems of multiscale and multiphysics involving complex geometries that are emerging as national priorities.

10. The Advanced Network Technologies Division is doing an excellent job in responding to several national priorities in both the short and long term, including its continued outstanding activities in Internet infrastructure protection and its newer efforts in smart grids and public safety communications. The division has also improved the quality of its internal and external collaborations, as well as the quality of its publications. The ANTD is understaffed for the portfolio of activities that it is undertaking. The various teams handling projects with short deadlines do not have as much time to dig into the subjects as they would like or would be useful. Another consequence of the understaffing is that basic research activities are perhaps below levels that are healthy. ANTD management has not yet articulated a long-term, strategic view of networking.

RECOMMENDATIONS

1. At least two ITL divisions, the Statistical Engineering Division and the Computer Security Division, are feeling the constraints of increasing workloads and insufficient staffing (the SED more so than the CSD). If the ITL is to maintain its prominence in these areas, it should consider plans to address the growth that will be needed to support the expanding workload of each of these divisions.
2. Because the trend toward simulations of increasing model fidelity and numerical accuracy is expected to continue, the Applied and Computational Mathematics Division will be called on to play an increasing role in addressing problems that are multidisciplinary. To ensure that the ITL is ready to support this work, the ACMD should devise a strategy to change the scientific culture of the division to meet those increased challenges.
3. The ITL should fill the position of chief of the Advanced Network Technologies Division with a permanent chief. ANTD management should address the understaffing issue in the division, and in particular it should ensure that there are adequate resources to handle both the short- and long-term needs of the division. ANTD management should create a strategic roadmap for the technical work of the division. The roadmap should be useful in managing the division’s resources and portfolio of activities.
4. The ITL should devote attention now to strategic, long-term technical needs in cloud computing that NIST may be called on to address in the future, including questions surrounding the scale of cloud computing and how such a scale could be accommodated in a laboratory or simulation environment.
5. The ITL should consider creating a collaborative effort between the Computer Security Division and the Software and Systems Division that

would be responsible for the creation of standards and guidelines on secure software development for application by government, industry, and academia.

6. The ITL and the Software and Systems Division should reconsider the SSD mission statement, given the fresh focus of the new leadership, and after the SSD strategic planning process is complete.

7. The ITL and the Software and Systems Division should hire additional formally trained individuals in the SSD’s core foundational areas.

8. The Information Access Division (IAD) supports the development of technologies and their transition into the commercial marketplace as well as government applications. The division currently relies on substantial and sustained amounts of other agency (OA) funding (approximately 60 percent of IAD funding). Most of the OA funding is security-related (from the Department of Homeland Security, the Department of Defense, the Federal Bureau of Investigation, and the Intelligence Advanced Research Projects Activity). The reports, standards, and evaluation studies of the IAD are closely followed by academia and industry. In light of increasing foreign dominance of the biometric industry, IAD’s reliance on OA funding, and IAD’s work in support of biometrics technology development, it is important that the IAD and the ITL remain mindful of the NIST mission to promote U.S. innovation and industrial competitiveness, and so IAD efforts should continue to place highest priority on the needs of the nation’s commerce even while pursuing activities involving international sponsors.

9. The ITL should review the approval process of the Institutional Review Board⁶ to maximize the efficiency of the process and minimize unnecessary latency.

_____________________

⁶ See http://www.hhs.gov/ohrp/humansubjects/commonrule/. Accessed July 11, 2011. The Office for Human Research Protections at the Department of Health and Human Services provides oversight for the protection of human subjects in research through the regulations that are spelled out for Institutional Review Boards in the so-called Common Rule (45 C.F.R. 46).