Evolving the High Performance Computing and Communications Initiative to Support the Nation's Information Infrastructure

Committee to Study High Performance Computing and Communications:
Status of a Major Initiative

Computer Science and Telecommunications Board

Commission on Physical Sciences, Mathematics, and Applications

National Research Council

NATIONAL ACADEMY PRESS
Washington, D.C. 1995

Page ii

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce Alberts and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council.

Support for this project was provided by the U.S. Department of Defense through the Advanced Research Projects Agency under Grant No. MDA972-94-1-0008. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Department of Defense or the Advanced Research Projects Agency.

Library of Congress Catalog Card Number 95-67707

International Standard Book Number 0-309-05277-7

Copyright 1995 by the National Academy of Sciences. All rights reserved.

Available from:

National Academy Press
2101 Constitution Avenue, NW
Washington, DC 20418

B-540

Printed in the United States of America

Page iii

COMMITTEE TO STUDY HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS: STATUS OF A MAJOR INITIATIVE

FREDERICK P. BROOKS, JR., University of North Carolina at Chapel Hill, Co-chair

IVAN E. SUTHERLAND, Sun Microsystems Laboratories, Co-chair

ERICH BLOCH, Council on Competitiveness

DEBORAH ESTRIN, University of Southern California/Information Sciences Institute

JOHN HENNESSY, Stanford University

BUTLER W. LAMPSON, Digital Equipment Corporation

EDWARD D. LAZOWSKA, University of Washington

WILLIAM A. LESTER, JR., University of California at Berkeley

JANE PRESTON, Telemedical Interactive Consultative Services Inc.

W. DAVID SINCOSKIE, Bell Communications Research Inc.

LARRY SMARR, National Center for Supercomputing Applications/University of Illinois at Urbana-Champaign

JOSEPH F. TRAUB, Columbia University

Staff

MARJORY S. BLUMENTHAL, Director

JAMES E. MALLORY, Staff Officer

LESLIE M. WADE, Project Assistant

Page iv

COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD

WILLIAM WULF, University of Virginia, Chair

FRANCES ALLEN, IBM T.J. Watson Research Center

JEFF DOZIER, University of California at Santa Barbara

DAVID J. FARBER, University of Pennsylvania

HENRY FUCHS, University of North Carolina

CHARLES M. GESCHKE, Adobe Systems Inc.

JAMES GRAY, San Francisco, California

BARBARA J. GROSZ, Harvard University

DEBORAH A. JOSEPH, University of Wisconsin

RICHARD M. KARP, University of California at Berkeley

BUTLER W. LAMPSON, Digital Equipment Corporation

BARBARA H. LISKOV, Massachusetts Institute of Technology

JOHN MAJOR, Motorola Inc.

ROBERT L. MARTIN, AT&T Network Systems

DAVID G. MESSERSCHMITT, University of California at Berkeley

WILLIAM H. PRESS, Harvard University

CHARLES L. SEITZ, Myricom Inc.

EDWARD SHORTLIFFE, Stanford University School of Medicine

CASMIR S. SKRZYPCZAK, NYNEX Corporation

LESLIE L. VADASZ, Intel Corporation

MARJORY S. BLUMENTHAL, Director

LOUISE A. ARNHEIM, Senior Staff Officer

HERBERT S. LIN, Senior Staff Officer

JAMES E. MALLORY, Staff Officer

RENEE A. HAWKINS, Staff Associate

JOHN M. GODFREY, Research Associate

GLORIA P. BEMAH, Administrative Assistant

LESLIE M. WADE, Project Assistant

Page v

COMMISSION ON PHYSICAL SCIENCES, MATHEMATICS, AND APPLICATIONS

RICHARD N. ZARE, Stanford University, Chair

RICHARD S. NICHOLSON, American Association for the Advancement of Science, Vice Chair

STEPHEN L. ADLER, Institute for Advanced Study

SYLVIA T. CEYER, Massachusetts Institute of Technology

SUSAN L. GRAHAM, University of California at Berkeley

ROBERT J. HERMANN, United Technologies Corporation

RHONDA J. HUGHES, Bryn Mawr College

SHIRLEY A. JACKSON, Rutgers University

KENNETH I. KELLERMANN, National Radio Astronomy Observatory

HANS MARK, University of Texas at Austin

THOMAS A. PRINCE, California Institute of Technology

JEROME SACKS, National Institute of Statistical Sciences

L.E. SCRIVEN, University of Minnesota

A. RICHARD SEEBASS III, University of Colorado

LEON T. SILVER, California Institute of Technology

CHARLES P. SLICHTER, University of Illinois at Urbana-Champaign

ALVIN W. TRIVELPIECE, Oak Ridge National Laboratory

SHMUEL WINOGRAD, IBM T.J. Watson Research Center

CHARLES A. ZRAKET, MITRE Corporation (retired)

NORMAN METZGER, Executive Director

There was a problem loading page R6.

Page vii

Preface

In early 1994, acting through the Defense Authorization Act for FY 1994 (Public Law 103160), Congress asked the National Research Council (NRC) to examine the status of the High Performance Computing and Communications Initiative (HPCCI). Broad-based interest in and support for the HPCCI exist. Given its scope and size, concerns had been raised about its goals, management, and progress. Congress asked that at a minimum the study address:

· The basic underlying rationale(s) for the program, including the appropriate balance between federal efforts and private-sector efforts;
· The appropriateness of its goals and directions;
· The balance between various elements of the program;
· The effectiveness of the mechanisms for obtaining the views of industry and users for the planning and implementation of the program;
· The likelihood that the various goals of the program will be achieved;
· The management and coordination of the program; and
· The relationship of the program to other federal support of high-performance computing and communications, including acquisition of high-performance computers by federal departments and agencies in support of the mission needs of these departments and agencies.

For this study the NRC's Computer Science and Telecommunications Board (CSTB) convened a committee of 12 members, expert on pioneering applications of computers and communications and the major components of the HPCCI: High-Performance Computing Systems, Advanced Software Technology and Algorithms, the National Research and Education Network, Basic Research and Human Resources, and Information Infrastructure Technology and Applications. Congress asked the committee to accelerate the normal NRC study process in order to provide an interim report by July 1, 1994, and a final report by February 1, 1995. The committee was able to meet this rapid turnaround by drawing on the knowledge and experience of its members as expressed in committee deliberations and by obtaining input from numerous outside experts.

The full committee met six times between March 10, 1994, and December 20, 1994, to hear more than 25 high-performance computing and communications users, builders, and scientists; to discuss the HPCCI in detail; and to produce this report. Additionally, smaller groups of committee members made site visits to discuss first-hand the use of high-performance technologies. These visits involved another 6 individuals from the Ford Motor Company and approximately 50 high-performance computing and communications users who had gathered at a workshop to discuss the use of high-performance systems in environmental research and simulation. In addition to examining the current status of the program, the committee considered the evolution of the HPCCI

Page viii

and its goals and alternate government investment strategies related to technological development. The committee took into account varying perspectives on the initiative's goals and available assessments of progress toward achieving them.

The committee's interim report provided technical background and perspective on the overall development of high-performance computing and communications systems, as well as on the HPCCI, formally started in 1991.1 The interim report made two recommendations: (1) strengthen the National Coordination Office to help it meet increasing future demands for program coordination and information functions; and (2) immediately appoint the congressionally mandated HPCCI Advisory Committee to provide broad-based, active input to the initiative. As this final report goes to press, both recommendations have yet to be acted on and thus require executive and legislative attention.

This final report, Evolving the High Performance Computing and Communications Initiative to Support the Nation's Information Infrastructure, purposely adopted a broad perspective so as to examine the HPCCI within the context of the evolving information infrastructure and national economic competitiveness generally. Committee deliberations consistently pointed out the important contributions that computing and communications research have made to the nation's economy, scientific research, national defense, and social fabric. That research has nourished U.S. leadership in information technology goods, services, and applications. Traditionally, the most powerful computers and the fastest networks made many of those contributions. Recently, however, the widespread availability of significant computing and communications capabilities on the nation's desktops and factory floors has also produced many benefits. The broadening and interconnection of more and more computer-based systems call attention to research needs associated with system scale as well as performance.

The Committee to Study High Performance Computing and Communications: Status of a Major Initiative is grateful for the help, encouragement, and hard work of the NRC staff working with us: Marjory Blumenthal, Jim Mallory, Susan Maurizi, and Leslie Wade. Our meetings went smoothly because of their careful preparation. This report came together because of their attention and diligence. They patiently assembled sometimes conflicting text from diverse authors and helped reconcile it with the critique of our reviewers. They searched out facts of importance to our deliberations. Their excellent staff preparation helped us focus on the substance of our task.

Many others also made valuable contributions to the committee. In addition to individuals listed in Appendix F who briefed the committee, the committee appreciates inputs from Sally Howe and Don Austin (National Coordination Office); Bob Borchers, Paul Young, Dick Kaplan, and Robert Voight (National Science Foundation); Eric Cooper (FORE Systems); Stephen Squires (Advanced Research Projects Agency); Sandy MacDonald (National Oceanic and Atmospheric Administration); Robert Bonometti (Office of Science and Technology Policy); and Al Rosenheck (former congressional staffer). It is also grateful to the anonymous reviewers who helped to sharpen and focus the report with their insightful comments. Responsibility for the report, of course, remains with the committee.

1Computer Science and Telecommunications Board (CSTB), National Research Council. 1994. Interim Report on the Status of the High Performance Computing and Communications Initiative. Computer Science and Telecommunications Board, Washington, D.C.

Page ix

Finally, we want to acknowledge the contributions to our present task of research projects 30 years past. Of course this text was all word processed. Of course the charts were drafted on computers. Of course we used Internet communication nationwide to plan our meetings, share our thoughts, reconcile our differences, and assemble our report. We plugged our portable computers into a local network at each of our meetings, sending drafts to local laser printers. In short, we have partaken fully of the fruits of the HPCCI's precursors. We thank the visionaries of the past for our tools.

Frederick P. Brooks
Ivan E. Sutherland
Co-chairs
Committee to Study High Performance Computing and Communications:
Status of a Major Initiative

There was a problem loading page R10.

Page xi

Contents

Page xii

Page xiii

Industry, if left to itself, will naturally find its way to the most useful and profitable employment. Whence it is inferred that manufacturers, without the aid of government, will grow up as soon and as fast as the natural state of things and the interest of the community may require.

Against the solidity of this hypothesis . . . very cogent reasons may be offered . . . [including] the strong influence of habit; the spirit of imitation; the fear of want of success in untried enterprises; the intrinsic difficulties incident to first essays towards [competition with established foreign players]: the bounties, premiums, and other artificial encouragements with which foreign nations second the exertions of their own citizens ... To produce the desirable changes as early as may be expedient may therefore require the incitement and patronage of government.
—Alexander Hamilton, 1791, Report on Manufactures

There was a problem loading page R14.