Statistical Software Engineering (1996)
Chapter: Front Matter
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievement of engineers. Dr. Harold Liebowitz is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce Alberts and Dr. Harold Liebowitz are chairman and vice-chairman, respectively, of the National Research Council.
This project was supported by the Advanced Research Projects Agency, Army Research Office, National Science Foundation, and Department of the Navy's Office of the Chief of Naval Research. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors. Furthermore, the content of the report does not necessarily reflect the position or the policy of the U.S. government, and no official endorsement should be inferred.
Copyright 1996 by the National Academy of Sciences. All rights reserved.
Library of Congress Catalog Card Number 95-71101
International Standard Book Number 0-309-05344-7
Additional copies of this report are available from:
National Academy Press,
Box 285 2101 Constitution Avenue, N.W. Washington, D.C. 20055 800-624-6242 202-334-3313 (in the Washington metropolitan area)
B-676
Printed in the United States of America
PANEL ON STATISTICAL METHODS IN SOFTWARE ENGINEERING
DARYL PREGIBON,
AT&T Bell Laboratories,
Chair
HERMAN CHERNOFF,
Harvard University
BILL CURTIS,
Carnegie Mellon University
SIDDHARTHA R. DALAL,
Bellcore
GLORIA J. DAVIS,
NASA-Ames Research Center
RICHARD A. DEMILLO,
Bellcore
STEPHEN G. EICK,
AT&T Bell Laboratories
BEV LITTLEWOOD,
City University, London, England
CHITOOR V. RAMAMOORTHY,
University of California, Berkeley
Staff
JOHN R. TUCKER, Director
COMMITTEE ON APPLIED AND THEORETICAL STATISTICS
JON R. KETTENRING,
Bellcore,
Chair
RICHARD A. BERK,
University of California, Los Angeles
LAWRENCE D. BROWN,
University of Pennsylvania
NICHOLAS P. JEWELL,
University of California, Berkeley
JAMES D. KUELBS,
University of Wisconsin
JOHN LEHOCZKY,
Carnegie Mellon University
DARYL PREGIBON,
AT&T Bell Laboratories
FRITZ SCHEUREN,
George Washington University
J. LAURIE SNELL,
Dartmouth College
ELIZABETH THOMPSON,
University of Washington
Staff
JACK ALEXANDER, Program Officer
BOARD ON MATHEMATICAL SCIENCES
AVNER FRIEDMAN,
University of Minnesota,
Chair
LOUIS AUSLANDER,
City University of New York
HYMAN BASS,
Columbia University
MARY ELLEN BOCK,
Purdue University
PETER E. CASTRO,
Eastman Kodak Company
FAN R.K. CHUNG,
University of Pennsylvania
R. DUNCAN LUCE,
University of California, Irvine
SUSAN MONTGOMERY,
University of Southern California
GEORGE NEMHAUSER,
Georgia Institute of Technology
ANIL NERODE,
Cornell University
IMGRAM OLKIN,
Stanford University
RONALD F. PEIERLS,
Brookhaven National Laboratory
DONALD ST. P. RICHARDS,
University of Virginia
MARY F. WHEELER,
Rice University
WILLIAM P. ZIEMER,
Indiana University
Ex Officio Member
JON R. KETTENRING,
Bellcore
Chair,
Committee on Applied and Theoretical Statistics
Staff
JOHN R. TUCKER, Director
JACK ALEXANDER, Program Officer
RUTH E. O'BRIEN, Staff Associate
BARBARA W. WRIGHT, Administrative Assistant
COMMISSION ON PHYSICAL SCIENCES, MATHEMATICS, AND APPLICATIONS
ROBERT J. HERMANN,
United Technologies Corporation,
Chair
STEPHEN L. ADLER,
Institute for Advanced Study
PETER M. BANKS,
Environmental Research Institute of Michigan
SYLVIA T. CEYER,
Massachusetts Institute of Technology
L. LOUIS HEGEDUS,
W.R. Grace and Company
JOHN E. HOPCROFT,
Cornell University
RHONDA J. HUGHES,
Bryn Mawr College
SHIRLEY A. JACKSON,
U.S. Nuclear Regulatory Commission
KENNETH I. KELLERMANN,
National Radio Astronomy Observatory
KEN KENNEDY,
Rice University
THOMAS A. PRINCE,
California Institute of Technology
JEROME SACKS,
National Institute of Statistical Sciences
L.E. SCRIVEN,
University of Minnesota
LEON T. SILVER,
California Institute of Technology
CHARLES P. SLICHTER,
University of Illinois at Urbana-Champaign
ALVIN W. TRIVELPIECE,
Oak Ridge National Laboratory
SHMUEL WINOGRAD,
IBM T.J. Watson Research Center
CHARLES A. ZRAKET,
Mitre Corporation (retired)
NORMAN METZGER, Executive Director
Preface
The development and the production of high-quality, reliable, complex computer software have become critical issues in the enormous worldwide computer technology market. The capability to efficiently engineer computer software development and production processes is central to the future economic strength, competitiveness, and national security of the United States. However, problems related to software quality, reliability, and safety persist, a prominent example being the failure on several occasions of major local and national telecommunications networks. It is now acknowledged that the costs of producing and maintaining software greatly exceed the costs of developing, producing, and maintaining hardware. Thus the development and application of cost-saving tools, along with techniques for ensuring quality and reliability in software engineering, are primary goals in today's software industry. The enormity of this software production and maintenance activity is such that any tools contributing to serious cost savings will yield a tremendous payoff in absolute terms.
At a meeting of the Committee on Applied and Theoretical Statistics (CATS) of the National Research Council (NRC), participants identified software engineering as an area presenting numerous opportunities for fruitful contributions from statistics and offering excellent potential for beneficial interactions between statisticians and software engineers that might promote improved software engineering practice and cost savings. To delineate these opportunities and focus attention on contexts promising useful interactions, CATS convened a study panel to gather information and produce a report that would (1) exhibit improved methods for assessing software productivity, quality, reliability, associated risk, and safety and for managing software development processes, (2) outline a program of research in the statistical sciences and their applications to software engineering with the aim of motivating and attracting new researchers from the mathematical sciences, statistics, and software engineering fields t o tackle these important and pressing problem areas, and (3) emphasize the relevance of using rigorous statistical and probabilistic techniques in software engineering contexts and suggest opportunities for further research in this direction.
To help identify important issues and obtain a broad range of perspectives on them, the panel organized an information-gathering forum on October 11-12, 1993, at which 12 invited speakers addressed how statistical methods impinge on the software development process, software metrics, software dependability and testing, and software visualization. The forum also included consideration of nonstandard methods and select case studies (see the forum program in the appendix). The panel hopes that its report, which is based on the panel's expertise as well as information presented at the forum, will contribute to positive advances in software engineering and, as a subsidiary benefit, be a stimulus for other closely related disciplines, e.g., applied mathematics, operations research, computer science, and systems and industrial engineering. The panel is, in fact, very enthusiastic about the opportunities facing the statistical community and hopes to convey this enthusiasm in this report.
The panel gratefully acknowledges the assistance and information provided by a number of individuals, including the 12 forum speakers—T.W. Keller, D. Card, V.R. Basili, J.C. Munson, J.C. Knight, R. Lipton, T. Yamaura, S. Zweben, M.S. Phadke, E.E. Sumne r, Jr., W. Hill, and J. Stasko—four anonymous reviewers, the NRC staff of the Board on Mathematical Sciences who supported the various facets of this project, and Susan Maurizi for her work in editing the manuscript.
