National Academy of Sciences

National Academy of Engineering

Institute of Medicine

National Research Council

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M.Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm.A.Wulf is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M.Alberts and Dr. Wm.A.Wulf are chairman and vice chairman, respectively, of the National Research Council.

This report could not have happened without support from a number of people, and I am grateful for their contributions. On behalf of the Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education, I want first of all to acknowledge our sponsor, the National Science Foundation, and in particular Janice Earle and Jane Butler Kahle for making this project possible.

A number of former staff members from the Center for Science, Mathematics, and Engineering Education (the precursor to the current Center for Education) were instrumental in getting this project off the ground. Susan Loucks-Horsley was the initial project director and saw the Committee through the first year of its work. The former executive and associate executive directors, Rodger Bybee and Joan Ferrini-Mundy, conceptualized the project and provided ongoing support. Their presentations on the nationally developed science and mathematics standards were invaluable to the Committee. Harold Pratt, former director of the Center’s Division on K-12 Policy and Practice, gave ongoing advice throughout the first year of the project. Kristance Coates, the former senior project assistant, provided essential support during the early months of the Committee’s work. We are also grateful to Suzanne Woolsey, the Center’s acting director in fall 1999, for her vigilance in seeing that this project received the support it needed during the Center’s transition.

The Committee was aided greatly by individuals who partici-

pated in our meetings and helped us understand the complex issues involved in examining how nationally developed mathematics, science, and technology standards have influenced K-12 education in the United States. The participants in the May 2000 workshop, listed in Appendix A, also very generously gave two days of their time to discuss the Committee’s work and provided critical input for the final stages of the Committee’s work following the workshop.

Many others have supported this project. The Mathematical Sciences Education Board and the Committee on Science Education K-12 provided oversight and advice to the Committee throughout the project. Several individuals provided advice along the way, including Richard Askey, Audrey Champagne, James J.Gallagher, Daniel Heck, Thomas Romberg, and Mark St. John. Joan Ferrini-Mundy and Harold Pratt, who were instrumental in the early stages of the project, rejoined our efforts in the final stages of the project, providing in-depth comments and advice on drafts of the report. Tina Winters and Viola Horek provided valuable support, coordinated many aspects of the project, and were important in seeing that the project moved forward.

The Committee was resolute in working through the many difficult issues that arose until we achieved consensus on the document. Special thanks are due to Mike Knapp, whose substantial written contributions throughout the process led us to include him as an editor, and to a subset of the Committee that participated in numerous conference calls and attended informal meetings, e.g., at the American Educational Research Association Annual Meeting, as we continued to hammer out the details: Peg Goertz, Mike Knapp, Maria Alicia Lopez Freeman, Patrick Shields, Judy Sowder, and Elizabeth Stage. And it is certainly no exaggeration to say that without the ceaseless support from National Research Council (NRC) staff members Karen Hollweg and Gail Burrill, this document would never have seen the light of day.

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:

Henry L.Alder, University of California, Davis

Lynn Cornett, Southern Regional Education Board, Atlanta, GA

Karen Economopolous, TERC, Cambridge, MA

Manuel Gomez, University of Puerto Rico

Daniel Koretz, RAND, Arlington, VA

Ronald M.Latanision, Massachusetts Institute of Technology

Bill Linder-Scholer, SciMathMN, Roseville, MN

Roland J.Otto, E.O.Lawrence Berkeley National Laboratory, Berkeley, CA

Richard Satchwell, Illinois State University

Scott Stowell, Spokane Public Schools, WA

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Michael W.Kirst, Stanford University, and R. Stephen Berry, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments

were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

Iris R.Weiss, Chair

Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education

Standards have served as a basis of educational reform across the nation as educators and policy makers respond to the call for a clear definition of desired outcomes of schooling and a way to measure student success in terms of these outcomes. Public policies at the federal, state, and local levels have focused on standards as a way to raise achievement levels for all students. At the national level, content standards have been developed in nearly every academic discipline.

As the standards movement has gained momentum, there has been increasing interest among educators and the public concerning the impact of standards on what happens in classrooms. In response to this interest, and as part of its mission of working for improved mathematics and science teaching and learning across the nation, the National Research Council’s Center for Science, Mathematics, and Engineering Education (which became the Center for Education in 2000)—with funding from the National Science Foundation—undertook the task of designing a framework that would aid in the design, conduct, and interpretation of research regarding the influence on student learning of the standards created at the national level in mathematics, science, and technology. In 1999, the Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education began its work. The charge of the Committee was to “develop a framework that can be used to understand the influence of science, mathematics, and technology education standards on programs, policies, and

practices.” This Framework was to provide guidance for the design, conduct, and interpretation of research regarding the influences of nationally developed standards on student learning in mathematics, science, and technology education. Hence, the Committee’s primary concern has been to develop a structure and process for understanding the influence of these educational standards on practices, programs, and policies in K-12 education. The Committee was responsible to two standing boards in the Center, the Committee on K-12 Science Education (COSE K-12) and the Mathematical Sciences Education Board (MSEB), and the Committee’s work reflects the oversight of these two boards.

An overarching problem faced by the Committee was how to structure thinking about the impact of standards in a way that would consider the complex system in which the standards operate—a system that consists of several layers of policy makers, various educators involved in multiple ways, and an array of players with different responsibilities for what takes place in classrooms. It was clear from the beginning that we were not to answer the question of what influence the standards have had on educational reform. It was a constant struggle, however, to keep the focus on a framework and not an analysis of where and how the standards had taken root in education systems and whether they were affecting student learning.

As the Committee worked, we came to a common understanding of some of the issues in the daunting task of laying out a way to think about the influence of the nationally developed standards on what happens within this complex system and eventually on what happens to student learning. We agreed, for example, to focus our thinking on separate but interacting channels within the education system—curriculum, teacher development, and assessment and accountability—and to consider how reform ideas might traverse these channels to reach the classroom. At the same time, we recognized that contextual forces outside the education system, in the public and political world, directly and indirectly affect what

happens in the classroom, and we wanted to be sure the Framework considered those as well. We agreed to concentrate on the standards created by the National Council of Teachers of Mathematics, the National Research Council, and the International Technology Education Association that focus on teaching and learning in mathematics, science, and technology. We also agreed that the focus of our work was not on the quality of the standards documents themselves but rather on their influence. We did not analyze the nationally developed documents nor make judgments about their focus and content, but concentrated on developing ways to think about whether and how these standards were being used in the system and with what results.

As our work progressed, we came to see even more clearly the importance of having the document be neutral, neither advocating nor denigrating the standards and their impact. Our work was to lay out questions that should be answered in order to come to understand the influence of standards, regardless of whether or not the answers supported the thinking in the national documents. The Framework presented in this document is the result of our discussions.

We eventually came to agree that the Framework would describe the system in which the standards operate, then suggest places where the effect of the standards might be observed in this system, which became the organizing structure of the document. We have drawn on a broad body of research related to various aspects of the task, and have also relied heavily on the practical experience of the Committee members and outside experts in designing the Framework. The Committee produced a background paper for its work, and in spring 2000, convened a group of educators and researchers in a workshop to discuss and critique the paper. The Committee used the insights and input from workshop participants to help define the Framework and to stimulate ideas regarding its future uses.

Investigating the Influence of Standards is aimed at researchers

and those who use research in their work, including federal, state, and local policy makers. It is intended to provide a lens for considering the national standards and the role such standards play in improving mathematics, science, and technology education. This book is not intended to be a simple template prescribing a solution but rather lays out the major issues involved in considering the effects of standards in improving student learning, offering questions that should be asked when considering research about the standards. The document can be used to understand particular studies, to identify what the studies do and do not address, and to help consumers interpret their claims. The document is also designed to spur additional research in areas where answers to the questions are not well-formed. While the focus of this document is the nationally developed standards in mathematics, science, and technology, we believe the Framework is general enough to serve as a guide for those at state and district levels and in other disciplines to analyze the impact of their own standards.

Iris R.Weiss, Chair

Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education