Anton E. Lawson

Department of Biology, Arizona State University

The Arizona Collaborative for Excellence in the Preparation of Teachers (ACEPT) Program is a National Science Foundation (NSF)-sponsored program aimed at improving undergraduate science and mathematics instruction at Arizona State University (ASU) and in the surrounding community colleges. The primary reform mechanism has been summer workshops in which college faculty experience reformed teaching methods and then attempt to implement those methods in their courses. The reformed methods are based on the principles of effective teaching introduced by the American Association for the Advancement of Science (AAAS) in Science for All Americans (1989). In turn, the AAAS teaching principles (see Box A-1) are based on learning theory derived from years of cognitive research. That theory posits that learning results from active, learner-centered inquiry in which students construct new concepts and conceptual systems by connecting new information and concepts to what they already believe. Further, effective learning often requires restructuring, or even discarding, previous concepts and beliefs when they prove incompatible with, or contradictory to, new evidence and new concepts (e.g., Alexander and Murphy, 1999).

The ACEPT program has attempted to incorporate reformed teaching methods into several courses for nonmajors and majors. These include

Introduction to Physical Geology, Fundamentals of Physical Science, Theory of Elementary Mathematics, Patterns in Nature, The Living World, University Physics, and Methods of Teaching Biology. Evaluation has focused on two central questions: What effect, if any, have the summer workshops had on participant faculty’s use of reformed teaching methods? And what effect, if any, does the use of reformed methods have on student achievement? The following sections describe evaluation efforts in five courses and a brief evaluation of the teaching methods used by some recent graduates as they begin their elementary, middle, or high school teaching careers.

Fundamentals of Physical Science (PHS 110) is an introductory course designed specifically for preservice elementary school teachers. A test of physics concepts, developed by course instructors and the ACEPT evaluation team, was administered to four experimental and two control PHS 110 sections at the beginning and again at the end of a recent semester. A member of the ACEPT Program at ASU (the principal investigator) taught one experimental section. Community college instructors who had participated

in an ACEPT summer workshop taught the other three experimental sections. Importantly, these instructors were not selected at random. Rather, they were selected because they exhibited considerable variation in the extent to which they appeared to be embracing the reformed methods during the summer workshop. Community college instructors who had not participated in a summer workshop taught the two control sections.

Instructional methods were evaluated using an ACEPT-developed instrument called the Reformed Teaching Observation Protocol (RTOP). The RTOP consists of 25 statements about the extent to which reforms are incorporated into instructional practice (see Box A-2; details available at http://ecept.net/rtop/). Each statement is scored on a 0–4 “Never Occurred” to “Very Descriptive” scale. Thus, the RTOP allows observers to rate instruction on a 0–100 scale. Details of RTOP development and administration can be found in Sawada (1999), Sawada et al. (2000a), and Sawada et al. (2000b). Estimates of inter-rater reliability have been obtained using seven trained evaluators as they observed several math and science instructors and independently scored several lessons. Inter-rater reliabilities have been high as evidenced by the following pairs of independent observations and respective coefficients (16 pairs, r = 0.94; 4 pairs, r = 0.99; 7 pairs, r = 0.97; 6 pairs, r = 0.94; 5 pairs, r = 0.93; 9 pairs, r = 0.90).

Mean RTOP scores for each PHS 110 instructor and the respective normalized pre- to posttest achievement gains (i.e., percent gain/percent gain possible) for each instructor’s students (n = number of students in each section) were calculated. Among the experimental sections, mean RTOP scores varied from 27 to 73. Mean RTOP scores for the two control instructors were 28 and 37. Normalized achievement gains varied from 0–57 percent across all sections. Importantly, mean instructor RTOP scores correlated strongly with student achievement gains (r = 0.88, p < 0.05). This result supports the claim that reformed teaching methods promote higher achievement. Figure A-1 shows instructor RTOP scores and normalized gains on the test of physics concepts for ACEPT (experimental) and control sections.

Theory of Elementary Mathematics (MTE 180) is an introductory course designed specifically for preservice elementary school teachers. Four MTE 180 instructors participated in the initial ACEPT summer workshop. Subsequently, one of those instructors (from ASU) helped two additional ASU MTE 180 instructors develop reformed teaching methods. During a recent

semester, six sections of MTE 180 participated in a study. Three ACEPT-influenced instructors taught three sections at ASU and control instructors taught three sections (one at ASU and two at a nearby community college). A test measuring concept understanding, number sense, and computational skills was administered at the beginning and again at the end of the semester. During the semester, each instructor was evaluated at least twice using the RTOP.

Instructor mean RTOP scores and student posttest scores on the concept-understanding test were calculated for each section. Instructor mean RTOP

scores and student posttest scores were found to correlate strongly (r = 0.94, p < 0.001). Mean RTOP scores and normalized gains also correlated strongly (r = 0.86, p < 0.001). A very strong positive correlation was also found between instructors’ mean RTOP scores and student posttest number sense scores (r = 0.92, p < 0.001). These results further support the claim that reformed teaching methods improve student achievement. As predicted, no relationship was found between instructors’ mean RTOP scores and student posttest performance on the computational skills section. This result was

FIGURE A-1 Instructor RTOP scores and normalized gains on the test of physics concepts for ACEPT and control sections of PHS 110.

SOURCE: Lawson et al. (2002, p. 390). Reprinted with permission of National Science Teachers Association.

predicted because items in this section required only routine algorithmic procedures.

The Living World (BIO 100) is an introductory biology course enrolling about 750 students per semester. A faculty member presents three 50-minute lectures each week. Graduate teaching assistants (TAs) teach the labs. Labs meet once each week for two hours. Students must enroll for both the common lectures (all delivered by the faculty member) and one of several lab sections (each taught by one of the several TAs). TAs are introduced to reformed teaching methods during a three-day summer workshop followed by two-hour TA meetings each Friday during the fall semester.

A primary goal of BIO 100 is to improve students’ reasoning skills. Consequently, during the past several semesters, a 25-item pre- and posttest of reasoning skills has been administered (Lawson et al., 2000). Figure A-2 shows the frequency of students at each score on both the pre- and posttest and reveals substantial and statistically significant pre- to posttest gains from a recent semester (dependent T = 14.9, p < 0.001).

FIGURE A-2 Pre and posttest scientific reasoning scores for students enrolled in BIO 100.

SOURCE: Lawson et al. (2002, p. 391). Reprinted with permission of National Science Teachers Association.

During that semester, the nine TAs were independently evaluated using the RTOP. Regardless of the fact that all TAs were introduced to teaching reforms in the same manner, and all the BIO 100 labs are inquiry (learning cycle) based, TA mean RTOP scores varied from 42 to 90 (inter-rater reliability of r = 0.90, p < 0.001). Importantly, TA mean RTOP scores correlated significantly with normalized gains in student reasoning (r = 0.70, p < 0.05).

University Physics 1: Mechanics (PHY 121) is an introductory course designed for physics majors that focuses on mechanics. A course evaluation was conducted using three experimental sections of PHY 121 (i.e., sections taught by ACEPT-influenced instructors). Two experimental sections were taught at ASU and one was taught at a community college. A non-ACEPT-influenced instructor taught the control section at a community college. A diagnostic test of mechanics concepts called the Force Concept Inventory (Halloun and Hestenes, 1985) was administered to all sections to assess pre- to posttest gains. Instructors’ mean RTOP scores and normalized gains were compared and a strong positive correlation was found (r = 0.97, p < 0.01). Once again, this indicates that reformed teaching methods promote student achievement.

Methods of Teaching Biology (BIO 480) is taught at ASU each spring for preservice biology teachers after they have completed, or are about to complete, an undergraduate biology major. In addition to using reformed methods to teach the preservice teachers about those reformed methods, the course attempts to help students develop their reasoning skills and improve their understanding of the nature of science (NOS). During a recent semester, students’ reasoning skills (classified into developmental stages 3, 4, and 5) were assessed using the previously mentioned reasoning test (Lawson et al., 2000). Students were also pre- and posttested using a 13-item ACEPT-developed survey of the nature of science (see Box A-3). The survey includes items that focus on the meaning of terms such as hypothesis, prediction, theory, law, proof, truth, fact, and conclusion. These are terms that are not only central to the business of doing science but are also terms that are used inconsistently and sometimes even contradictorily by many, if not most,

FIGURE A-3 Pretest and posttest performance on BIO 480 students at each developmental level.

SOURCE: Lawson et al. (2002, p. 392). Reprinted with permission of National Science Teachers Association.

scientists. The assumption is made that these inconsistencies and contradictions are confusing to students who are trying to better understand the research process.

As shown in Figure A-3, pretest NOS scores were low and unrelated to developmental level. However, posttest NOS scores were considerably higher. Further, posttest NOS scores were strongly related to developmental level (F_3,22 = 7.38, p < 0.01). These results are important because they suggest that: (1) without explicit NOS instruction, biology majors learn very little about the nature of science, (2) inquiry instruction that includes explicit NOS instruction is effective at improving NOS understanding, but (3) substantial gains in NOS understanding depend, at least in part, on students’ developmental level. Although current research on this last point is preliminary, a plausible prediction is that becoming a skilled inquiry teacher requires advanced reasoning skills and a good understanding of the nature of science. If this is indeed the case, then additional changes in the undergraduate curriculum will need to be made to insure that all students, particularly those who will become teachers, develop advanced reasoning skills.

An important component of the ACEPT evaluation has focused on the teaching effectiveness of recent graduates as they begin their public school teaching careers. A preliminary look at first-year teacher performance reveals significant differences (p = 0.05) in favor of ACEPT-trained teachers (i.e., mean RTOP score of 48 among 20 teachers who had enrolled in an ACEPT-influenced science or mathematics course as undergraduate students compared with a mean RTOP score of 40 among a sample [n = 8] of teachers who had not encountered one or more ACEPT-reformed courses during their teacher preparation program). Similar data for second- and third-year teachers were found. Importantly, the ACEPT-influenced teachers continue to outperform the non-ACEPT teachers (mean RTOP score of 62 versus 45, p < 0.05). Also RTOP performance improved from the first year for both groups. This improvement is encouraging because it suggests that a statewide movement to reform science and mathematics instruction (Arizona Department of Education, 1997) and complementary local reform efforts are having a positive and general impact on instructional reform.

More recently, we have found that ACEPT-influenced high school biology teachers have significantly higher RTOP scores than a group of control teachers. Further, their students demonstrated significantly higher achievement in terms of scientific reasoning, NOS understanding, and understanding of biology concepts than students of control teachers (teacher n = 28, student n = 1,115). Results were most divergent for scientific reasoning. Depending on the amount of ACEPT influence, reasoning skills were from 25–46 percent better among students of ACEPT-influenced teachers (Adamson et al., 2002).

The primary result of the present evaluation is that, when implemented, the AAAS teaching principles lead to improved student achievement in a variety of undergraduate science and mathematics courses. This result not only supports the usefulness of the AAAS teaching principles, but also supports the active, learner-centered, theory upon which those principles are based. Another important aspect of the present evaluation is the development

and validation of a teaching observational protocol (the RTOP). The RTOP enables trained observers to reliably evaluate instruction in terms of the extent to which it incorporates reformed teaching methods. In addition to evaluating current teaching methods, the RTOP could become an important instrument to help instructors improve their classroom instruction. Perhaps a useful extension of the present results would be a study of the sort envisioned by Feuer, Towne, and Shavelson (2002) that explores the relationship between RTOP scores and student achievement over a much larger number and diversity of courses.

The present evaluation indicates that when preservice teachers encounter reformed instruction as undergraduates they are more likely to incorporate those reforms into their own teaching practices after graduation. This result supports the familiar adage that “teachers teach as they have been taught.” This is an important finding as it offers a possible solution to the well-documented need for K–12 curricular reform. Namely, reform the way in which preservice teachers learn science and mathematics as undergraduates and they will carry those reforms with them to K–12 classrooms.

Finally, the results indicate that preservice biology teachers, at least the ones in the present sample, initially know very little about the nature of science. Importantly, acquiring such knowledge appears to be linked to reasoning skill. This suggests that many science majors may not only need help in acquiring understanding of the nature of science, but they may also need help in developing scientific reasoning skills (Anderson and Mitchener, 1994; Coble and Koballa, 1996; Haney, Czerniak, and Lumpe, 1996; Lawson, 1999; Lawson et al., 2000). Clearly, much work remains to be done for college faculty to become more effective in the classroom. Perhaps the present results will contribute to that ongoing process by suggesting one way in which such improvements can be made.

Further questions include:

• Why are some faculty members resistant to reform?

• What can be done to overcome that resistance?

• What, if any, important reformed method/strategy does RTOP not measure?

• What is the best way to help faculty members become skilled teachers?

• What support system needs to be in place to encourage reform?

• What misconceptions exist regarding reform?

This material is based upon research supported by NSF under grant No. DUE 9453610. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author and do not necessarily reflect the views of the NSF. We would like to express gratitude to Terry Woodin, Harriet Taylor, and Joan Prival of the NSF, and O. Roger Anderson, chair of the ACEPT National Advisory Committee, for their continuing support and guidance.

Adamson, S.L., Banks, D., Benford, R., Burtch, M., Cox, F., Judson, E., Turley, J.B., and Lawson, A.E. (2002). Reformed undergraduate instruction and its impact on secondary school teaching practice and student achievement: Does systemic reform work? (ACEPT Technical Report). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.

Alexander, P.A., and Murphy, P.K. (1999). The research base for APA’s learner-centered psychological principles . In N.M. Lambert and B.L. McCombs (Eds.), How students learn: Reforming schools through learner-centered education (pp. 25–60). Washington, DC: American Psychological Association.

American Association for the Advancement of Science. (1989). Science for all Americans. Washington, DC: Author.

Anderson, R.D., and Mitchener, C.P. (1994). Research on science teacher education. In D.L. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 3–44). New York: MacMillan.

Arizona Department of Education. (1997). Arizona academic standards. Phoenix, AZ: Author.

Coble, C.R., and Koballa, T.R. (1996). Science education. In J. Sikula, T. Buttery, E. Guyton (Eds.), Handbook on teacher education (2nd ed.) (pp. 459–484). New York: MacMillan.

Feuer, M.J., Towne, L., and Shavelson, R.J. (2002). Scientific culture and educational reform. Educational Researcher, 31(8), 4–14.

Halloun, I.A., and Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11), 1043– 1055.

Haney, J.J., Czerniak, C.M., and Lumpe, A.T. (1996). Teacher beliefs and intentions regarding implementation of science education reform strands. Journal of Research in Science Teaching, 33(9), 971–993.

Lawson, A.E. (1999). What should students know about the nature of science and how should we teach it? Journal of College Science Teaching, 28(6), 401–411.

Lawson, A.E., Alkhoury, S., Benford, R., Clark, B., and Falconer, K.A. (2000). What kinds of scientific concepts exist? Concept construction and intellectual development in college biology. Journal of Research in Science Teaching. 37(9), 996–1018.

Lawson A.E., Benford, R., Bloom, I., Carlson, M., Falconer, K., Hestenes, D., Judson, E., Piburn, M., Sawada, D., Turley, J., and Wyckoff, S. (2002). Evaluating college science and mathematics instruction. Journal of College Science Teaching, 31(6), 388–393.

Sawada, D. (1999). Psychometric properties of RTOP (ACEPT Technical Report No. IN99-2). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.

Sawada, D., Piburn, M., Falconer, K., Turley, J., Benford, R., Bloom, I., and Judson, E. (2000a). Reformed teaching observation protocol (RTOP) (ACEPT Technical Report No. IN00-1). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.

Sawada, D., Piburn, M., Turley, J., Falconer, K., Benford, R., Bloom, I., and Judson, E. (2000b). Reformed teaching observation protocol (RTOP) training guide (ACEPT Technical Report No. IN00-2). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.

Susan B. Millar

Wisconsin Center for Education Research, University of Wisconsin–Madison

For some 10 years now, I have been learning through my work as an evaluator about education faculty innovators who work in the fields of science, technology, engineering, and mathematics (STEM). As evaluators do, I moved without pause from one interesting evaluation project to the next. In fleeting moments, I have become increasingly certain that the faculty whose courses I was studying³ share certain characteristics, but I had no opportunity to systematically consider and then articulate what these characteristics might be. So it was that upon considering Bob DeHaan’s request to write about how to promote curricular and pedagogical improvements, I reasoned that findings on characteristics shared by STEM faculty who are largely successful at effecting change might help others (faculty as well as professional develop-

ment staff and policy makers on campuses and in funding agencies) effect faculty change.⁴

How would such findings be helpful? Trained as an anthropologist, it is evident to me that people are more able to recognize who they are, and who they are not, by comparing themselves to others. And this process is more effective when the characteristics of the “others” are articulated in some detail. My hope, then, is that individuals who assess themselves and key colleagues in light of a set of characteristics shared by STEM education faculty innovators might better identify, for example, habits or implicit assumptions that may be thwarting their efforts to achieve their goals as educators.

To illustrate, a faculty member who cares deeply about teaching and learning and knows on some level that students should be more actively involved in classroom activities might realize that he differs from the innovators described here, in that he is not willing to hand over some decision-making authority to the students. Noticing this, he may then realize that he believes that students will take responsibility for their own learning but that his practice (based on teaching as he was taught) of maintaining control in the class at all times is at odds with his beliefs about student responsibility. Or, the process of reviewing this set of characteristics of successful STEM education innovators might help a faculty member realize that some of her teaching practices, while unusual in her department, are common among innovators across the country. I also reasoned that knowledge of these characteristics might enable faculty and other change agents to recognize others who have these characteristics, and who need a word of encouragement, or a new skill or contact in order to keep the faith, or, better yet, to really flourish. In other words, this kind of learning through reflection might help faculty become more accomplished and productive as reflective practitioners (Schön, 1983, 1995).

I also chose this topic for two other reasons. One is that I anticipated that two other participants in the workshop, Elaine Seymour and Robert Zemsky, would complement my focus on faculty as individuals with talks that focused on the organizational parameters that promote and constrain faculty efforts to improve how undergraduate students learn in STEM courses and programs.

The other reason was that I knew this paper would benefit from input from the workshop participants—many of whom I know have deep knowledge of STEM faculty innovators—about the adequacy of these characterizations and about how we might use them to inform action strategies that faculty and other change agents might use to good effect.⁵

The group about whom I am generalizing includes essentially all the STEM faculty innovators whose innovations I have studied during the last decade, plus many others with whom I have worked and held extended discussions about teaching and learning. (I list many of these individuals in Box A-4.) Almost all of these people are successfully promoting pedagogical improvements, and some are successfully promoting curricular improvements (the latter being more difficult in that curriculum tends to be a “canon” for which an entire discipline, or at least a department, shares responsibility). Moreover, most are also effecting change among their colleagues.

The process I used to formulate these common characteristics was to conduct an informal thematic analysis based on findings that appear in evaluation reports and case studies, and on points that some of these faculty made in conversation. I organized the emergent characteristics of STEM faculty innovators into topic areas pertaining to general personality features, attitudes and habits of interpersonal interaction, learning and teaching practices, processes for changing one’s own teaching practices, processes for fostering change in the teaching practices of communities, and the characteristic of “peripheral vision.” In some places, I provide references to work in the emerging “learning sciences” literature that presents many of these same characteristics as key to effective learning situations, and include some of the responses of those listed in Box A-4 to these themes and to earlier drafts of this paper.

Certain general personality features stand out as common to the successful STEM education faculty innovators whose work informs this paper. In

short, they are risk takers and very hard workers. They make commitments and stick with them to the end. Many are inspired by a sense of mission. And they are savvy and persistent about obtaining resources, including moral and material support from proactive administrators and external funding agencies. They take pride in doing a good job for their students and often for their departments, disciplines, and/or institutions as well. Most are not especially charismatic in their personal style.

Many people who are not engaged in STEM education innovation can, however, be described by the general characteristics listed above. Thus, while perhaps necessary, these general features certainly are not exclusive to successful STEM education innovators. That is, they are not defining characteristics. By contrast, I believe that unless a person has the characteristic attitudes and habits of interpersonal interaction discussed below, they will not be in this group of successful STEM education innovators. For brevity, I list these features as follows:

• Their identity as a scholar does not depend on placing themselves above other faculty members, academic staff, graduate students or undergraduates (Wilshire, 1990). Accordingly, they listen respectfully to students (“there are no dumb questions”), strive to build on students’ questions and ideas, and quickly recognize and are delighted by the occasional startling insight that a student presents. Mike Bleicher responded to this point by reminding me of the Biblical saying, “A wise man learns more from a fool than a fool learns from a wise man.”

• These faculty not only are comfortable admitting to students when they did not know something or made a mistake, but also value these situations as opportunities to engage their students in the kind of problem solving that is central to the scientific process. Most of these educators are at least as interested in teaching the process by which discoveries are made as the outcomes of those discoveries. In his response to a draft of this paper, John Wright affirmed this point and illustrated how he makes good use of mistakes:

• They view students not as “outsiders” but as less experienced potential peers. Accordingly, they design their courses and interact with students with a “we’re in this together” attitude. They make the effort to walk in students’ shoes by taking time to recall what it was like to not have concepts and skills that they, as experts, take for granted (Leamnson, 1999). Viewing students as novice potential members of their communities, they include them in the real talk and real work of their “communities of practice” (Lave and Wenger, 1991). They therefore do not view maintaining constant control of the classroom as a virtue, but rather seek out ways to give students at least some decision-making power.

• In contrast to faculty who consider teaching a burden (“teaching load”) to be accomplished in the least amount of time possible, these individuals feel genuinely excited about students and teaching.⁶ They enjoy seeing their students learn, and take a certain pride in their students’ accomplishments. As Josefina Arce put it, “We find pleasure in seeing our students learn—a pleasure similar to the one we feel when an experiment works well.” Jean-Pierre Bayard expressed this point in a slightly different way by writing that he, for example, really cares about students, and is motivated to earn his students’ respect.

• They view graduate teaching assistants as full members of the team and are eager for their input and feedback. They are willing to discuss their failures (and what they have learned from them) as well as their successes with colleagues who also are experimenting with innovation.⁷

I turn now to learning and teaching practices that are common to the successful STEM education faculty innovators whom I know. I would venture that this set of characteristics also constitutes a basic requirement for the people I describe here, but my hunch is that, compared to the characteristics listed above pertaining to attitudes and habits of interpersonal interaction, those listed below can more easily be developed with experience.

• Successful STEM education faculty innovators experience teaching as intellectually exciting—as another opportunity to learn that is no less engaging than the scholarship they

pursue in their STEM discipline. In other words, they make learning about learning a part of their scholarship.⁸ Many have explained that the challenges of teaching force them to put their research in a larger context, which often leads to new insights useful in their research. For example, John Wright finds that his best ideas—in his research as well as teaching—come from students.

• These innovators hold the conviction that good teaching demands ongoing creative effort, believe that it is important to “understand understanding” (Wiggins and McTighe, 1998), and take the time to learn about teaching. As one member of the CUSE workshop put it, they recognize that “self-reeducation takes years.” These individuals eschew recipes or quick fixes, and believe that everything one tries—whether successful or not—enhances their capacity to do better the next time (Stevens, 1988).

• They understand that learning depends on feeling puzzled, perturbed, and curious, and on tolerating ambiguity. They value cognitive dissonance as a precursor to the process of changing a person’s understanding (Jonassen and Land, 2000). Thus, in their own practice (as educators and researchers), they are quick to question the status quo and their own beliefs when they notice an inconsistency.

• They have very high expectations of students. They want them to go beyond “knowing that” to “knowing how” (Brown and Duguid, 2000) and “knowing why” (Hieggelke, personal communication). For that matter, they want students to get so engaged with their learning that “they try on for size” the identity of scientist, mathematician, or engineer (Seymour et al., 2002).

• They hold the conviction that if faculty will demand it, students will accept the challenge of becoming independent thinkers. Accordingly, they expect their students to push themselves to comprehend and use difficult ideas and acquire new skills. Persuaded that attempts to think for students or to control their thinking may actually interfere with their learning, they seek to provide course materials and an environment that pushes the students to do the thinking and, as Jerry Uhl put it, to “learn to learn.” As Ann Burgess commented, “Students learn more when you figure it out together than when you just tell them the answer!” Accordingly, they know that they and any other (graduate or undergraduate) course instructors must eschew the role of authoritative provider of answers and instead play the role of a guide—some-

one who has traveled these paths and remembers how it was the first time.⁹

• They believe that learning entails a constant moving back and forth between “practice” (trying things out, making things happen) and “beliefs” (theories about the nature of things and why things happen) (Lave and Wenger, 1991; Wertsch, 1993). Thus, they design their courses to provide students with “practice” by using hands-on problems and challenges. (Some of them refer to this approach as “learning on demand.”) They design their courses to provide learning processes that engage students in reflection through genuine dialogue with senior peers (e.g., fast and context-sensitive feedback from teachers and expert practitioners), other students (e.g., problem-solving in groups), and self-reflection (e.g., individual writing and problem solving).

• They believe that they should only ask students to learn things that there is good reason to believe will “matter” to the students. Thus, while fully expecting them to eventually master difficult and abstract concepts, they use “real stuff” in the curriculum, that is, open-ended problems, hard problems that they can relate to their everyday lives. They resist including material that not only the students will never use, but the faculty themselves would never use.¹⁰

Aligned with their efforts to engage students in genuine dialogue is their tendency to use assessment not to grade/judge (a process that closes opportunities to learn), but rather to figure out what their students are assuming and concerned about (a process that opens up opportunities to learn). They consciously use these “formative” assessment practices to help keep themselves aware that most of their students do not possess the mental models and habits that they had when they were students, let alone now that they are experts in their discipline. Evaluation findings on courses taught with this approach revealed that for many of the students in their classes, the learning, not the grade, was paramount (see, for example, Courter and Millar, 1995; Millar, Alexander, and Lewis, 1995; and Wright et al., 1998).

The STEM education faculty innovators with whom I have worked or with whom I have discussed teaching at length are similar in the ways they go about changing their own teaching practices.

• They are proactive and very pragmatic problem solvers. By this I mean, borrowing a concept from Covey (1990), that they work in their “circle of influence,” and while aware of problems in their “circle of concern,” they spend little if any time or emotional energy on these concerns (pp. 82–83).¹¹ Related to this point, they tend to be people who do not waste time casting blame (on the students, K–12 teachers or the K–12 system, or the system in place at their college or university) when they realize there are problems. Instead, they focus their energies on the business of doing what they can to address these problems.

• They take an experimental—that is, an intentional, systematic, and sustained—approach to solving their problems with teaching and learning. In particular, they transform their concerns into actionable “problems,” develop plans and strategies that they hypothesize will solve their problems, and have in mind from the beginning what outcomes they will accept as sufficient evidence of success. They constantly seek and reflectively use feedback information. That is, they gather information on how well their strategies work, analyze and reflect on this feedback—often mulling it over with colleagues (Lave and Wenger, 1991)—and adjust their strategies accordingly.

• They use this act-feedback-reflect-adjust and act cycle on an ongoing and cumulative basis, working step-by-step and bringing their entire store of past feedback information to bear on each new adjustment (Stevens, 1988). I illustrate and improve on this point by quoting Steve Ackerman’s response upon reading it in an earlier draft:

• They purposely engage with peer learning communities (2–10 people) and/or networks (up to 100) of people who are interacting about shared problems and pursuing similar action strategies (Hutchings, 1996; Shulman, 1993). That is, they develop new ideas and insights, and obtain new information, about teaching by interacting with “near peers” (Rogers, 1995) in local communities and in professional societies, or at least the education branches of those societies. In a few cases these peer groups are department based. Most often they are cross-departmental or cross-institutional. The latter usually are or were externally funded by a foundation (predominately the National Science Foundation, often the Howard Hughes Medical Institute, among others). In some cases, these cross-institutional networks consist of faculty who worked with the same professor or research groups as graduate students.¹² I saw no case in which the group consisted of a faculty developer and an individual STEM faculty member.

• Last, I would list an eventual turn to the larger community of educators as a characteristic that these STEM faculty share with regard to how they go about making change in their own courses. Once they are quite certain that they have accomplished something valuable as innovative teachers, most of them begin to notice that there is a body of research on learning, and that there is a big network of people in diverse disciplines who are involved in this business. At this point, they begin to participate in larger networks through meetings, email, and listservs, sharing citations and reading certain key pieces that make the rounds in their disciplinary communities (Shulman, 1993). In response to this point, Steve Ackerman wrote, “I would add that the large networks, in turn, play a role in seeking out the innovators.” And Diane Ebert-

May’s response to this same point beautifully illustrates Steve’s addition:

Essentially all of the STEM education faculty innovators about whom I generalize here are taking leadership roles in order to foster change in their departments, specific disciplines, and/or in STEM education overall. They are similar not only in their willingness to play these roles, but also with respect to their basic reason for doing so: they are committed to helping others benefit from the innovations in teaching and learning about which they have learned. Whether operating at very local or national levels, each of these leaders has left the lab and the classroom for at least some of their time in order to marshal and then manage the resources that leaders need to be successful. Each is guided by homegrown models of change (such as the “dipping the toe” model that guided Art Ellis for many years) that they may or may not have articulated explicitly.¹³ Each knows the importance of building networks with other innovator colleagues and with campus administrators, and many are very skilled at building communities focused on finding ways to make change. They know that unless they collaborate and build on one another’s efforts, it is not likely that their innovations will become the new status quo, that is, be institutionalized.¹⁴

With the help of participants at the CUSE workshop, I realized that these innovator-leaders might be organized into different groups, depending on their motivation:

• Some of these leaders’ primary motivation is to support and encourage other science education innovators, many of whom feel, and indeed are, marginalized and are vulnerable. Among these leaders are some who find that while their efforts are not appreciated in their own departments, this price is worth paying because of the influence they have on colleagues elsewhere across the nation. To be sure, well-known researchers, such as Eric Mazur (physics, Harvard University) and John Wright (chemistry, University of Wisconsin-Madison), are more likely to have this type of national influence, due to their visibility within their disciplines.

• Others are primarily motivated to help their colleagues by developing reliable new knowledge about how students in their discipline learn, and by providing tested innovative curriculum resources that can make others’ efforts to adapt these new methods much easier. Faculty at all types of institutions are taking this leadership path with noted success. Examples include John Jungck (biology, Beloit College), Lillian McDermott (physics, University of Washington), Curt Hieggelke (physical sciences, Joliet Junior College), Deborah Allen (biology, University of Delaware), and Dave Gosser (chemistry, City College of New York).

• Some are primarily motivated by the belief that improving the way we teach science is an excellent way to improve conditions in the world at large. Gretchen Kalonji (materials science, University of Washington) exemplifies individuals with this motivation. As she explained, in response to a draft of this paper:

• Yet other innovator-leaders believe they can most effectively help others by influencing policy at the national level. People in these positions generally need excellent reputations as researchers and administrators, in addition to their credentials as education innovators. Examples of these leaders include Judith Ramaley (assistant director,

Education and Human Resources Directorate, National Science Foundation), and Robert DeHaan (director of the Committee on Undergraduate Science Education, Center for Education, National Research Council).

• Last, there is a group of STEM education innovator-leaders who, as members of the National Academy of Sciences or as Nobel Laureates, wield enormous influence because their credibility as researchers and leaders within their disciplines and departments is beyond question. In this group are, for example, in mathematics—Hyman Bass (University of Michigan) and Richard Tapia (Rice University); in physics—Leon Lederman (University of Chicago) and Carl Wieman (University of Colorado, Boulder); in astronomy— Richard McCray (University of Colorado, Boulder); in chemistry—Bradley Moore (Ohio State University); and in biology—Bruce Alberts (president, the National Academy of Sciences).

Before concluding, I add a characteristic about which Elaine Seymour reminded me. Upon reading a draft of this document, she wrote:

I have noted this same characteristic repeatedly, and especially when listening to these educators (those in Box A-4 and many others) respond to my request that they recount why and how they got so involved in this education reform business. Each told a story of being, or at least observing the traditional classroom as, an “outsider.” Of note, almost to a person, when telling their stories, these individuals appeared to enjoy their “off-center” position, even though in some cases it placed them at some risk.

Upon reflection, I am coming to believe that just as their ability to discover patterns in the unfamiliar is a key to these faculty members’ success as scientists, a key to their success as education innovators is their ability to discover patterns in the all-too-familiar world of traditional classrooms and other higher education settings. This ability to see the familiar anew depends on the capacity to see, as Bateson (1994) puts it, with “peripheral vision,” to notice out of the corner of the eye something important about what is in front of you. It entails taking what anthropologists call the “participant observer” stance toward situations in our everyday world, and not taking these situations for granted. Standing outside the taken-for-granted mainstream, a person is better able to see things in a new light, to perceive the need for and possibilities for change, and then return to the mainstream to work on accomplishing those changes.

The capacity to use peripheral vision depends in part on one’s choice to do so. However, as Seymour suggested, people who are different, for some reason or other, are more likely to take this participant observer stance. Pursuing this point, Lillian McDermott noted that using peripheral vision is a great source of intellectual excitement, a fascinating way to learn—and in particular, an excellent way to learn about teaching.

This set of characteristics shared by STEM faculty who are largely successful at effecting change (at least in their own courses) begs the question of how these people fare within their departments and institutions. As noted above, they are wise enough to work in their sphere of influence and avoid wasting their energies on things that they cannot affect. To be sure, there are cases where these “things they cannot affect” affect them altogether too much, as when assistant professors are denied tenure because their departments did not recognize or sufficiently value their accomplishments in the scholarship of teaching. But, as a growing body of

evaluation and research findings indicate, there are many other cases where one can see that these innovators have become leaders whose spheres of influence have grown into and positively transformed features of their departments and institutions.

I venture that the extent to which STEM education innovators thrive and achieve their goals as educators depends not only on how well they manage those “things they can affect,” but also on the constraints posed, and opportunities afforded, by the institutional and cultural circumstances in which they are embedded. Several of the innovators who responded to the initial draft of this paper wanted to pursue questions about what circumstances pose constraints that are too risky and for whom and, more generally, about what lessons we can learn from innovators who successfully maneuver around the constraints and play into the opportunities. I do not attempt to address those questions here.

However, in their presentations at the CUSE workshop, Robert Zemsky and Elaine Seymour highlighted some of the constraints that innovators face. For example, Zemsky (2002) called our attention to the enduring resistance to change that is characteristic of universities, while Seymour (2002b) brought to light the daunting power of a cultural system, supported by myriad organizational practices, in which faculty and students tacitly agree to dispense with the formal educational tasks required of them by following paths of least resistance. They also pointed out a number of key opportunities that current circumstances afford STEM education innovators, such as market pressures for more effective and efficient learning that are experienced by, for example, medical and business schools (Zemsky); and a climate of trust among faculty, graduate students, and undergraduates in STEM classrooms (Seymour).

I would ask you to consider yet another factor—one that can just as powerfully “afford” as it can “constrain” the kind of change sought by these faculty innovators. This factor is the faculty themselves. Here is my argument. If universities are enduring institutions, it is not because they resist, but rather because they selectively embrace, change—following the best lessons learned and principles held by their typically “ungovernable” faculty. Innovation in STEM education is underway in the myriad decisions made and actions taken by ungovernable faculty who are learning from their students and one another, and who are encouraging one another in loose, cross-national, and inexorably expanding networks comprised of people like those featured here.

Moreover, it is important to note that

the successful STEM education innovators featured here include a number of our “radicalized seniors.”¹⁶ These people are important because faculty, however ungovernable, are inclined to learn from respected peers and to notice the values and actions of the most esteemed and altogether credible members of their disciplines.

In conclusion, I venture that, as STEM faculty innovators—“radicalized seniors” and many, many others— expand their spheres of influence, they are reshaping and redefining what it is that “the faculty” takes as acceptable norms for teaching STEM courses. And (coming full circle), to the degree that this paper helps restructure how we perceive these innovators among us, helps make them visible in new ways, it participates in this process of reshaping what we take for granted in STEM education.

Bateson, M.C. (1994). Peripheral visions: Learning along the way. New York: HarperCollins.

Brown, J.S., and Duguid, P. (2000). The social life of information. Boston: Harvard Business School Press.

Courter, S.S., and Millar, S.B. (1995). Final evaluation report of first-year design course 1994–95: Introduction to engineering. Madison, WI: University of Wisconsin-Madison, Learning through Evaluation, Adaptation, and Dissemination Center. Available: http://www.cae.wisc.edu/~lead/pages/internal.html [March 26, 2003].

Covey, S. (1990). The seven habits of highly successful people. New York: Simon & Schuster.

Hutchings, P. (1996). Building a new culture of teaching and learning. About Campus, 1, 4–8.

Jonassen, D.H., and Land, S.M. (2000). Theoretical foundations of learning environments. Hillside, NJ: Erlbaum.

Lave, J., and Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.

Leamnson, R. (1999). Thinking about teaching and learning: Developing habits of learning with first year college and university students. Sterling, VA: Stylus.

Millar, S.B. (2002, November). Effecting faculty change by starting with effective faculty: Characteristics of successful STEM education innovators. Paper commissioned for National Research Council’s workshop Criteria and Benchmarks for Increased Learning from Undergraduate STEM Instruction, Washington, DC.

Millar, S.B., Alexander, B.B., and Lewis, H.A. (1995). Final evaluation report on the pilot Wisconsin Emerging Scholars Program, 1993– 1994 (vol. 1). Madison, WI: University of Wisconsin-Madison, LEAD Center.

Palmer, P.J. (1992). Divided no more: A movement approach to educational reform. Change, 24(2), 10–17.

Palmer, P. J. (1993). Good talk about good teaching: Improving teaching through conversation and community. Change, 25(6), 8–13.

Palmer, P. J. (1998). The courage to teach: Exploring the inner landscape of a teacher’s life. San Francisco: Jossey-Bass.

Rogers, E.M. (1995). Diffusion of innovations (4th ed.). Westport, CT: Free Press.

Schön, D.A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.

Schön, D.A. (1995). The new scholarship requires a new epistemology: Knowing-in-action. Change, 27(6), 27–34.

Seymour, E. (2002a). Tracking the process of change in U.S. undergraduate education in science, mathematics, engineering, and technology. Science Education, 86, 79–105.

Seymour, E. (2002b, November). Barriers to change: Resistance is the normative mode. Talk presented at Criteria and Benchmarks for Increased Learning from Undergraduate STEM Instruction Workshop, Committee on Undergraduate Science Education, National Research Council, Washington, DC.

Seymour, E., Hunter, A-B., Laursen, S., and DeAntoni, T. (2002). Establishing the benefits of research experiences for undergraduates: First findings from a three-year study. Manuscript submitted for publication.

Shulman, L.S. (1993). Teaching as community property: Putting an end to pedagogical solitude. Change, 25(6), 6–7.

Stevens, E. (1988). Tinkering with teaching. Review of Higher Education, 12, 63–78.

Wertsch, J. (1993). Voices of the mind: A sociocultural approach to mediated action. Boston: Harvard University Press.

Wiggins, G., and McTighe, J. (1998). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.

Wilshire, B.W. (1990). The moral collapse of the university: Professionalism, purity, and alienation (SUNY Series in Philosophy of Education). Albany, NY: State University of New York Press.

Wright, J.C., Millar, S.B., Kosciuk, S.A., Penberthy, D.L., Williams, P.H., and Wampold, B.E. (1998). A novel strategy for assessing the effects of curriculum reform on student competence. Journal of Chemical Education, 75(8), 986–992.

Zemsky, R. (2002, November). On encouraging faculty to pursue instructional reform. Paper presented at Criteria and Benchmarks for Increased Learning from Undergraduate STEM Instruction Workshop, Committee on Undergraduate Science Education, National Research Council, Washington, DC.

Robert Zemsky

Graduate School of Education, University of Pennsylvania

When challenged to defend the staying power of their institutions, university presidents often invoke Clark Kerr’s (1987) observation:

Kerr was testifying to the enduring nature of the university—its ability to survive when challenged, to adopt when necessary. For defenders of the faith, nothing more is needed; however, for the naysayers among us, the image suggests something more than Kerr intended. What many see as enduring resilience, others perceive to be the academy’s early resistance to alteration and later its resistance to change.

Kerr’s observation also suggests the near impossibility of the assignment I have accepted: to explore “some of the options that university administrators— presidents, deans, department chairs— have at their disposal to encourage and support their faculty in instructional reform.” What Kerr and his appeal to the historic university make clear is that change in the academy is slow, probably imperceptible, and not likely to be the result of the strategies of individual presidents, deans, or department chairs.

For purposes of discussion, let me suggest three propositions that lend a practical perspective to this traditional tension between resilience and resistance. The first is simply that most universities—and almost all research universities—are presided over by faculty guilds. Membership is for life. Independence and autonomy are guaranteed, as long as the guild members respect the privileges and honor the obligations that membership confers, including the obligation not to meddle too deeply in the practices of one another. We teach largely as we were taught. When we experiment with new modes of instruction, we tend to do so quietly, not wanting to draw too much attention to ourselves. We tend to work alone, largely eschewing group projects. As in most guilds, acceptable practice is what everybody does—a kind of implicit regression to the mean—so that changes in curricula and instructional format require broad agreement that something, in fact, is broken and requires fixing.

My second proposition concerns the nature of the offices that presidents, deans, and department chairs occupy. We know they are administrators; we can hope they are—or eventually become—leaders. What we cannot expect them to be, however, are managers. They seldom command significant resources. Most of their funds, regardless of the size of their budgets, are spent before they can make a single decision or investment. Beyond their immediate staffs, they, like the pope, command no troops. Even the very words that frame this session reflect the problems nearly every president, dean, and department chair face: they cannot enforce change but merely explore “options…to encourage and support their faculty in instructional reform.”

At the University of Pennsylvania, Nichole Rowles is completing a dissertation (2003) that will update Cohen and March’s application of the garbage can model (Cohen, March, and Olsen, 1972) to describe decision making in the modern university. Rowles is documenting the extent to which presidents and their staffs, in particular, are attempting to adapt corporate models of decision making while their faculties cleave to the older, more established norms representative of guilds and garbage cans. The most striking differences involve the roles of strategy and data. In the corporate model, a strategy is what sport enthusiasts will recognize as a game plan: an envisioning of the job at hand, an enumeration of the resources available to achieve the desired goal, and a focusing on the tactics necessary to make one’s strategy operational. In all three modes, data play a critical role in defining possibilities, calculating risks, and measuring results. On the

other hand, most faculty think in different terms—not of strategies, but of strategic plans that, for the most part, are lists of things other people should be doing.

What is most striking, however, is the relative absence of calls for data from the faculty’s perspective. They enforce no culture of evidence for institutional decision making, despite the fact that most scholars spend their lives in pursuit of data and empirical observation. Instead, there are experiences and lessons learned—and, above all, principles derived from firmly held beliefs. In one institution in Rowles’ study, the faculty came to believe that athletes were being given preferential treatment and were being credited with higher grade point averages (GPAs) than they deserved. Despite the presence of a study conducted by that university’s office of institutional research, which documented that athletes’ GPAs were not being inflated by the suspect practice, the faculty overwhelmingly voted to outlaw the practice. When Rowles asked the head of the faculty senate why they had ignored the study, he responded simply, “You have to understand, it was not a matter of data but of principle.”

Hence, the problem faced by presidents, deans, and department chairs. Curricular reform, like all academic decisions, becomes more a matter of principle than of strategy—a matter of what is intrinsically right as broadly understood by those vested with responsibility for determining what is to be taught and how. It is a perspective that is too easily caricatured, as when members of the faculty are quoted as saying, “It’s not a matter of what students want but what they need.” As faculty, we have spent our lives learning what students need; we are collectively responsible for the knowledge base they must master, as well as exemplars of the role free and unfettered inquiry needs to play within every educational institution. When a president or dean speaks of the need to update the curriculum, incorporate more technology in the classroom, or recruit different kinds of teachers, the faculty not surprisingly ask: “Why? Who says what we must do?” And if the president or dean says, “Because we need to pay attention to the market in order to enroll the kinds of students we want to teach,” the natural response is: “But markets do not know what we know.”

Actually there is a better rejoinder which faculty are not likely to deliver, largely because, as a matter of principle, they seldom pay attention to the workings of the market for undergraduate education. What those of us who study those markets know is that there is no market for good teaching—and that is my third proposition. There is precious

little evidence that students choose where they enroll based on how faculty teach. Alverno College has learned that lesson all too well. Universally acclaimed for its pioneering curriculum and innovative ways of teaching, Alverno remains an institution that has proven far more successful at attracting academic visitors and foundation grants than students. Not surprisingly, the U.S. News and World Report rankings hardly bother to talk about teaching or curricula, choosing instead to focus on resources and reputations. Most presidents and deans know that the building of their institutions—and not so incidentally the building of their personal portfolios—depends fundamentally on increasing revenue and building reputations, neither of which rest on instructional reform.

Having defined the challenge, let me hasten to add that achieving instructional reform is not impossible, just very difficult. To understand what it might take to overcome the inertia of the guild, on the one hand, and the disinterest of the market in good teaching on the other, I want to focus on a few examples of success. They suggest the necessary conditions that an innovative president, dean, or department chair might exploit in pursuit of instructional reform.

The first is medical education leading to the M.D. Schools of medicine were among the first to experiment with and then broadly adopt self-paced and computer-assisted instruction. They have adapted a host of strategies to cope with an exploding knowledge base that can no longer be mastered, in the sense that basic anatomy can be mastered. And they have welcomed—some would say shamelessly embraced— nonphysician and non-Ph.D. instructors.

Why has medical education been able to achieve what most reformers of undergraduate education have only flirted with? There are several answers. In the first place, medical educators teach very smart, highly disciplined students for whom efficient learning is of enormous benefit. If self-paced, computer-assisted instruction promises that one can learn more and faster, then earnest students will believe it is worth a try. It is also the case that, in medical schools, teaching loads do not determine the size of the faculty group. In undergraduate education, learning efficiency all too often means the need for fewer faculty slots. And not to be overlooked is the fact that most medical schools have had and continue to have ample resources with which to experiment with new instructional technologies. Finally, there is a measurable

premium attached to good or at least successful teaching: better performance on board exams and better placements for the class as a whole in the competition for residencies. Outcome measures spur reform, particularly when those both within and beyond the academy sense the value and appropriateness of the measures themselves.

My second example derives from the growth of executive education programs at most of the nation’s leading business schools and their subsequent impact on the general business curriculum at both the graduate and undergraduate levels. In the early 1990s, when most of these programs were being launched, I asked the dean of one business school to account for the popularity of this particular form of education. Poised to build a hotel for his own new executive education program, he gave an answer that has stuck with me ever since.

The trend began as a kind of copycat phenomena, after Northwestern’s Kellogg School and then Penn’s Wharton School had launched their big, expensive initiatives. Soon, more and more schools followed suit; as they began to attract seasoned executives and managers to their “exec-ed” classrooms, the deans and faculty of these schools made a crucial discovery. Enrolled executives and managers began telling them that their traditional bread-and-butter business programs were in danger of precipitating out of the market. As one executive was reported to have said, in the past we did not so much care what you taught your undergraduates and M.B.A. students. What we expected from you was screening and certification, and figured that what happened in the classroom could do no harm. Now we are not so sure. Maybe what you are teaching really is standing in the way of the kinds of companies we are trying to build. The result across this set of select business schools was a rush to introduce educational experimentation and reform—a development that eventually came to energize business faculty across a wide spectrum of schools.

My last three examples are drawn from the world of undergraduate science and math instruction. In the 1980s, Bill Massy and I conducted a study of how departments make decisions about who teaches what (Zemsky, Massy, and Oedel, 1993). It was fundamentally an interview study, in which Bill and I spent upwards of an hour with every chair from a department that taught undergraduates at ten selective colleges and universities. What struck us was the degree to which physics departments seemed to be different; their chairs evidenced a passion for

teaching and a willingness to be judged by the quality of both their curricula and their teaching efforts.

Several years later I came across Jack Wilson’s experiments with Studio Physics (http://www.rpi.edu/dept/phys/education.html) at Rensselaer Polytechnic Institute (RPI) and was again reminded of the unique commitment to teaching evidenced across this discipline. What helped to make Studio Physics work at RPI was the presence of an established means of verifying the quality of this alternate form of instruction. All of the roughly 900 freshmen each year who take the basic introductory physics course sit for the same set of examinations, regardless of the section to which they were assigned. Studio Physics was able to win adherents because it could prove not only that it was more efficient in terms of the resources it consumed, but also that it produced as good or better results than teaching physics the old-fashioned way.

Collegiate mathematics instruction provides the same pair of lessons: that a disciplinary commitment is required, paired with a way to ensure the discipline that alternate ways of teaching produce measurable improvement. In the 1980s, the mathematician I knew best was Mort Lowengrub, then dean of arts and sciences at Indiana University. I asked him one night over dinner what accounted for his discipline’s interest in improving mathematics instruction. His answer, as I best remember, went something like this: “We are an endangered species, and we know it. We are not educating enough young people to sustain ourselves. We are in a down-ward spiral: fewer young people interested in mathematics translates into less demand for mathematics instruction, which then increases the probability that among the next student cohort there will be even less interest in mathematics—and so the cycle repeats itself. To break the cycle we need to be in the business of actively seeking converts.”

My last example derives from the experiences of undergraduate geology programs, particularly those offered at liberal arts colleges, over the last three decades. The oil and related energy crises of the 1970s resulted in a boom in geology majors, which in turn resulted in rapid increases in the sizes of geology departments. By the 1990s, however, the boom had gone bust, and the departments that had enjoyed rapid expansion suddenly found themselves teaching fewer students and warding off aggressive deans who wanted to shift their faculty billets elsewhere. At the time, I was engaged in a major study of the coherence of the collegiate curriculum, which examined the transcripts of graduates from more than 200 colleges and universities. Overall, we found what most observers expected: there was

little coherence, little course sequencing, little sense of an ordered progression through an established body of knowledge. The principal exceptions were the sciences, primarily physics, chemistry, and engineering.

Using the computer printouts of the statistical models that produced these results, the research team developed an elaborate parlor game in which we would look at the structure of courses and prerequisites and then try to guess the department and kind of institution to which the particular printout belonged. Although the output had been stripped of all departmental identifiers, we became very good at noticing the subtle differences among disciplines and between institutional type. But one profile stumped us nearly every time: those of departments of geology at liberal arts colleges, which for the most part we mistook for departments of English. When we followed up the statistical analysis with a set of interviews, I gingerly asked the first geology chair I encountered if he was surprised that the structure of his curriculum was indistinguishable from that of the English department. He replied, “Not at all. Actually we face the same challenge of convincing undergraduates that what we know and teach is intrinsically interesting—that it can be fun!” (Zemsky, 1989).

There are four basic lessons I would extract from these stories and observations, as a means of promoting the kind of discussion we need to have:

1. The first is that the guild itself must feel threatened before it is ready to change. No amount of talking or trying to explain that instructional reform is “good for you” is likely to substitute for the cumulative experience of witnessing the marginalization of what you consider to be important.

2. Curricular and instructional change, when it comes, is more likely to extend from the top of the institutional hierarchy down rather than bubble up from the bottom. What makes change so unlikely is the fact that it must come from those most advantaged by current arrangements and practices.

3. Curricular and instructional change is easier to promote when the students to be taught differently are not only smart and disciplined but also have a vested interest in the outcomes of the experiment.

4. Curricular change is inherently expensive, since the old ways of teaching will not be abandoned until the new means have fully demonstrated their staying power.

And, finally, some advice:

• To deans, in particular, don’t tilt at windmills—rhetoric is nice, but the frustration of unfulfilled promises in the end overwhelms.

• While it sounds good, the recruitment of star teachers is likely to have little impact.

• Changing the tenure rules only serves as a long-term strategy when the goal is curricular and instructional reform.

• Pick your targets, spend your money.

• Invest in strong programs.

• Experiment with breaking the rules—particularly those governing the time and mode of delivery.

• Look for external markets to develop and then harvest those which provide visibility plus funds for experimentation. Three markets are readily available:

• The teaching of science in primary and secondary schools.

• Making corporate groups scientifically literate.

• Building a public policy understanding of science.

Cohen, M.D., March, J.G., and Olsen, J.P. (1972). A garbage can model of organizational choice. Administrative Science Quarterly, 17(1).

Kerr, C. (1987). A critical age in the university world. European Journal of Education, 22(2), 183–193.

Rowles, N.S. (2003). Halfway in the can: An examination of data use in college and university decision making. Doctoral dissertation in preparation, School of Education, University of Pennsylvania.

Zemsky, R. (1989). Structure and choice. Curriculum Data Base Series. Washington, DC: Association of American Colleges and Universities.

Zemsky, R., Massy, W.F., and Oedel, P. (1993). On reversing the ratchet. Change, 25(3), 56–62.