9:15-9:45

Welcome

Stuart Elliott, Director, Board on Testing and Assessment

Judith Rizzo, Executive Director and CEO, James B. Hunt, Jr. Institute for Educational Leadership and Policy

Overview of Workshop Goals

Diana Pullin, Chair, Committee on Best Practices for State Assessment Systems

• Précis of previous workshop series and report

• Overview of the goals and plans for the current workshop series

• Introduction of the idea of innovative assessment

• Discussion of the current status of the common standards movement

(9:45-10:15) Overview of Current Assessment Practices

This session will provide a review of the current test-based accountability system, the goals and purposes it has developed to serve, and its strengths and limitations.

Presenter: Margaret Goertz, University of Pennsylvania

(10:15-10:45) Changes in Assessments and Assessment Systems Over the Past Decade

This session will review the ways assessments and approaches to assessment have changed over the past decade, including changes in item types, uses of local and interim assessments, and advancements in assessment of special populations.

Presenter: Scott Marion, National Center for Improvement of Educational Assessment (NCIEA)

(10:45-11:15) Synthesis of Key Ideas

Discussant: Joan Herman, National Center for Research on Evaluation, Standards, & Student Testing (CRESST)

(11:15-11:30) Focused Discussion

Moderators lead focused discussion with presenters and audience members.

11:30-12:15

Working Lunch

12:15-3:45

Session II. Changing the Status Quo

Moderators: Joan Herman, Rebecca Maynard

(12:15-1:00) Developing Standards That Lead to Better Instruction and Learning

This session will discuss ways to specify standards so that they (1) more accurately delineate the skills and knowledge to be learned and (2) can be more accurately and readily translated into instruction and assessment. Examples will be drawn from the draft common core standards.

Presenters: Joe Krajcik and Shawn Stevens, University of Michigan

skill and knowledge, with a particular focus on ways to ensure that assessments measure higher-order, critical thinking skills using a variety of item types.

Presenter: Mark Wilson, University of California, Berkeley

(1:45-2:30) Technical Challenges of Implementing Innovative Assessments

This session will explore the technical challenges associated with developing more innovative assessment tasks that measure challenging content and skills, tradeoffs associated with these kinds of assessments/tasks, and ways that the information gathered from innovative assessments might be used to support better decision making about students and instruction.

Presenter: Stephen Lazer, Educational Testing Service (ETS)

2:30-2:45

Break

2:45-3:15

Synthesis of Key Ideas

Discussant: Scott Marion, NCIEA

3:15-3:45

Focused Discussion

Moderators lead focused discussion with presenters and audience members.

3:45-5:00

Session III.A What Is the Status of Innovative Assessment?

Moderators: Diana Pullin, Mark Wilson

(3:45-4:30) Lessons from the Past and Current Efforts

This session will provide an overview of the experiences of pioneers in the area of innovative assessment, such as programs developed for Kentucky (KIRIS), Maryland (MSPAP), Vermont (Portfolio Assessment Program), and California (CLAS performance assessment) which are no longer in operation. Examples from currently operational assessment programs, international assessments, and in fields other than K-12 education will also be discussed.

Presenters: Brian Stecher and Laura Hamilton, RAND

5:00

Presentation by Lauren Resnick, Learning Research and Development Center, University of Pittsburgh

5:30

Adjourn Workshop

Reception

8:30-10:30

Session III.B What Is the Status of Innovative Assessment?

Moderators: Diana Pullin, Mark Wilson

(8:30-9:30) Panel Discussion: Political Considerations

This session will explore the political/practitioner perspective on the pioneer program discussed in Part A. Panelists representing several of the programs will address the following questions:

• What was the motivation for the assessment? Why was it considered? Who wanted it? Who wasn’t in favor of it?

• What was involved in getting the assessment adopted? What, if any, obstacles were encountered? How were they overcome?

• What was involved in developing the assessment? What, if any, obstacles were encountered? How were they overcome?

• What issues were encountered with implementation of the assessment? What, if any, obstacles were encountered? How were they overcome?

• What were primary reasons for the demise of the program?

Panelists: Steve Ferrara (MSPAP), Brian Gong (KIRIS), and Dirk Mattson (Minnesota)

(10:00-10:30) Focused Discussion

Moderators lead focused discussion with presenters and audience members.

10:30-10:45

Break

10:45-2:30

Session IV. Exploring the Opportunities

Moderators: Rebecca Maynard, Dirk Mattson

(10:45-11:30) What Opportunities Does the Common Standards Movement Offer for Improving Assessment?

This session will explore the opportunities the common standards movement might offer for moving to more innovative assessments that assess challenging content and also give more information to teachers and local decision makers. The presentation will address technical issues and potential benefits of collaboration across states, drawing on examples from the experiences of Vermont, New Hampshire, Rhode Island, and Maine (New England Common Assessment Program)

Presenter: Laurie Wise, HumRRO

11:30-12:30

Working Lunch

12:30-2:15

Session IV. Exploring the Opportunities

(12:30-1:15) Using Common Standards to Enable Cross-State Comparisons

This session will focus on the elements that would need to be in place in order for test results to be compared across states, including issues associated with adding state-specific items. The presentation will address the inferences that policy makers and test users might want to make and what is required to support each kind of inference.

Presenter: Ron Hambleton, University of Massachusetts, Amherst

(1:15-1:45) Synthesis of Key Ideas

Discussant: Rebecca Zwick, ETS and University of California, Santa Barbara

2:15-2:30

Break

2:30-4:00

Session V. Setting Research Priorities

Moderators: Diana Pullin, Scott Marion

(2:30-3:00) Research Priorities

The U.S. Department of Education has set aside $350 million for developing tests to measure common standards. This panel will listen to the workshop discussions and consider the implications for research. The presenter and discussants will address the following questions:

• Given the issues raised during the workshop, what are realistic priorities for research?

• What projects/efforts are most in need of research?

• How would you proportionally allocate the funding?

Presenter: Lorrie Shepard, University of Colorado

(3:00-3:45) Responses

Discussants: Laurie Wise, HumRRO; Joan Herman, CRESST; Rebecca Maynard, University of Pennsylvania

(3:45-4:15) Focused Discussion

Moderators lead focused discussion with presenters and audience members.

8:00

Working breakfast for committee

Discussion of agenda

8:30

Discussion of plans for the workshop and moderator assignments

8:45-9:00

Guests arrive, register

9:15-9:45

Opening Remarks

• Stuart Elliott, Director, Board on Testing and Assessment

• Judith Rizzo, Executive Director and CEO, James B. Hunt Jr. Institute for Educational Leadership and Policy

(9:30) Goals for the Workshop

• Diana Pullin, Boston College, Workshop Steering Committee Chair

• What are the different purposes of assessment and what assessment strategies best suit each purpose? How can the information provided by different forms of assessment be used to guide instruction, improve student learning, and evaluate effectiveness of teachers and schools for accountability purposes?

• What is involved in moving to a coherent system of assessment that incorporates multiple assessment types? What examples of such systems exist (nationally and internationally)?

• How can assessment of higher level thinking skills and deeper understandings be incorporated into a coherent system of assessments? What examples are currently being tried or considered?

(9:45-10:30) Coherent System of Assessments

Moderators:

Diana Pullin

Rebecca Maynard, University of Pennsylvania, Workshop Steering Committee

Presenter: Joan Herman, CRESST, Workshop Steering Committee

(10:30-10:45) Break

(10:45-11:30) Moderated Discussion: Policy Perspective

Panelists:

Roy Romer, College Board

Ed Roeber, Michigan State University

(11:30-12:00) Audience Discussion

12:00-1:00

Working Lunch

(1:00-2:00) Moderated Discussion: Research and Practice Perspective

Moderators:

Diana Pullin

Dirk Mattson, Minnesota Department of Education

Panelists:

Linda Darling-Hammond, Stanford

Tony Alpert, Oregon Department of Education

Marc Tucker, National Center for Education and the Economy

(2:00-2:30) Audience Discussion

2:30-2:45

Break

2:45-5:00

Session II. Issues to Consider with Innovative Assessments

Issues to Address

• Innovative assessments pose challenges with respect to item development, administration, scoring, equating, and other technical issues. What are these challenges and how can they be addressed? How can we ensure that scores obtained from innovative assessments are reliable and valid?

• What issues need to be considered in using innovative assessment approaches with special populations?

• How can we ensure that these students have the opportunity to learn the material/skills needed for assessments that measure deeper understandings and higher order thinking skills?

(2:45-3:15) Ensuring That Innovative Assessments Provide Reliable and Valid Information

Moderators:

Scott Marion, National Center for Improvement of Educational Assessment, Workshop Steering Committee

Mark Wilson, UC Berkeley, Workshop Steering Committee

(3:15-4:00) Moderated Discussion: Special Populations Perspectives

Panelists:

Robert Linquanti, West Ed

Martha Thurlow, National Center on Education Outcomes

(4:00-4:45) Moderated Discussion: Policy and Practice Perspective

Panelists:

Joe Willhoft, Washington Department of Education

Deborah Sigman, California Department of Education

(4:45-5:00) Audience Discussion

5:00-5:15

Closing Remarks for Day 1

Gene Wilhoit, Council of Chief State School Officers

Diana Pullin

5:15

End of Formal Agenda for Day 1

Reception

8:30-11:45

Session III. Strategies for Using Results from a Coherent System to Inform Instructional Decision Making

(8:30-9:15) Strategies for Using Assessment to Guide Instruction

Moderators:

Joan Herman

Mark Wilson

Presenter: Linda Darling-Hammond, Stanford

(9:15-10:15) Moderated Discussion: Researcher Perspective

Panelists:

Margaret Heritage, CRESST

Karin Hess, National Center for the Improvement of Educational Assessment

10:15-10:30

Break

(10:30-11:15) Moderated Discussion: Policy Maker, Practitioner Perspectives

Panelists:

Teri Siskind, South Carolina Department of Education

Peg Cagle, LA Unified School District, California Teachers Advisory Council

(11:15-11:45) Audience Discussion

11:45-12:45

Working Lunch

12:45-2:15

Session IV: The Role of Technology

(12:45-1:15) Technology and Innovative Assessment

Moderators:

Rebecca Maynard

Dirk Mattson

Presenter: Randy Bennett, ETS

(1:15-2:00) Policy maker, Practitioner Perspective

Panelists:

Wendy Pickett, Delaware Department of Education

Tony Alpert, Oregon Department of Education

(2:00-2:15) Audience Discussion

2:15-2:30

Break

2:30-4:00

Session V. Synthesis and Extension of Ideas

(2:30-3:15) Aggregating Information from Different Sources

Moderators:

Diana Pullin

Scott Marion

Presenter: Laurie Wise, HumRRO

The concept of “through-course” or “distributed summative assessment components” (e.g., assessment evidence collected over multiple points of time) has been discussed at several of the U.S. Department of Education-sponsored assessment hearings. This presentation will discuss how multiple types of assessment evidence can be used and aggregated to inform summative judgments. The presentation will address using aggregated information to evaluate students, teachers, and schools.

(3:15-4:00) Moderated Discussion with full steering committee

4:00

Closing Remarks

Diana Pullin