Cynthia Baur, U.S. Department of Health and Human Services
Beth Beuhlmann, U.S. Chamber of Commerce, Workforce Preparation
Richard Colvin, Hechinger Institute
Leslie Farr, Ohio State University
Milton Goldberg, Education Commission of the States (ECS)
Richard Long, International Reading Association
Christopher Mazzeo, National Governors Association
Gemma Santos, Miami Dade Public Schools
Tony Sarmiento, Senior Service America, Inc.
Linda Taylor, Comprehensive Adult Student Assessment System
Robert Wedgeworth, Proliteracy Worldwide
Joan Auchter, GED Testing Service/American Council on Education
Justin Baer, American Institutes for Research (AIR)
Amy Baide, Department of Homeland Security
Sandra Baxter, The National Institute for Literacy (NIFL)
Jaleh Behroozi, The National Institute for Literacy (NIFL)
Martha Berlin, Westat
Peggy Carr, Department of Education, National Center for Education Statistics (NCES)
June Crawford, The National Institute for Literacy (NIFL)
Elizabeth Greenberg, American Institutes for Research (AIR)
Ricardo Hernandez, Office of Vocational and Adult Education (OVAE)
Shannon Holmes, U.S. Conference of Mayors
Eugene Johnson, American Institutes for Research (AIR)
Linda Johnston Lloyd, Health Resources & Services Administration (HRSA)
Michael Jones, Office of Vocational and Adult Education (OVAE)
Cheryl Keenan, Office of Vocational and Adult Education (OVAE)
Irwin Kirsch, Educational Testing Service (ETS)
Andy Kolstad, National Center for Education Statistics (NCES)
Mark Kutner, American Institutes for Research (AIR)
Mariann Lemke, National Center for Education Statistics (NCES)
Anne Lewis, freelance journalist
Lennox McLendon, National Adult Education Professional Development Consortium
Wendy Mettger, Mettger Communications
Leyla Mohadjer, Westat
Gerri Ratliff, Department of Homeland Security
Lyn Schaefer, GED Testing Service
Peggy Seufert, American Institutes for Research (AIR)
Sondra Stein, The National Institute for Literacy (NIFL)
Lynn Thai, Department of Homeland Security
Peter Waite, Proliteracy America
Dan Wagner, National Center on Adult Literacy
Maria White, Department of Health and Human Services
Sheida White, National Center for Education Statistics (NCES)
Kentaro Yamamoto, Educational Testing Service (ETS)
Bob Bickerton, Massachusetts
Steve Coffman, Missouri
Donna Cornelius, Massachusetts
Cheryl King, Kentucky
Tom Orvino, New York
Ann Serino, Massachusetts
Reecie Stagnolia, Kentucky
Linda Young, Oklahoma
In what ways did you use the results from the 1992 NALS? What were the strengths and weaknesses of these performance levels? To what extent did these performance levels provide you with the information that you needed?
NAAL measures skills in the areas of prose, document, and quantitative literacy. To what extent is it useful and informative to have different performance level descriptions for each area? Are results from the three areas of literacy used differently? If so, how?
The attachment presents three alternative versions of performance-level descriptions for the prose literacy scale. Sample 1 is simply a reformatted version of the existing performance-level descriptions with 5 levels. Sample 2 is a 4-level model, and Sample 3 is a 3-level model. Please comment on how many levels are needed. What types of decisions are made at the various levels? What are the critical distinctions that need to be made?
Level Labels: The three samples present different labels for the levels. Sample 1 uses numbers (Col. 2). Samples 2 and 3 use phrases as labels (Col. 2). In addition, Sample 3 presents a narrative description of the label (Col. 3). Please comment on these alternative labels. What types of labels are useful and informative? Feel free to make suggestions for alternative labels.
Level Descriptions: The three samples present different ways of describing the skills represented by the performance level. Sample 1 describes the tasks associated with the level (Col. 3). Sample 2 describes what an average respondent who scores at this level should be able to do (Col. 3). Sample 3 (Col. 4) describes what the average respondent who scores at this level is able to do and not able to do in probabilistic terms (i.e., likely, not likely). Please comment on these alternative ways of describing the skills associated with the levels. What types of descriptions are useful and informative? Feel free to make suggestions for alternative descriptions.
Sample Tasks: The three samples present different ways of exemplifying the tasks respondents who score at the level should be able to do. Samples 1 and 2 (Col. 4) are similar and provide examples drawn from actual assessment. Sample 3 (Col. 5) attempts to generalize from assessment tasks to real world tasks. Please comment on the extent to which these exemplifications are useful and informative.
Relationships Between Prose Scores and Background Data: Samples 2 and 3 present the relationships between NAAL scores and key real-world factors as measured on the background questionnaire. Sample 2 (Col. 5) uses societal factors (income, education, voting) and Sample 3 (Col. 6) uses reading related factors. (Please be aware that the percentages in
Sample 1: Five-Level Model, Based on Current PLDs for Prose Literacy
|
(Col. 1) Level |
(Col. 2) Label |
(Col. 3) Description of tasks |
(Col. 4) Sample NAAL tasks associated with the level |
|
I |
Level 1 |
Most of the tasks in this level require the reader to read relatively short text to locate a single piece of information which is identical to or synonymous with the information given in the question or directive. If plausible but incorrect information is present in the text, it tends not to be located near the correct information. |
|
|
II |
Level 2 |
Some tasks in this level require readers to locate a single piece of information in the text; however, several distractors or plausible but incorrect pieces of information may be present, or low-level inference may be required. Other tasks require the reader to integrate two or more pieces of information or to compare and contrast easily identifiable information based on a criterion provided in the question or directive. |
|
|
III |
Level 3 |
Tasks in this level tend to require readers to make literal or synonymous matches between the text and information given in the task, or to make matches that require low-level inferences. Other tasks ask readers to integrate information from dense or lengthy text that contains no organizational aids |
|
|
|
|
such as headings. Readers may also be asked to generate a response based on information that can be easily identified in the text. Distracting information is present, but is not located near the correct information. |
|
|
IV |
Level 4 |
These tasks require readers to perform multiple-feature matches and to integrate or synthesize information from complex or lengthy passages. More complex inferences are needed to perform successfully. Conditional information is frequently present in tasks in this level and must be taken into consideration by the reader. |
|
|
V |
Level 5 |
Some tasks in this level require the reader to search for information in dense text which contains a number of plausible distractors. Others ask readers to make high-level inferences or use specialized background knowledge. Some tasks ask readers to contrast complex information. |
|
Sample 2: Four-Level Model for Prose Literacy
|
(Col. 1) Level |
(Col. 2) Label |
(Col. 3) The average respondent who scores at this level should be able to: |
(Col. 4) Sample of NAAL tasks the average respondent should be able to: |
(Col. 5) Relationships with societal factors |
|
I |
Below Basic |
|
|
|
|
II |
Basic |
|
|
|
Sample 3: Three-Level Model for Prose Literacy
|
(Col. 1) Level |
(Col. 2) Label |
(Col. 3) Description of label for individuals who score at this level: |
(Col. 4) The average respondent who scores at this level: |
(Col. 5) Sample real-world tasks the average respondent at this level should be able to do: |
(Col. 6) Relationship to reading-oriented factors |
|
I |
Minimally Literate in English |
Are not able to independently handle most of the tasks of daily living that require literacy skills in English. |
|
|
|
|
II |
Somewhat Literate in English |
Should be able to independently handle some of the tasks of daily living that require literacy skills in English. |
|
|
|