This chapter begins with a discussion of the types of literacy demands adults encounter in their daily lives and the reasons for assessing their literacy skills. We then give a brief overview of the National Adult Literacy Survey (NALS) and its successor, the National Assessment of Adult Literacy (NAAL). One of the chief uses of the results of the earlier survey was to determine needed programmatic interventions, many of which are offered through adult education systems. The chapter concludes with information about adult education services in this country.

In a rapidly changing world, literacy is an essential skill, one that helps people thrive individually, socially, and economically. Literacy is important for all aspects of an individual’s life, from handling personal affairs, to raising children, to engaging in the workforce, to participating in a democratic society.

In the home, individuals use their literacy skills for a wide range of activities, such as reading mail, paying bills, handling contracts and leases, and helping children with school matters. Regardless of one’s occupation, literacy skills are needed in a variety of work contexts—applying for a job, traveling to and from work, choosing a benefits package, and understanding and handling paychecks.

Adults also use their literacy skills to handle health and safety matters, such as reading and using product safety and nutrition labels, filling out

insurance forms, using tools and measurement devices, and reading dosage directions on prescription and over-the-counter medicines. Literacy skills are essential to keep family members healthy and safe and to assist elders as they make life-enhancing or life-changing decisions.

Literacy skills are also needed for adults to participate in a democratic society. Such activities as keeping apprised of local and national issues, understanding one’s rights and responsibilities, reading ballots, and voting all require literacy skills.

Although some of these tasks can be accomplished in languages other than English (e.g., newspapers in various languages provide information; bilingual ballots are available in most states), American society places a high priority on literacy skills in English. Literacy in English accrues significant benefits to individuals in this country, including the opportunity to attain U.S. citizenship, to work in a well-paying job, and to fully participate in the democratic process.

While literacy skills are important for individuals’ functioning and well-being, they are also critical for the social good and for a well-functioning society. Literacy skills have an impact on a nation’s economic status, the health and well-being of its citizens, the capabilities of its workforce and military, and its ability to compete in a global society. Deficiencies in literacy skills and mismatches between the skills of citizens and the needs of an economy can have serious repercussions.

Policy makers rely on assessments of literacy skills to evaluate both the extent of such mismatches and the need for services that provide basic literacy skills to adults. Such assessments can provide the foundation and impetus for policy interventions. The NALS, mandated by the Adult Education Amendments of 1988 (amendments to the Adult Education Act of 1966), was designed to provide such information. This legislation required the U.S. Department of Education to evaluate the nature and extent of literacy among adults.

In response, the National Center for Education Statistics (NCES) and the Office of Vocational and Adult Education planned a nationally representative household survey to assess the literacy skills of the adult population in the United States. The NALS was administered in 1992; it was revised and repeated in 2003 under a new name, the NAAL. A great deal of information is available about the two assessments on the NCES web site (http://www.nces.ed.gov). In this chapter, we briefly summarize the assessments to acquaint the reader with relevant background information, but interested readers are referred to the NCES web sites for further details about the assessments.

At the time that NALS was being designed, two prior large-scale assessments of subsets of the adult population in the United States had been conducted: the Young Adult Literacy Survey (YALS),¹ conducted in 1985, and the Department of Labor Survey of Workplace Literacy,² conducted in 1990. The group appointed to guide the development of NALS, called the Literacy Definition Committee, recommended adopting the same conceptual framework for NALS as was used for these two prior surveys. One reason for this decision was to enable comparisons of trends between NALS and the prior surveys. As a result of this decision, the methodologies and approaches used for the prior surveys were applied to NALS, and about half of the literacy tasks developed for the earlier surveys were readministered. In addition, much of the Technical Manual for NALS (Kirsch et al., 2001) covers procedures used for the earlier surveys.

The stated goals of NALS and NAAL are to describe the status and progress of literacy in the nation. Both assessments were comprised of the following: an introductory screening interview, a background questionnaire, and a literacy assessment. The total time for the interview is about 90 minutes. Scores are reported for three types of literacy—prose, document, and quantitative.

The definition of literacy that guided the development of NALS was the same as for the prior surveys (Kirsch et al., 2001, p. 70): “Literacy is the ability to use printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential.”

As noted earlier, NALS and NAAL are considered to be measures of functional literacy in English, in that they focus on how adults use printed and written information. The assessments are intended to evaluate literacy demands encountered in everyday settings at home, in the workplace, and in the community and to profile adults’ literacy skills in these contexts.

Each assessment task includes a stimulus, which is designed to simulate

materials adults frequently encounter, and a series of questions about the stimulus. The questions are presented before the stimulus to represent the way adults often approach a task in real life, in which functional reading is often driven by a need to know. Questions are open-ended, not multiple choice, again out of a desire to mimic realistic tasks. The tasks are categorized into the three types of literacy:

The 1992 assessment consisted of a total of 165 tasks, of which 82 were newly developed and 83 were reused from the prior surveys. Development of the new tasks was guided by a test blueprint that specified the characteristics of the items according to the structure of the stimulus (exposition, narrative, tables, graphs, forms, maps, etc.), the cognitive process required to respond to the question (locate, integrate, generate, add, subtract, etc.), the difficulty of the item, and the context from which the stimulus was drawn. The materials were drawn from six contexts of everyday life: home and family, health and safety, community and citizenship, consumer economics, work, and leisure and recreation. Additional information about item development can be found in Chapter 4 of the Technical Manual (Kirsch et al., 2001).

In 1992, there were some participants who had such limited literacy skills that they were able to complete only part of the assessment, and others who attempted to perform the literacy tasks they were given and were unsuccessful (Kirsch et al., 1993). In order to provide literacy tasks that even very low-literate adults could complete successfully, NAAL added a new component, the Adult Literacy Supplemental Assessment (ALSA), designed to assess skills in identifying numbers, letters, and comprehension of simple prose and documents. This component is interactive and uses a one-on-one format. The assessor presents each item to the respondent and

then asks questions orally. This format is designed to minimize the chance that low-functioning respondents fail to respond correctly to an item because they misunderstand the written directions or have difficulty with texts appearing outside their everyday environments.

NAAL’s literacy assessment begins with a relatively easy set of seven literacy tasks, referred to as the “core questions,” that are used to decide whether test takers should take the main NAAL assessment or the supplemental ALSA. Individuals who performed well on the core questions were assessed using the main NAAL, and individuals who performed poorly on the core questions were assessed with ALSA. NAAL consisted of 152 tasks, 54 prose tasks, 52 document tasks, and 64 quantitative tasks.

ALSA was designed to use highly familiar stimulus materials that are real and contextualized. The materials offer respondents the opportunity to see items as part of the product in which they appear in real life. For example, rather than just reading recipe directions on a printed page (e.g., for making soup), the respondent is asked to read directions that appear on the actual package (e.g., on the soup can).³ Similarly, respondents may be asked to point to letters or words on a product (e.g., point to the words “apple juice” on a juice can), identify the price of a food item on a grocery flyer, or interpret the directions on a medicine warning label. ALSA begins with simple word and letter identification tasks presented in context; proceeds to short, simple prose texts and documents (e.g., advertisements, road signs); and concludes with several tasks that involve location of information in documents that contain more distracting information, such as newspapers or more complicated advertisements. Oral directions and questions are provided in either English or Spanish by the interviewer. ALSA also allows participants to answer in either English or Spanish, although the stimulus materials themselves contain only English text.

Also new to the NAAL is a component designed to evaluate reading fluency. The fluency assessment uses speech recognition software to assess decoding, word recognition, and reading fluency. All participants in NAAL complete the fluency assessment after they have answered the background questionnaire, the core questions, and either the main NAAL or ALSA. Fluency tasks include lists of words and numbers as well as text passages to be read aloud by the respondent. Oral directions and questions are provided in English or Spanish, depending on the respondents’ preference, but the text itself appears only in English, and answers must be given in English. Only a bilingual interviewer, fully proficient in English and Spanish, is allowed to give directions or ask questions in Spanish when such support is

desired. The fluency tasks are administered at the end of the assessment, in an attempt to preserve comparability of the main NAAL with the 1992 assessment.

The 2003 NAAL also includes tasks that require application of literacy skills to the understanding of health-related materials and forms. Some health-related tasks were included on NALS, but the number of such tasks was increased for NAAL to allow reporting of a “health literacy” score.⁴ There are 28 health-related tasks as well as 10 health-related background questions.

Additional information about the new features included on NAAL can be found at http://www.nces.ed.gov/NAAL/design/about02.asp#C.

NALS and NAAL are designed to provide reliable group-level estimates of literacy skills. The assessment is not designed to provide reliable scores for individuals (although statistical estimates of individuals’ performance on the assessment can be derived and used for research purposes). Because individual scores are not reported, the assessments can utilize a matrix sampling approach to the assignment of test questions to individuals. The approach involves splitting a large set of tasks into smaller sets, or blocks. A similar design has long been used for the National Assessment of Educational Progress; it provides a means to minimize the number of test questions an individual must take and is efficient when the goal of an assessment is to provide reliable estimates of group-level performance.

With this approach, literacy tasks are assigned to blocks that can be completed in about 15 minutes, and these blocks are compiled into booklets, so that each block appears in each position (first, middle, and last) and each block is paired with every other block. Blocks of simulation tasks are assembled into booklets, each of which could be completed in about 45 minutes, although there were no time constraints placed on the participants for completing the tasks. Additional information about this can be found at http://www.nces.ed.gov/naal/design/design92.asp#design.

One chief aim of the 2003 assessment is to measure the trend from the previous assessment. NAAL consists of 13 blocks of tasks, 6 that were

repeated from 1992 and 7 newly developed for the 2003 assessment. The new blocks were based on the 1992 frameworks and were designed to be similar to the replaced blocks of items with regard to skills measured, content, and item statistics. After collection of the 2003 data, statistical linking procedures were used to place NALS and NAAL scores on the same scale.

The Department of Education plans to make NAAL data publicly available to researchers and others interested in conducting studies on the results. The results can be grouped according to the score ranges used for the old 1992 performance levels or the new 2003 levels. Trend comparisons will be possible based either on the 1992 levels or on the new levels adopted for NAAL.

For both assessments, data were collected via a household survey of a stratified random sample of adults age 16 and older. Additional samples were obtained in specific states in order to provide state-level results; this portion of the assessment was referred to as the State Adult Literacy Survey. In 1992, 12 states participated (California, Florida, Illinois, Indiana, Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington); in 2003, 6 states participated (Kentucky, Maryland, Massachusetts, Missouri, New York, and Oklahoma). For both assessments, an additional sample was obtained of individuals incarcerated in federal and state prisons.

Approximately 26,000 individuals age 16 and older participated in the 1992 NALS. Of these, 9.8 percent reported speaking a language other than English at home. The overall number included 13,600 selected as part of the national sample as well as about 12,000 (1,000 per state) selected through the State Adult Literacy Survey. The survey also included 1,147 inmates from 87 state and federal prisons who were selected to represent the inmate population in the United States. Their participation helped to provide better estimates of the literacy levels of the total population and made it possible to report on the literacy proficiencies of this important segment of society. See http://www./nces.ed.gov/naal/design/about92.asp for additional details about the sampling.

Sampling procedures were similar for the 2003 NAAL. The nationally representative sample of 19,714 adults included 18,541 participants living in households, about 6,500 of whom were selected from six states (approximately 1,000 per state). An additional 1,173 participants were selected from adults living in state or federal prisons.

The assessment is administered in the home by trained interviewers. The first step in the process is to administer the “screener,” a set of questions used to determine the number of eligible respondents (e.g., age 16 or older) in the household. The screening process involves recording the names, relationships, sex, age, and race/ethnicity of all household members at the selected household. Some bilingual interviewers are trained to administer the screener in either English or Spanish. Interviewers can also ask a household member or a family friend to translate the screening questions into Spanish or other languages, although there is no check on the accuracy of the translation.

To select respondents, interviewers list the names and ages (in descending age order) of all eligible household members and then refer to a sampling table. In households with three or fewer eligible household members, one is randomly selected for the interview; in households with four or more eligible persons, two are selected. Selected participants receive an incentive payment for participating in the assessment ($20 in 1992, $30 in 2003). See http://www.nces.ed.gov/naal/design/data92.asp#collection for additional details about the data collection process.

After completion of the screener and selection of participants, the background questionnaire is administered. The background questionnaire can also be administered in Spanish or English. If the participant does not speak Spanish and is not proficient enough in English to understand and respond to the interviewer, the interaction is terminated and the person is classified as having a language problem. These procedures were changed slightly in 2003 in an effort to obtain literacy information from as many participants as possible. In 2003, participants who did not speak Spanish or who were not sufficiently proficient in English to respond to the entire background questionnaire, were allowed to skip one or more of the background questions that they were not able to answer and move to the literacy assessment. Interviewers recorded each skipped background question as “don’t know” and documented the reason for skipping the question.

The background questionnaire collects general background information about language experience (e.g., language spoken at home), educational background and experiences, political and social participation, labor force participation, employment and earnings experiences, health, literacy activities and practices, and demographic information. The background questions are read to the respondent by an interviewer who then marks the answers on an answer sheet. The actual background questionnaire used in 1992 can be found in Appendix G of the Technical Report and Data File Users Manual for the 1992 National Adult Literacy Survey (http://www./nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2001457). The background questionnaire used in 2003 is available from NCES.

After administration of the background questionnaire, the interviewer proceeds to the literacy assessment. The literacy tasks (a stimulus and a set of questions) on the assessment are presented one at a time. There are no time limits for responding. Participants continue until they reach the final item or they choose to stop. When a participant stops before finishing the set of tasks, the reason for stopping is recorded.

There are many uses for the results of adult literacy assessments. Over the past decade, NALS results have been used by a wide variety of audiences, including those concerned about the status of the workforce in this country and for evaluating the need for training programs, officials in the public health sector who are concerned about the extent to which adults make wise and informed health and safety decisions, researchers studying the relationships between literacy and participation in civic activities and political processes, and experts in family literacy evaluating the extent to which parents are able to participate in their children’s educational process. NALS results have been widely cited in the research literature and continue to be used to argue for needed resources for adult education services.

Although NALS results have been used in a variety of ways, one of the chief uses over the past decade has been to determine the extent to which adult basic education services are available to meet the needs of adults with low levels of literacy. In the next section, we provide a brief overview of the adult education system in this country to provide context for discussions that appear in later sections of the report and several of the decisions the committee made about performance levels for NAAL.

Ideally, literacy skills are acquired as people progress through the K-12 education system in this country. However, this system does not always work for all who pass through it, and many who have immigrated to the United States have never participated in it. In addition, increasing societal and workplace demands may exceed what is taught in school, creating situations in which the skills of the populace are not aligned with the needs of the nation.

The adult education system is intended to remedy basic skill deficiencies and mismatches between skill requirements and adults’ proficiencies and to provide developmental English language and literacy services to immigrants and refugees not yet proficient in English. The adult education system is, for the most part, guided legislatively by the Adult Education Act of 1966 and the Workforce Investment Act of 1998, Title II, Adult

Education and Family Literacy Act. Through a combination of federal, state, and local funding, the adult education system sponsors adult basic education (ABE) programs through which individuals can improve their literacy skills and prepare for the general educational development (GED) assessment, the test taken to acquire the equivalent of a high school diploma. According to recent statistics available from the Office of Vocational and Adult Education, 2,891,895 individuals age 16 and older were enrolled in adult education in 2000 (http://www.ed.gov/about/offices/list/ovae/pi/AdultEd/2000age.html). In 2001, over 600,000 adults were issued GED credentials after passing the test (GED Testing Service, 2004).

The adult education system also provides courses in English for speakers of other languages (referred to as ESOL programs), designed to assist immigrants to learn and function in English. Of the close to 3 million adults enrolled in ABE programs in 1999, 1 million were enrolled as ESOL students (National Center for Education Statistics, 2003). ESOL programs serve people with a wide array of literacy skills in English and in their native language. For example, immigrants to the United States may not be literate in English, but they may have strong literacy skills in another language, skills that are likely to transfer to English literacy once their English skills improve.

Other immigrants may struggle with literacy both in the native language and in English, such as those who had only an elementary school education or less in their native country. Acquiring an education is a substantial challenge for this group as a consequence of their weak foundation in literacy (in any language) and lack of the background knowledge that others obtain through formal schooling. These individuals, most of whom are immigrants from Mexico and Central America and refugees from Haiti, Laos, and Africa, will need adult education services that are quite different from those offered to their more educated counterparts. Both groups, highly literate and low-literate English learners, may need services that focus on oral communication skills along with literacy. Newcomers may need skills and strategies associated with acculturation to the United States as well. In that respect, the services that adult language learners require are quite different from those provided for adult learners who grew up speaking English.

The relative literacy skills of a third group, often referred to as “generation 1.5,” may need to be considered as well.⁵ These individuals are generally bilingual young adults who were born elsewhere but partially educated in the United States or who were born and educated in the United States but

who grew up in linguistically isolated communities in which a language other than English was spoken at home. Their conversational skills in both the home language and in English may be quite strong, but some generation 1.5 adults are still learning formal, written English, while others have learned a nonstandard form of English that carries over to their writing.⁶ The literacy education needs of these adults are varied. Some of them may benefit from traditional ESOL programs. Others may benefit more from mainstream education with an emphasis on identifying and correcting the errors associated with the use of nonstandard English. In many ways, this second group of generation 1.5 adults has similar characteristics to nativeborn young adults who grew up speaking English but are academically unprepared.

While most individuals enroll in adult education programs voluntarily, some are encouraged or required to do so by their employers. Still others are required to attend classes in order to receive funds or services that are contingent on participation in education or training, including ABE/ESOL programs. Some individuals receiving income support through Temporary Assistance to Needy Families (commonly known as welfare) and displaced workers receiving stipends under the North American Free Trade Agreement fall into this category.

Increasingly, adult education programs are offered in the nation’s prisons. Recent statistics (1999) indicate that 25 percent of jail jurisdictions offer an ABE program (National Institute for Literacy, 2002). According to the Bureau of Justice Statistics (2000), 80 percent of state prisons, nearly all federal prisons, about 70 percent of private prisons, and over half of jails offered high school level classes, which generally focus on GED preparation. In some states, completion of the GED preparation program and passing the GED result in early release from prison (e.g., Nevada, personal communication with Sanford Marks; and Indiana, Lawrence et al., 2002). Studies of reincarceration rates for adults in Virginia, Ohio, Minnesota, and Maryland suggest that enrolling in literacy and ABE education programs lower the likelihood of recidivism (Wedgeworth, 2003). Research also indicates that only 7 to 10 percent of inmates who qualify for literacy education programs actually take advantage of the opportunity (Langley, 1999), despite the fact that 70 percent of incarcerated adults are estimated to read below the fourth-grade level (Haigler et al., 1994).

An important aspect of adult education programs is an accountability system, known as the National Reporting System (NRS), that was implemented through Title II of the Workforce Investment Act of 1998 (Public Law 105-220).⁷ As part of the NRS, adult education participants are administered an assessment (state’s choice) upon entry into an adult education program and take a posttest after a period of instruction as determined by each state. Results for the assessment are reported in terms of the NRS educational functioning levels, a set of six levels that include brief descriptions of the skills that students are expected to demonstrate in the areas of reading and writing, numeracy, and functional and workplace skills. There are separate level descriptions for ABE and ESOL (see Tables 2-1 and 2-2). States report the percentage of adult education participants who move from one level to the next as measured by pre-posttest gains in a program year. The Workforce Investment Act includes a system of incentives provided to states based on their students’ test performance as well as on other criteria.

The lowest NRS level (i.e., beginning literacy) ranges from an individual having “no or minimal reading and writing skills” to being able to “recognize, read, and write letters and numbers, but has a limited understanding of connected prose.” The second NRS level (i.e., beginning basic education) is described as equating to reading grade levels of 2.0-3.9 on standardized tests and includes individuals who can “read simple material on familiar subjects and comprehend simple compound sentences in single or linked paragraphs containing a familiar vocabulary.” The third NRS level (i.e., low intermediate basic education) is described as equating to reading grade levels 4.0-5.9 on standardized reading tests and includes individuals who can “read text on familiar subjects that have a simple and clear underlying structure (e.g., clear main idea, chronological order)” and “can interpret actions required in specific written directions.” The fourth NRS level (i.e., high intermediate basic education) is described as equating to reading grade levels 6.0-8.9 on standardized tests and including readers who can “read simple descriptions and narratives on familiar subjects or from which new vocabulary can be determined by context.” They can also “make some minimal inferences about familiar texts and compare and contrast information from such texts, but not consistently.”

The adult education field relies on information from assessments such as NALS/NAAL to project program needs. Most individuals served by adult education programs have skills in the ranges described by the current

NALS Levels 1 and 2.⁸ Knowing about the skill levels of those likely to enroll in adult education programs assists policy makers and program coordinators in determining where to focus funding initiatives and in designing instruction programs.⁹ The greatest interest and concern among adult educators relates to information about potential students at the lowest four NRS levels: beginning literacy, beginning basic education, low intermediate basic education, and high intermediate basic education. Adult students at the very lowest level may need one-on-one instruction and a good deal of guidance. Adult students at the next two levels can function a bit more independently but still are not ready to take classes focusing on the GED high school equivalency diploma. Adult students at the fourth level read at levels comparable to middle school students and are likely to be ready to pursue GED instruction.

The committee was aware of the fact that the adult education community hoped to be able to map the performance levels adopted for NAAL onto the NRS levels. Although NAAL did not appear to have been designed for this purpose, we kept this desire in mind as we proceeded with our work and, as spelled out in the report, attempted to develop performance levels that would meet the needs of this community.