The National Adult Literacy Survey (NALS) used a balanced incomplete block (BIB) spiraling design for assigning items to test booklets and booklets to test takers, much like what is done for NAEP. There are 26 NALS test booklets. Each booklet contains 3 blocks of items, and test takers are given about 15 minutes per block. Each block of items appears in three different test booklets; each block appears with every other block at least once.

In 1992, all test takers were given a set of six “core” questions to familiarize them with the examination and testing procedures. The core questions were relatively easy and consisted of two questions per literacy area (prose, document, and quantitative).

NALS included some questions/tasks that had been developed for the 1985 Young Adult Literacy Survey (n = 85) and some newly developed questions/tasks (n = 81). Table B-1 shows the distribution of the tasks across the three literacy areas.

For this analysis, six booklets were selected, and the responses of the household survey participants (n = 24,944) were studied. This analysis replicated some of the procedures that had been used for prior dimensionality analyses (e.g., Rock and Yamamoto, 2001) but selected different blocks of items. Table B-2 shows the booklets and blocks of items included in the analysis along with the distribution of the tasks across the three literacy areas and the number of test takers who received each of the tasks.

Using LISREL, six confirmatory factor analyses were run in which a

three-factor model was specified. Analyses were run separately for each test booklet. Because the six core items evidenced limited variability, the analyses were repeated with these six items removed. Tables B-3 and B-4 present the results with the core items included and excluded, respectively.

Correlations between the literacy scales were quite high. When the core items were included in the analyses, correlations between the prose and document scales ranged from .89 to .94 for the six booklets, from .77 to .97 for the document and quantitative scales, and from .80 to .97 for the prose and quantitative scales. When the core items were removed from the analyses, correlations between the prose and document scales ranged from .86 to .94 for the five booklets (data matrix for Booklet 1 was not positive definite), from .75 to .95 for the document and quantitative scales, and from .79 to .97 for the prose and quantitative scales.

Model fit was evaluated using the root mean square error of approximation (RMSEA). Fit tended to decrease slightly when the core items were

removed. These results suggest that a three-factor model provided acceptable fit to the data.

Another set of statistical analyses addressed questions about the relationships between the prose, document, and quantitative scores and an array of literacy outcomes. First, were the dimensions of literacy associated differentially with social and economic characteristics? For example, was prose more highly associated with outcome x than with outcome y, while

quantitative was more highly associated with outcome y than with outcome x? If so, there would be empirical support for use of each separate dimension to guide adult education policy and the activities of adult educators. If not, either the assessments do not measure the dimensions independently, or there is little practical significance to the distinctions among them.

Second, were some dimensions of literacy more highly related to the social and economic characteristics than others? For example, is prose the most important type of literacy, or are document and quantitative equally important? The answer to the second question is both simpler and more important if the answer to the first is that the dimensions of literacy are not associated differentially with their correlates. That is, if one weighted combination of the prose, document, and quantitative scores adequately describes the relationship of measured literacy to the several possible correlates, the weights of prose, document, and quantitative become more instructive.

These analyses were based upon the national and state household samples from the 1992 NALS.¹ The total sample size was 25,987. The possible literacy correlates used in the analysis were similar to those used in the panel’s search for break points in the distributions of literacy scores (as described in Chapter 4):

• years of school completed,

• immigration within the last five years,

• reporting at least one health impairment,

• reporting a health problem that limits work,

• reporting not reading well,

• voting within the last five years,

• being in the labor force,

• weekly earnings (log),

• reporting never reading newspaper,

• reporting reading no books,

• working in an occupation with high formal training requirements (professional, technical, managerial, nonretail sales),

• working in an occupation with low formal training requirements (skilled worker, semi-skilled worker, labor, service work, farm work),

• using Food Stamps within the past year,

• having interest income in the past year,

• reporting use of reading on the job,

• reporting help needed with written material, and

• reporting use of math on the job.

The prose, document, and quantitative scores used were the first set of plausible values in the public release of the data (http://www.nces.ed.gov/naal/analysis/resources.asp).

The analyses were based on a multiple-indicator, multiple-cause (MIMIC) model (Hauser and Goldberger 1971; Joreskog and Goldberger 1975), estimated by maximum likelihood. Rather than estimating separate regressions, one for each of the correlates of prose, document, and quantitative, the model posits that there is one linear composite of prose, document, and quantitative—much like the predicted values in a single regression equation—and that the statistical effects of prose, document, and quantitative on the correlates are completely described by the relationships of the correlates with the composite variable. An equivalent way of describing the model is that the statistical effects of prose, document, and quantitative on each of the correlates are in the same proportion.

This model was estimated in the total household sample of NALS and in groups defined by race-ethnicity (black, Hispanic), gender, and age (16-29, 20-44, 45-59, and 60 and older). The constrained model of the effects of prose, document, and quantitative on literacy correlates never fits statistically. This is to be expected because the sample is so large that any deviation from the model, no matter how trivial in substance, is statistically reliable. However, the actual deviations of the data from the constraints of the model are neither large nor numerous.² Typical deviations from the model are that (a) using mathematics on the job is more highly correlated with quantitative literacy, (b) voting within the past five years is less highly correlated with document literacy, and (c) earnings are more highly correlated with quantitative literacy. Nevertheless the model provides a useful framework for assessing the relative importance of prose, document, and quantitative.

As shown in summary in Table B-5, estimates from the constrained model are roughly similar across all of the groups. For fully constrained models, the left hand panel shows the effects of each of the dimensions of literacy. Prose, document, and quantitative are in the score metric, and the coefficients show effects on grouped levels of educational attainment.³ The right hand panel shows corresponding standardized coefficients. That is,

the variables are all expressed in standard-deviation units. All three types of literacy have statistically significant associations with the correlates. The effect of document literacy is much less than that of prose or quantitative literacy and the effect of prose literacy is slightly larger than that of quantitative literacy. Model fit deteriorates markedly, however, if the effect of either document or quantitative literacy is ignored.

Thus, while the panel notes the apparently prime importance of prose literacy, the other dimensions should not be ignored, and for some purposes it may be useful to construct a composite of the three literacy scores. It is not clear how to interpret the separate effects of the three literacy dimensions because they are so highly confounded by design in NALS and NAAL. That is, as long as the same task yields items scored on multiple dimensions, prose, document, and quantitative scores are intrinsically confounded.

Hauser, R.M., and Goldberger, A.S, (1971). The treatment of unobservable variables in path analysis. In H.L. Costner (Ed.), Sociological methodology 1971 (pp. 81-117). San Francisco: Jossey-Bass.

Joreskog, KG., and Goldberger, A.S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70, 631-639.