Previous Chapter: Household and Individual Nonresponse
Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.

survey universe are also weighted so long as they were successfully interviewed while they were part of the sample.) Currently, up to three sets of positive longitudinal weights are provided for each record: a weight for people who provided a complete set of interviews for the first calendar year covered by the panel; a weight for people who provided a complete set of interviews for the second calendar year covered by the panel; and a weight for people who provided a complete set of interviews for the entire span covered by the panel.

Several studies of SIPP nonresponse have been conducted with the 1984 panel. The results show that household noninterview rates (that is, rates for households not responding after the first interview) tend to be higher for renters, for households located in large metropolitan areas, and for households headed by young adults. Individuals who did not complete all of the interview waves, compared with those who did, tend to include more residents of large metropolitan areas, renters, members of racial minorities, children and other relatives of the reference person, people aged 15–24, movers, never-married people, and people with no savings accounts or other assets; see Table 2.

From the viewpoint of simulating income support programs, these findings are disturbing since noninterview adjustment procedures are unlikely to be able to correct completely for these kinds of biases. One evaluation of the effectiveness of the cross-sectional weighting procedure to adjust for household nonresponse developed two sets of weights for wave two households in the 1984 panel—one set based on all wave two households and one set based just on those wave two households that provided interviews at wave six. Comparing estimates from these two samples showed that the latter set produced higher estimates of median income and fewer households with low monthly income compared with the former set, evidence that the weighting adjustments probably do not adequately compensate for differential attrition (Petroni and King, 1988). A subsequent study that compared samples from the 1985 panel of all wave two households and those that provided interviews at wave six obtained similar findings (King et al., 1990).

Item Nonresponse

In addition to population undercoverage and to household and person nonresponse, there is nonresponse on the part of individuals to particular items in the March CPS and the SIPP. In the CPS, nonresponse rates for some items are quite high. Indeed, as much as 20 percent of the total income in the March CPS is imputed by the Census Bureau (including income imputed due to person nonresponse as well as individual item nonresponse). Income imputation rates

Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.

TABLE 2 Selected Characteristics of Persons by Their Interview Experience for the Full 1984 SIPP Panel

 

Interview Experience (% distribution)

Characteristics of Persons in Wave One

Completed All Nine Waves

Missed Last Two Waves

Othera

Missed One Wave

Missed Two or More Waves

Relationship

 

Reference person

37.3

33.1

33.9

30.7

Primary individual

12.7

14.7

13.4

12.0

Spouse

31.3

23.9

25.4

23.9

Child

13.9

17.9

18.4

24.1

All other

4.8

10.3

9.0

9.4

Age

 

15–24

18.2

24.2

25.3

29.5

25–34

22.1

21.4

22.6

22.9

35–44

17.3

14.4

17.0

14.9

45–64

27.0

23.2

21.5

23.2

65 and over

15.4

16.8

13.6

9.5

Race

 

White

88.5

83.8

82.9

78.2

Black

9.2

12.9

13.9

17.6

Other

2.3

3.3

3.2

4.2

Living quarters

 

Owned

72.7

62.9

65.6

59.8

Rented

24.8

35.1

32.1

37.3

Rent free

2.4

2.0

2.3

2.8

Marital status

 

Never married

20.8

28.8

28.3

33.6

Married

63.5

52.2

53.3

50.4

Other

15.8

18.9

18.5

15.9

Savings account

 

Yes

59.9

51.1

51.9

51.0

No

40.1

48.9

48.1

49.0

aInterview experience categories are mutually exclusive. At least one of the last two interviews was completed for persons in the “other” category.

SOURCE: Jabine, King, and Petroni (1990: Table 5:4)

Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.

vary by income source, with rates as high as one-third for recipients of nonfarm self-employment income, interest, and dividends.8

The Census Bureau supplies values for missing income and other items in the CPS by use of its “statistical matching” procedures, which have come to replace “hot-deck” imputation procedures for more and more items. (See Welniak [1990] for a review of the imputation systems used in the March CPS over time and the effects of the current system, which was introduced in 1989, on estimates of aggregate income and poverty rates compared with the previous system.) Generally speaking, hot-deck imputation methods, instead of searching the data file for the best donor to supply a value for a missing item, assign a “nearest neighbor” value. That is, the data records are sorted by geographic area and processed sequentially, and reported values are used to update matrices of characteristics. A record with a missing item has the most recently updated value assigned from the appropriate matrix (for example, a matrix of earnings for people with specified demographic and occupational characteristics). Ford (1983) provides an overview of hot-deck and related imputation methods (see also Kalton and Kasprzyk, 1982).

Oh and Scheuren (1980) evaluated the impact of the Census Bureau’s procedures on the variance of the CPS income estimates. Welniak and Coder (1980) evaluated biases in earnings amounts, by creating a sample of pseudo-nonrespondents (by deleting reported values) and comparing the imputed values for these people with their actual reports. They estimated that the CPS imputation procedures produced a downward bias for earnings of about 2 percent overall. Among other results, they determined that earnings were underimputed by 6 percent for people who worked year-round (50 or more weeks) and overimputed, in some cases substantially, for people working fewer weeks.

David et al. (1986) compared the CPS imputation procedures with a regression-based imputation for earnings, using Internal Revenue Service (IRS) data from a 1981 exact-match CPS-IRS file as the measure of truth, and found that the CPS methodology performed reasonably well.9 However, they and other analysts determined that the CPS imputations were less successful for small

8  

See, for example, Bureau of the Census (1989: Table C-1). About one-half of the item nonresponse to income questions in the March CPS is attributable to the refusal of respondents to answer any part of the supplement. CPS interviewers are urged to concentrate on obtaining answers to the core labor force questions and on keeping the household willing to participate in subsequent months; hence, they do not press for response to any of the supplements. One study found that nonresponse rates to the March income questions rose over the decade of the 1970s and that nonresponse rates were higher for telephone compared with personal interviews and for people with high or low total money income compared with middle-income people (Coder, 1978: Table 2, Figure 3).

9  

David et al. (1986) were able to use the CPS-IRS exact-match file, which is not publicly available, when they were in residence at the Census Bureau and sworn in as special census employees under the fellowship program sponsored by the National Science Foundation and the American Statistical Association. Their study summarizes earlier analyses of the impact of alternative imputation procedures for earnings that used the 1973 exact-match CPS-IRS-Social Security Administration file.

Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.

subgroups, such as minorities and specific occupations (Coder, no date; Lillard, Smith, and Welch, 1986).

There have been no similar evaluations of the impact of the CPS imputation procedures from the viewpoint of modeling the low-income welfare-eligible population. Admittedly, an evaluation would be difficult. For example, imputation of a high annual income to a household reporting receipt of food stamps at some time during the previous year would not necessarily be an anomalous result. The household could have had a sizable intrayear variation in income, such that for 1 or more months the household was indeed eligible for food stamp benefits.

SIPP also exhibits item nonresponse, although its performance is generally much better than that of the CPS for income amounts. Table 3 shows comparisons of item nonresponse rates for selected income amounts for the 1984 SIPP and the March 1985 CPS and for the 1985 SIPP and the March 1986 CPS. Generally, SIPP item nonresponse rates are only one-half to two-thirds of the CPS rates. Overall, only 11 percent of total regular money income is imputed in SIPP, compared with 20 percent in CPS. SIPP nonresponse rates for labor force activity and for recipiency status with regard to income and asset holdings are generally very low (1–3%); however, nonresponse rates for asset amounts are quite high (although lower than corresponding rates in the ISDP, SIPP’s predecessor). For example, nonresponse rates are as high as 40 percent for the market value of stocks, debt on stocks, and value of own business.

The Census Bureau uses the older hot-deck imputation methods for treating item nonresponse in the SIPP cross-sectional data files for each wave. (Over the years, both the hot-deck and statistical match procedures have become more complex.) In the longitudinal panel files, when possible, the Census Bureau replaces cross-sectionally imputed values with interpolated values from surrounding waves and performs other editing of the data on the basis of the entire array of information available for each case.

There have been a few studies of the impact of the SIPP imputation procedures from the viewpoint of modeling income support programs. A troubling characteristic of current procedures is that they do not take low income or receipt of program benefits into account in imputing program-related variables. Thus, Doyle and Dalrymple (1987) found that the imputation of income in the 1984 SIPP panel for households reporting receipt of food stamps produced a larger proportion of such households with high monthly incomes that would make them ineligible for food stamp program benefits than households that reported both their cash income and food stamps. (The overall income distribution of food stamp households was not greatly affected because income was imputed for only a small number of such households.)

Allin and Doyle (1990) found similar and more striking effects from the imputation of asset amounts for low- to moderate-income households in the 1984 SIPP panel. Of all households with incomes at or below 250 percent

Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.

TABLE 3 Item Nonresponse Rates in the SIPP and CPS, Selected Income Types (percent)

a. 1984 SIPP and March 1985 CPS

 

SIPP Monthly Average 1984

CPS March 1985

Income Type

1st Quarter

2nd Quarter

3rd Quarter

4th Quarter

Wage and salary

7.2

7.5

7.5

7.6

18.9

Self-employment income

16.8

16.2

16.0

16.1

26.5

Supplemental security income (federal)

7.6

8.4

8.1

8.4

19.9

Social security

10.8

11.6

11.7

12.3

21.9

Aid to Families with Dependent Children

6.1

6.9

6.5

5.5

16.0

Unemployment compensation

10.1

13.6

10.4

12.7

21.8

Company or union pension

13.9

14.0

12.8

14.7

24.0

Food stamp allotment

5.2

6.3

6.7

6.6

13.7

Veterans’ compensation or pension

11.3

11.2

11.9

13.5

18.3

b. 1985 SIPP and March 1986 CPS

 

SIPP Monthly Average 1985

CPS March 1986

Income Type

1st Quarter

2nd Quarter

3rd Quarter

4th Quarter

Wage and salary

7.8

6.7

8.5

8.3

16.5

Self-employment income

17.0

9.3

19.4

17.6

21.9

Supplemental security income (federal)

8.5

8.7

8.0

8.5

16.5

Social security

13.0

12.5

12.7

13.2

19.3

Aid to Families with Dependent Children

8.9

8.1

8.5

8.5

14.4

Unemployment compensation

15.0

15.6

14.9

16.0

18.6

Company or union pension

17.0

16.1

16.2

16.5

21.0

Food stamp allotment

7.1

5.9

6.3

6.6

11.1

Veterans’ compensation or pension

13.1

13.4

13.2

16.2

18.8

NOTE: Noninterviews or complete failure to obtain cooperation from any household member has not been considered in this examination of nonresponse rates.

SOURCE: Jabine, King, and Petroni (1990: Table 5.9).

of the poverty threshold that reported checking account balances (17% of the total of such households), one-fifth had the amount of the balance imputed, and the mean value of the imputed balances was seven times that of the reported balances ($1,969 versus $268), Of the one-third of low- to moderate-income households reporting life insurance policies, 27 percent had the face value

Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 28
Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 29
Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 30
Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 31
Suggested Citation: "Item Nonresponse." National Research Council. 1991. Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers. Washington, DC: The National Academies Press. doi: 10.17226/1853.
Page 32
Next Chapter: Reporting Errors
Subscribe to Emails from the National Academies
Stay up to date on activities, publications, and events by subscribing to email updates.