survey universe are also weighted so long as they were successfully interviewed while they were part of the sample.) Currently, up to three sets of positive longitudinal weights are provided for each record: a weight for people who provided a complete set of interviews for the first calendar year covered by the panel; a weight for people who provided a complete set of interviews for the second calendar year covered by the panel; and a weight for people who provided a complete set of interviews for the entire span covered by the panel.
Several studies of SIPP nonresponse have been conducted with the 1984 panel. The results show that household noninterview rates (that is, rates for households not responding after the first interview) tend to be higher for renters, for households located in large metropolitan areas, and for households headed by young adults. Individuals who did not complete all of the interview waves, compared with those who did, tend to include more residents of large metropolitan areas, renters, members of racial minorities, children and other relatives of the reference person, people aged 15–24, movers, never-married people, and people with no savings accounts or other assets; see Table 2.
From the viewpoint of simulating income support programs, these findings are disturbing since noninterview adjustment procedures are unlikely to be able to correct completely for these kinds of biases. One evaluation of the effectiveness of the cross-sectional weighting procedure to adjust for household nonresponse developed two sets of weights for wave two households in the 1984 panel—one set based on all wave two households and one set based just on those wave two households that provided interviews at wave six. Comparing estimates from these two samples showed that the latter set produced higher estimates of median income and fewer households with low monthly income compared with the former set, evidence that the weighting adjustments probably do not adequately compensate for differential attrition (Petroni and King, 1988). A subsequent study that compared samples from the 1985 panel of all wave two households and those that provided interviews at wave six obtained similar findings (King et al., 1990).
In addition to population undercoverage and to household and person nonresponse, there is nonresponse on the part of individuals to particular items in the March CPS and the SIPP. In the CPS, nonresponse rates for some items are quite high. Indeed, as much as 20 percent of the total income in the March CPS is imputed by the Census Bureau (including income imputed due to person nonresponse as well as individual item nonresponse). Income imputation rates
TABLE 2 Selected Characteristics of Persons by Their Interview Experience for the Full 1984 SIPP Panel
|
|
Interview Experience (% distribution) |
|||
|
Characteristics of Persons in Wave One |
Completed All Nine Waves |
Missed Last Two Waves |
Othera |
|
|
Missed One Wave |
Missed Two or More Waves |
|||
|
Relationship |
|
|||
|
Reference person |
37.3 |
33.1 |
33.9 |
30.7 |
|
Primary individual |
12.7 |
14.7 |
13.4 |
12.0 |
|
Spouse |
31.3 |
23.9 |
25.4 |
23.9 |
|
Child |
13.9 |
17.9 |
18.4 |
24.1 |
|
All other |
4.8 |
10.3 |
9.0 |
9.4 |
|
Age |
|
|||
|
15–24 |
18.2 |
24.2 |
25.3 |
29.5 |
|
25–34 |
22.1 |
21.4 |
22.6 |
22.9 |
|
35–44 |
17.3 |
14.4 |
17.0 |
14.9 |
|
45–64 |
27.0 |
23.2 |
21.5 |
23.2 |
|
65 and over |
15.4 |
16.8 |
13.6 |
9.5 |
|
Race |
|
|||
|
White |
88.5 |
83.8 |
82.9 |
78.2 |
|
Black |
9.2 |
12.9 |
13.9 |
17.6 |
|
Other |
2.3 |
3.3 |
3.2 |
4.2 |
|
Living quarters |
|
|||
|
Owned |
72.7 |
62.9 |
65.6 |
59.8 |
|
Rented |
24.8 |
35.1 |
32.1 |
37.3 |
|
Rent free |
2.4 |
2.0 |
2.3 |
2.8 |
|
Marital status |
|
|||
|
Never married |
20.8 |
28.8 |
28.3 |
33.6 |
|
Married |
63.5 |
52.2 |
53.3 |
50.4 |
|
Other |
15.8 |
18.9 |
18.5 |
15.9 |
|
Savings account |
|
|||
|
Yes |
59.9 |
51.1 |
51.9 |
51.0 |
|
No |
40.1 |
48.9 |
48.1 |
49.0 |
|
aInterview experience categories are mutually exclusive. At least one of the last two interviews was completed for persons in the “other” category. SOURCE: Jabine, King, and Petroni (1990: Table 5:4) |
||||
vary by income source, with rates as high as one-third for recipients of nonfarm self-employment income, interest, and dividends.8
The Census Bureau supplies values for missing income and other items in the CPS by use of its “statistical matching” procedures, which have come to replace “hot-deck” imputation procedures for more and more items. (See Welniak [1990] for a review of the imputation systems used in the March CPS over time and the effects of the current system, which was introduced in 1989, on estimates of aggregate income and poverty rates compared with the previous system.) Generally speaking, hot-deck imputation methods, instead of searching the data file for the best donor to supply a value for a missing item, assign a “nearest neighbor” value. That is, the data records are sorted by geographic area and processed sequentially, and reported values are used to update matrices of characteristics. A record with a missing item has the most recently updated value assigned from the appropriate matrix (for example, a matrix of earnings for people with specified demographic and occupational characteristics). Ford (1983) provides an overview of hot-deck and related imputation methods (see also Kalton and Kasprzyk, 1982).
Oh and Scheuren (1980) evaluated the impact of the Census Bureau’s procedures on the variance of the CPS income estimates. Welniak and Coder (1980) evaluated biases in earnings amounts, by creating a sample of pseudo-nonrespondents (by deleting reported values) and comparing the imputed values for these people with their actual reports. They estimated that the CPS imputation procedures produced a downward bias for earnings of about 2 percent overall. Among other results, they determined that earnings were underimputed by 6 percent for people who worked year-round (50 or more weeks) and overimputed, in some cases substantially, for people working fewer weeks.
David et al. (1986) compared the CPS imputation procedures with a regression-based imputation for earnings, using Internal Revenue Service (IRS) data from a 1981 exact-match CPS-IRS file as the measure of truth, and found that the CPS methodology performed reasonably well.9 However, they and other analysts determined that the CPS imputations were less successful for small
subgroups, such as minorities and specific occupations (Coder, no date; Lillard, Smith, and Welch, 1986).
There have been no similar evaluations of the impact of the CPS imputation procedures from the viewpoint of modeling the low-income welfare-eligible population. Admittedly, an evaluation would be difficult. For example, imputation of a high annual income to a household reporting receipt of food stamps at some time during the previous year would not necessarily be an anomalous result. The household could have had a sizable intrayear variation in income, such that for 1 or more months the household was indeed eligible for food stamp benefits.
SIPP also exhibits item nonresponse, although its performance is generally much better than that of the CPS for income amounts. Table 3 shows comparisons of item nonresponse rates for selected income amounts for the 1984 SIPP and the March 1985 CPS and for the 1985 SIPP and the March 1986 CPS. Generally, SIPP item nonresponse rates are only one-half to two-thirds of the CPS rates. Overall, only 11 percent of total regular money income is imputed in SIPP, compared with 20 percent in CPS. SIPP nonresponse rates for labor force activity and for recipiency status with regard to income and asset holdings are generally very low (1–3%); however, nonresponse rates for asset amounts are quite high (although lower than corresponding rates in the ISDP, SIPP’s predecessor). For example, nonresponse rates are as high as 40 percent for the market value of stocks, debt on stocks, and value of own business.
The Census Bureau uses the older hot-deck imputation methods for treating item nonresponse in the SIPP cross-sectional data files for each wave. (Over the years, both the hot-deck and statistical match procedures have become more complex.) In the longitudinal panel files, when possible, the Census Bureau replaces cross-sectionally imputed values with interpolated values from surrounding waves and performs other editing of the data on the basis of the entire array of information available for each case.
There have been a few studies of the impact of the SIPP imputation procedures from the viewpoint of modeling income support programs. A troubling characteristic of current procedures is that they do not take low income or receipt of program benefits into account in imputing program-related variables. Thus, Doyle and Dalrymple (1987) found that the imputation of income in the 1984 SIPP panel for households reporting receipt of food stamps produced a larger proportion of such households with high monthly incomes that would make them ineligible for food stamp program benefits than households that reported both their cash income and food stamps. (The overall income distribution of food stamp households was not greatly affected because income was imputed for only a small number of such households.)
Allin and Doyle (1990) found similar and more striking effects from the imputation of asset amounts for low- to moderate-income households in the 1984 SIPP panel. Of all households with incomes at or below 250 percent
TABLE 3 Item Nonresponse Rates in the SIPP and CPS, Selected Income Types (percent)
a. 1984 SIPP and March 1985 CPS
|
|
SIPP Monthly Average 1984 |
CPS March 1985 |
|||
|
Income Type |
1st Quarter |
2nd Quarter |
3rd Quarter |
4th Quarter |
|
|
Wage and salary |
7.2 |
7.5 |
7.5 |
7.6 |
18.9 |
|
Self-employment income |
16.8 |
16.2 |
16.0 |
16.1 |
26.5 |
|
Supplemental security income (federal) |
7.6 |
8.4 |
8.1 |
8.4 |
19.9 |
|
Social security |
10.8 |
11.6 |
11.7 |
12.3 |
21.9 |
|
Aid to Families with Dependent Children |
6.1 |
6.9 |
6.5 |
5.5 |
16.0 |
|
Unemployment compensation |
10.1 |
13.6 |
10.4 |
12.7 |
21.8 |
|
Company or union pension |
13.9 |
14.0 |
12.8 |
14.7 |
24.0 |
|
Food stamp allotment |
5.2 |
6.3 |
6.7 |
6.6 |
13.7 |
|
Veterans’ compensation or pension |
11.3 |
11.2 |
11.9 |
13.5 |
18.3 |
|
b. 1985 SIPP and March 1986 CPS |
|||||
of the poverty threshold that reported checking account balances (17% of the total of such households), one-fifth had the amount of the balance imputed, and the mean value of the imputed balances was seven times that of the reported balances ($1,969 versus $268), Of the one-third of low- to moderate-income households reporting life insurance policies, 27 percent had the face value