people who correctly report their participation. In one state, a large proportion of AFDC recipients incorrectly reported their benefits as general assistance.
Other studies are investigating the so-called seam problem that first showed up in SIPP, but, on further investigation, turns up in other longitudinal surveys, such as the ISDP and the PSID. The seam problem concerns the tendency of respondents to report transitions as occurring between interviews rather than within interviews. Thus, in SIPP, exits from and entrances to programs such as AFDC and food stamps are reported more often between pairs of months that span waves than between pairs of months within waves. In the case of food stamps, it appears that the net effect on cross-sectional estimates is small—that is, total exits and entrances from SIPP are close to rates derived from food stamp administrative records, even though the timing of SIPP transition reports is often in error. On the other hand, for SSI, entrance rates from SIPP are significantly higher than those shown by program records (Jabine, King, and Petroni, 1990:59–60).
Another problem that confronts panel surveys is “time-in-sample” bias, whereby participation in the survey changes responses over time (either because respondents actually change their behavior, such as applying for benefits after learning about the existence of a program from the survey, or because respondents become better or worse at answering the questions after repeated exposure to the interview). Only limited and inconclusive studies of time-in-sample bias have been conducted of SIPP to date, although work on this topic is in progress (see Lepkowski, Kalton, and Kasprzyk, 1990).
Sample Size
The review of data quality problems up to this point generally favors the SIPP, particularly given the fact that data quality issues from the perspective of analyzing the low-income population have simply not received the same kind of scrutiny in the CPS as they have in SIPP. However, the CPS has a signal advantage for microsimulation modeling in terms of sample size. Microsimulation, as a technique for producing policy impact estimates, is distinguished by its ability to provide detailed distributional information about gainers and losers. Adequate sample size is essential to the reliability of distributional estimates. A particular requirement in the case of modeling the AFDC program, given that eligibility provisions and benefit levels vary by state, is that the sample be of sufficient size to permit identifying all 50 states and the District of Columbia. Ideally, the sample size would be sufficient to support state estimates for this program.
Table 6 indicates the estimated number of SIPP sample persons with specified characteristics in a sample of 20,000 households (the size of the 1984 and 1990 panels) and one of 12,000 households (the size of other panels). The numbers are small for many important income support programs. For example,
TABLE 6 Estimated Numbers of SIPP Sample Persons for Selected Subpopulations
|
Subpopulation |
For a Sample of |
|
|
20,000 Households |
12,000 Households |
|
|
All persons |
53,700 |
32,200 |
|
Adults |
41,400 |
24,850 |
|
Persons 65 and over |
5,965 |
3,580 |
|
Persons 75 and over |
2,600 |
1,560 |
|
Persons in households with income less than poverty (monthly) |
7,400 |
4,440 |
|
Recipients of |
|
|
|
Social security (aged and disabled) |
7,475 |
4,485 |
|
Railroad retirement |
175 |
105 |
|
AFDC |
705 |
420 |
|
General assistance |
245 |
150 |
|
SSI (federal and state) |
750 |
450 |
|
Medicare |
6,510 |
3,905 |
|
Medicaid |
4,125 |
2,475 |
|
WIC |
570 |
340 |
|
Multiple recipients of |
|
|
|
Food stamps and AFDC |
675 |
405 |
|
Food stamps and SSI |
285 |
170 |
|
Social security and food stamps |
385 |
230 |
|
Social security and housing assistance |
335 |
200 |
|
Medicaid and SSI |
795 |
480 |
|
Food stamps and housing assistance |
315 |
190 |
|
SOURCE: Jabine, King, and Petroni (1990: Table 9.1). |
||
the larger sample contains only 705 AFDC and 750 SSI recipients, while the smaller sample contains only 420 AFDC and 450 SSI recipients. Analyses of subgroups are severely compromised by these limited numbers of cases.12 For example, fewer than 50 cases would be available from the larger SIPP sample to analyze the small but important component of the AFDC caseload with earnings. Moreover, the SIPP data files do not separately identify all 50 states and do not support state-by-state estimates.
The corresponding sample size figures from the CPS would be about three times larger than those of the larger SIPP panel, providing a much more robust basis for estimating the impact of proposed policy changes. The CPS also identifies all states, although the sample size is currently not large enough to support reliable estimates for more than a few of the largest states.13
