Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers (1991)
Chapter: DOYLE AND TRIPPE (1989)
of the estimated parameters. This was accomplished by matching elasticities derived from the CPS.
Using these estimated variances, Betson showed that predicted earnings under two alternative parameter sets were not significantly different. However, the distribution of earnings under the two parameter sets was significantly different. This demonstrated the dependence of the inference on the parameter sets, one of which was collected from experimental results and the other from nonexperimental results.
DOYLE AND TRIPPE (1989)
Doyle and Trippe (1989) carried out a validation of MATH in forecasting the impact of the food stamp program in 1984. This validation was made up of two phases. In the first phase, they compared administrative data for August 1984 with simulated values from the model implemented with a March 1985 CPS database, using program parameters for that time period. This procedure removed the contribution of forecasting error, which is usually present in MATH simulations based on databases that have been aged forward from a previous year. They also compared the CPS-based MATH model results with simulations from the Food Stamp Eligibility Routines (FOSTERS) model based on the 1984 Survey of Income and Program Participation (SIPP).
In the second phase, Doyle and Trippe directly evaluated the aging process by projecting an unaged 1980 database (from the March 1981 CPS) to 1984, making use of historical data from the March 1985 CPS to generate the control totals, again eliminating the contribution of forecasting error due to incorrect control totals. They compared the distribution of household characteristics for the low-income population—including characteristics controlled for and those not controlled for in the aging process—across the unaged 1980 database, the aged 1984 database, and the actual 1984 database (from the March 1985 CPS).
Doyle and Trippe made the following assumptions about possible causes of problems originating in the data sources feeding into MATH: (1) data limitations in understanding behavioral decisions; (2) weak macroeconomic projections; (3) nonsampling errors in the CPS, such as underreporting of income; (4) undercoverage of selected population groups in the CPS; and (5) the omission of variables (e.g., assets) in the March CPS necessary to determine program eligibility and benefit level. For comparison values, they used the administrative data on the food stamp caseload, recognizing that these data are also subject to error. To account for the sampling error present in the administrative estimates, confidence intervals about the estimates were used for comparison, rather than simply the estimates themselves. In addition, tolerance levels were used that attempted to represent differences that were not important in a subject-matter sense. Tolerance levels were defined to be the greater of 5 percent of the value of the administrative estimate and twice the sampling standard error.
