Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers (1991)
Chapter: BEEBOUT AND HAWORTH (1989)
Doyle and Trippe’s first-phase analysis demonstrated that MATH provided estimates of program costs and caseloads that deviated from the administrative data by less than 3 percent. In addition, estimates of the distribution of the caseload by important characteristics, including household size and gross monthly income, were of similar quality.
However, Doyle and Trippe showed that in other respects MATH was less effective. A major problem was that it simulated too few food stamp households with public assistance and with school-age children and too many food stamp households with elderly persons and with earnings. For example, it estimated that 55.4 percent of the participating households would have children, while the administrative figure was 60.9 percent. For elderly persons, the MATH figure was 9.7 percent, while the administrative figure was 7.0 percent. Doyle and Trippe showed that these problems stemmed from errors in the CPS, namely the 25 percent underrepresentation of poor single-adult households with children. In truth, the target number of food stamp participants among this group could not be met because the CPS contained so few households of this type. Doyle and Trippe also examined whether basing simulations on SIPP in place of the CPS would eliminate the problem, but it persisted.
Doyle and Trippe showed that MATH produced 41 percent more disabled households than expected based on administrative data, a result of the attempt to meet the target number of households with means-tested cash income. Also, twice as many households with children were estimated to incur the child care deduction as shown by the administrative data. They also discovered that MATH estimated that 48.6 percent of households failed the assets test, compared with 64.7 percent from the SIPP-based FOSTERS simulation. Since Doyle and Trippe had previously simulated food stamps on aged files and noticed the same problems, the aging module was not thought to be associated with these deficiencies.
In the second-phase analysis, Doyle and Trippe examined the benefits of using the MATH aging module. As would be expected, aging was relatively beneficial for variables that were controlled to but performed poorer for variables that were not controlled to, such as the size of the poverty population.
BEEBOUT AND HAWORTH (1989)
Beebout and Haworth (1989) performed an external validation of the MATH model estimates that were produced during the debate leading up to the 1977 Food Stamp Act (P.L. 95–113). As is typical with most external validations of microsimulation models, given the conditional nature of the forecasts, certain unforeseen circumstances needed to be taken into account so that the conditional forecasts could be “unconditionally” compared with the actual results. Determining the effect on participation of program reform during this period was difficult because of the economic recession that occurred shortly
