usefulness of microsimulation in general, rather than about the usefulness of behavioral simulation. The inconsistencies between the MATH and the TATSIM forecasts imply that predictions of work effort are sensitive to imputation of the prereform environment and to unreliable estimates of income and substitution effects. TATSIM actually produced standard errors for some of its aggregate estimates. Burtless argued that the standard errors of the labor supply predictions were extremely small when compared with the large discrepancies observed in labor supply predictions.
In addition, Burtless supplied evidence implying that there is probably a complicated labor response of single mothers to negative income tax plans. He said that analysts should be encouraged to develop and estimate models that reflect the possibility of more complicated individual responses, possibly completely stochastic, to various social and economic changes.
DISCUSSION
Table 1 summarizes certain aspects of the 13 studies discussed here. The column entries for many of the studies were not always obvious, and many of the entries are debatable. For example, it was sometimes not clear whether a discovery was used to modify a model, and it was not always clear exactly what the difference was between a one-time analysis and 6 years of analysis for a dynamic model such as DYNASIM. Nonetheless, the following general impressions are true regardless of how these problems are resolved: (1) very few microsimulation models have had any validation analysis, and all but two or three have had at most one comprehensive study involving 1 year’s outputs, (2) validation often leads to model improvement, and (3) there has probably never been a detailed analysis of the size and origin of the errors of a microsimulation model’s outputs, including a separation of error into variance due to that from variability in the input data set and bias due to model component misspecifications and errors in the input data set, TATSIM seems to be the only example of any attempt at measuring variance, and it appears to have provided overly optimistic assessments of variability.
TABLE 1 Summary of Validation Studies for Microsimulation Models
