Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers (1991)
Chapter: HAVEMAN AND LACKER (1984)
The approach used was to examine the discrepancies between the cross-tabulated counts from DYNASIM and those given by either CPS or SSA. One result was that DYNASIM’s estimates of the percentage of the population with earnings were very close to those of the CPS. However, some rather large errors were discovered with respect to distributional estimates. These included (1) the estimates of average earnings for males and females 16–19 years old in 1978 (for males DYNASIM estimated $4,241 and CPS estimated $2,190), and (2) the percentages of workers with earnings who were covered by social security in 1978 (for males earning over $16,500, DYNASIM estimated 28.2%, and the 1977 SSA data gave 22.6%; for females earning under $1,200, DYNASIM estimated 9.3% and the 1977 SSA data gave 21.6%). Also, average earnings for people between 20 and 29 were off by about 8 percent. Generally, though, most errors were much less than 5 percent.
An interesting comparison not made would be that of simply using the 1973 CPS for estimates of means and for distributional information. This would provide a baseline to assess how much variability there was to explain and how much of it DYNASIM was successful at explaining.
HAVEMAN AND LACKER (1984)
Haveman and Lacker (1984) analyzed the differences between DYNASIM and PRISM in their estimation of future public and private pension benefits. The initial database for DYNASIM was the March 1973 CPS-SSA exact-match file; the initial database for PRISM was an exact match of the 1978 CPS-SSA exact-match file with the May 1979 and March 1979 CPS. Haveman and Lacker report that, even though the microdata simulation procedures on which DYNASIM and PRISM rest are marked improvements over previous methodologies, the ability to accurately project retirement income has yet to be demonstrated and that the baseline projections do diverge for these models. For example, for the year 2000, for private pension benefits, DYNASIM projected the average benefit for 65-year-old males to be $3,509; PRISM projected $6,160. The discrepancy for 65-year-old females was similar in relative magnitude.
Haveman and Lacker suggest five possible sources for these discrepancies: (1) the initial samples were different, (2) there were different specifications for the endogenous relationships, (3) the relationships were estimated based on different data sets, (4) different judgments were made in situations where no data existed, and (5) different exogenous parameter values were used. Rather than carry out a sensitivity analysis, budget constraints limited these authors to a qualitative assessment of the likely source of the differences. They admit that this approach does not provide a clear answer as to which model’s projections were more reliable. For example, they note that DYNASIM took race, education, and marital status into consideration in its mortality module, but that PRISM did not. On the other hand, PRISM made use of disability.
