Improving Information for Social Policy Decisions -- The Uses of Microsimulation Modeling: Volume II, Technical Papers (1991)
Chapter: OUTPUTS EXAMINED
which was run to provide simulations of 1987 law with the 1983 database (from the March 1984 CPS).
A fourth factor that was initially considered for study was the algorithm for deciding which eligible households choose to participate in social welfare programs. (See Giannarelli [1989b, 1990a] for a description of the current TRIM2 participation function and the calibration procedure that was used for the 1983 and 1987 baseline files [both adjusted and unadjusted for undercoverage] to align the participation function to match selected administrative control totals. See also Citro and Ross, Chapter 3 in this volume.) The possibility of replacing the current probit equation with a logit equation, least squares, or other alternative was considered but not carried through because of the time demands necessary for reestimating the equation and because the Urban Institute was then embarking on a project to reestimate the parameters of the participation function. Also, since the AFDC module depends on the participation function, reestimating it would have delayed portions of the experiment. Nevertheless, this is a module that clearly needs this type of examination. As Dick Michel (Urban Institute, personal communication, May 26, 1989) has pointed out:
The participation adjustments of all microsimulation models are really the core of the model’s credibility…. Further, to the extent that participation functions are up-to-date and can account for changes in policy variables (like the benefit level), they will automatically provide us with a best estimate of the potential behavioral responses to a policy change.
In addition to the 16 variations simulating 1987 law with the 1983 database (from the March 1984 CPS), we had available (1) the 1983 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1983 law with the March 1984 CPS;2 (2) the 1987 baseline files, both adjusted and unadjusted for undercoverage, which simulated 1987 law with the March 1988 CPS; and (3) tabulations from the IQCS of characteristics of the AFDC recipient population in both 1983 and 1987.3 These are shown as the QC83 and QC87 runs, respectively, in Table 1. (Table 1 provides a description of the 16+6=22 runs that constitute the experiment.)
OUTPUTS EXAMINED
The choice of which outputs to examine and the weight given to various
TABLE 1 Description of TRIM2 Experimental Runs
degrees of correspondence with the comparison values for those outputs (i.e., choice of loss function) will result in different rankings of the alternatives to TRIM2. In terms of policy relevance, the importance of various outputs will vary depending on the programs under consideration; the quality of the input data; and the economic, social, and demographic dynamics of the period in question. For this reason it is not meaningful to declare a specific loss function resulting from the choice and weighting of outputs as a method for scoring microsimulation models over situations. There are, however, three reasons to proceed informally in this direction. First, for a particular situation, an a priori choice of loss function does make comparisons between models more objective. Second, a comparison using several responses has the benefit of reducing attention to individual responses that might be subject to large
