William F. Eddy
John C. Warner Professor of Statistics
Carnegie Mellon University
Chair, Committee on National Statistics
Division of Behavioral and Social Sciences and Education
National Research Council
The National Academies
Joint Economic Committee
July 21, 2009
Good morning. My name is William Eddy, John C. Warner Professor of Statistics at Carnegie Mellon University and chair of the Committee on National Statistics of the National Research Council. The Research Council is the operating arm of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine of the National Academies, chartered by Congress in 1863 to advise the government on matters of science and technology.
My remarks today will address findings and recommendations in two of the Committee’s reports, both of which are relevant to the governance of the U.S. Census Bureau and the usefulness of the data it provides. The two reports are the fourth edition of Principles and Practices for a Federal Statistical Agency, issued by the Committee in 2009, and Using the American Community Survey: Benefits and Challenges, issued by a panel of the Committee in 2007. Both reports are available on the web site of The National Academies Press, www.nap.edu.
By way of introduction, the Committee on National Statistics was established at the National Research Council in 1972 at the recommendation of the President’s Commission on Federal Statistics to improve the statistical methods and information on which public policy decisions are based. The Committee carries out studies at the request of government agencies on statistical programs and methods. It also addresses the statistical policy and coordinating activities of the federal government, which are essential in a highly decentralized statistical system. Support for the Committee’s work is provided by a consortium of federal agencies through a grant from the National Science Foundation. Support for the Committee’s Panel on the American Community Survey was provided by the U.S. Census Bureau.
What Makes for an Effective Statistical Agency? The Importance of Independence
A major activity of the Committee to strengthen the federal statistical system is its signature white paper, Principles and Practices for a Federal Statistical System, known as P&P or the “purple book.” The Committee first issued P&P in 1992 in response to queries on what constitutes an effective statistical agency. Since 2001, the Committee has updated and reissued P&P every 4 years so that new cabinet appointees and others could be provided with a current edition. P&P has been widely cited and used by congressional and executive agencies including GAO and OMB.
The fourth edition lists four principles and eleven practices. The four principles are that a statistical agency must: (1) be in a position to provide objective information that is relevant to issues of public policy, (2) have credibility with those who use its data and information, (3) have the trust of those whose information it obtains (including households and businesses), and (4) have a strong position of independence within the government. The practices include, among others, a commitment to quality and professional practice and an active program of methodological and substantive research.
I want to elaborate on the fourth principle of a strong position of independence because it is relevant to many of the debates about governance of the U.S. Census Bureau and the decennial census. The Committee states in P&P that without the credibility that comes from a strong degree of independence, users may lose confidence in the accuracy and objectivity of a statistical agency’s data, and data providers may become less willing to cooperate with agency requests, thereby undermining the agency’s ability to carry out its mission to provide relevant, accurate, timely, and impartial statistics to serve all sides in the policy debate, as well as researchers, private and public sector planners, the media, and the general public. Of course, statistical agency independence is always exercised within a broad framework of departmental, OMB, and congressional oversight.
Characteristics related to a strong position of independence are that a statistical agency has the following:
- Authority for professional decisions over the scope, content, and frequency of data compiled, analyzed, or published within the framework set by its authorizing legislation. Most statistical agencies have such broad authority, limited by budgetary constraints, departmental requirements, OMB review, and congressional mandates.
- Authority for selection and promotion of professional, technical, and operational staff, including senior executive career staff.
- Recognition by policy officials outside the statistical agency of its authority to release statistical information, including accompanying press releases and documentation, without prior clearance.
- Authority to control information technology systems for data processing and analysis in order to securely maintain the integrity and confidentiality of data and reliably support timely and accurate production of key statistics.
- Authority for the statistical agency head and qualified staff to speak about the agency’s statistics before Congress, with congressional staff, and before public bodies.
- Adherence to fixed schedules in public release of important statistical indicators to prevent even the appearance of manipulation of release dates for political purposes.
- Maintenance of a clear distinction between statistical information and policy interpretations of such information by the president, the secretary of the department, or others in the executive branch.
- Dissemination policies that foster regular, frequent release of major findings from an agency’s statistical programs to the public via the media, the Internet, and other means.
The Committee has not undertaken a formal evaluation of the Census Bureau vis-à-vis these aspects of a strong position of independence. However, I note that the Department of Commerce has not always respected important aspects of statistical agency independence for the Bureau, such as authority for selection and promotion of staff.
Regarding the organizational placement of the Census Bureau or other statistical agencies, P&P takes no position as such. A variety of organizational structures can work. However, P&P makes clear that a statistical agency should be separate from the law enforcement, regulatory, and policy-making parts of a department. Moreover, steps that can usefully strengthen a statistical agency head’s independence include that the head be appointed for a fixed term by the President, with approval by the Senate, as is the case with the heads of the Bureau of Labor Statistics and the National Center for Education Statistics. For a fixed term, it is desirable that it not coincide with the presidential term so that professional considerations are more likely to be paramount in the appointment process. It is also desirable that a statistical agency head have direct access to the secretary of the department or the head of the independent agency in which the statistical agency is located. Such access allows the head to inform new secretaries about the appropriate role of a statistical agency and present the case for new statistical initiatives to the secretary directly. Similarly, it is desirable for a statistical agency to have its own funding appropriation from Congress and not be dependent on allocations from the budget of its parent department or agency, which may be subject to reallocation.
Today, some statistical agencies are headed by senior executive career officials, some have presidentially appointed heads with fixed terms, and some, including the Census Bureau, have presidentially appointed heads that lack fixed terms and serve at the pleasure of the president. Presidential appointment without a fixed term can be detrimental to the independence of a statistical agency because the agency head has political visibility but no guarantee against politically motivated pressure and even dismissal. A recently released report of a panel of the Committee, Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics, documents the firing of the director of the Bureau of Justice Statistics in 2005 because the director refused to alter a statistical press release to suit the policy views of departmental officials. This situation should never occur for a statistical agency head, and the report recommends a fixed term for the director of BJS. A fixed term of office for the Census Bureau director would also strengthen the independence and reputation for objectivity of this critically important statistical agency.
The American Community Survey: Major Innovation, Challenging to Use
Since its inception in 1972, panels of the Committee on National Statistics have produced over 30 interim, letter, and final reports on the decennial census and related programs, including several reports on the American Community Survey (ACS). The ACS, which went into full production beginning in 2005, represents a seismic shift in the landscape of small-area data on the U.S. population. This shift promises important benefits to data users in terms of much more timely, up-to-date, and higher quality information than the sample-based questions on the decennial census could ever provide. (Some of the census questions were first asked of a sample of the population in 1940; beginning in 1960, the sample questions were included on a separate “long form.”)
The benefits of the ACS can already be seen from the much more frequent articles in the media about important population changes in counties and cities—such as changes in the country of origin of immigrants in some areas—that formerly could only be identified at 10-year intervals. However, as the comprehensive review of the ACS in the 2007 report on Using the American Community Survey: Benefits and Challenges indicates, the ACS continuous design will initially challenge many small-area data users in federal, state, and local government agencies, researchers, the private sector, the media, and the public. These users were accustomed to the point-in-time estimates from the census long-form sample and must learn how to work with and interpret the 1-year, 3-year, and 5-year moving-average estimates from the ACS. Moreover, this learning process is still to occur for users of data for the smallest geographic areas given that the first 5-year estimates for areas with fewer than 20,000 people will not be released until late 2010, representing averages of data collected in 2005-2009.
The ACS without doubt is of great benefit for users of estimates for large geographic areas, such as states and large cities and counties. Not only are reliable 1-year average estimates produced every year from the ACS for such areas, but the 5-year estimates will enable users to compare estimates for user-defined areas within, say, a major city, by aggregating the estimates for census tracts and block groups. It is also undeniable that the sample size of the ACS is at present too small to provide precise estimates, even averaged over 5 years, for small counties, cities, and towns. For example, based on the calculations in our 2007 report, the 5-year average estimate of the poverty rate does not meet acceptable standards of precision until an area has about 10,000 people, and the 5-year average estimate of the poverty rate for school-age children does not reach acceptable standards of precision until an area has about 50,000 people. We understand that, historically, the sample size for the ACS represented a compromise between the size required for precise estimates for small areas and the budget that was deemed acceptable to the executive and Congress at the time the ACS was being designed. Additional funding will be required to increase the sample size sufficiently for precise small-area estimates.
Our 2007 report strongly supported the ACS but noted that the transition for users and the Census Bureau would be challenging. The continuous design of the ACS, in which data are collected, every month, is essential for a smooth field operation, but it does pose problems for users of interpreting estimates that are averages over 12, 36, or 60 months. It can also make it difficult to introduce new and revised questions to meet changing needs. The report urged support for the ACS. It recommended that the Census Bureau make sufficient funding of the ACS one of its top priorities and that the Bureau seek funding, not only for data collection and production, but also for ongoing programs of methodological research and evaluation and user outreach and education. Strong research and user education programs are essential for the ACS to fulfill its mission to provide relevant, useful, and accurate small-area information and to improve the survey in the future as experience is gained with its benefits and challenges.
I thank the Joint Economic Committee for this opportunity to testify and will be happy to respond to any questions the members may have.
An archived webcast of the hearing can be found on the Joint Economic Committee's Web site.