REVIEW PROCEDURES FOR WATER RESOURCES PROJECT PLANNING
Dr. Leonard Shabman, Ph.D.
Resources for the Future
Coordinating Committee on Assessing the U.S. Army Corps of Engineers Methods of Analysis and Peer Review for Water Resources Project Planning
National Research Council
The National Academies
Committee on Transportation and Infrastructure,
Water Resources and Environment Subcommittee
U.S. House of Representatives
Independent Peer Review of Products that Support Agency Decision-Making
March 5, 2003
Good afternoon Chairman Duncan and members of the committee. Thank you for the opportunity to address you today. My name is Leonard Shabman. I am a resident scholar at Resources for the Future in Washington, D.C. I currently serve as the chair of the coordinating committee for the National Research Council’s (NRC) assessment of the U.S. Army Corps of Engineers’ Methods of Analysis and Peer Review for Water Resources Project Planning. The National Research Council is the operating arm of The National Academies, chartered by Congress in 1863 to advise the government on matters of science and technology. I also am a member of the National Research Council’s Water Science and Technology Board. The focus of the board’s work is water resources science, technology, and policy.
In Section 216 of the 2000 Water Resources Development Act, the Congress asked the NRC to review Corps of Engineers methods and techniques of analysis. It also asked the NRC to report on ways the nation to implement independent review of feasibility reports for water resources projects. In responding to Section 216, the Academies appointed four panels, under the direction of a coordinating committee, to follow the panels’ progress and write a synthesis report. The panel on project review began its study in Fall 2001 and delivered its report, "Review Procedures for Water Resources Project Planning", to the Congress and the Administration in July 2002. My presentation today is made on behalf of the panel on project review. The panel members are listed at the end of this text and the report is available on the NRC website at http://www.nap.edu. The other panels will issue their reports later this year.
The Congress and the administration have increasingly been requesting reviews of Corps planning reports. Last year, the General Accounting Office issued a detailed review of the Corps analysis of the Delaware River deepening project. The Department of the Army asked the NRC to review the feasibility study on the Upper Mississippi River–Illinois Waterway after controversies arose over key assumptions within the study. The NRC was also requested by the Corps and the Environmental Protection Agency to review ecological changes and operating plans for the reservoir system on the Missouri River. When Congress authorized the Comprehensive Everglades Restoration Plan in WRDA 2000, it also requested that there be a review by an independent scientific review panel of the progress of that huge and complex planning effort.
The frequency of these requests is an indication of the interest in independent review of the most costly, complex, and controversial planning studies. To address this interest, the NRC panel proposed a structured process for securing independent external reviews. In addition, the panel recommended improvements in the internal review processes for Corps activities that may not warrant external reviews.
The NRC panel examined the technical review process conducted at the Corps’ district offices, the policy review process of the Corps Headquarters, and the nature of the reviews by the Assistant Secretary of the Army and the Office of Management and Budget. The panel also assessed the previous Washington-level review process, including that at Corps Headquarters, by the former Board of Engineers for Rivers and Harbors, and by the Office of the Assistant Secretary for Civil Works.
The panel recognized and sought to accommodate the schedule, budget and decision making requirements of the Corps programs. In particular, the Corps operates through individual project authorizations where the administration, the Congress and the cost-sharing sponsor may share the decision making, and sometimes planning, responsibility. The panel recommendations fit well with this process and will not significantly increase the overall cost of major planning studies. More importantly, such reviews may lead to savings when they resolve technical issues before they become matters of conflict that delay a decision on a study.
The panel recommended that the Congress direct the Secretary of the Army to establish a small professional staff to administer the review process and that resources be provided for its operations. This administrative group for project review could be housed either in the Office of the Assistant Secretary of the Army for Civil Works or within the Office of the Chief of Engineers, and depending on the office, would report directly to either the Secretary or the Chief. However, this staff group would not conduct reviews. Instead it would 1) decide on a case-by-case basis whether reviews of Corps planning studies ought to be conducted externally or internally, 2) would administer the review process for assuring timeliness of review, and 3) would assist the Chief of Engineers in responding to the reviews. I will comment on each responsibility in turn.
The administrative group determines whether an independent external group or an internally constituted body will conduct the review. External reviews would be required for projects or other activities that, in the judgment of the administrative group, would be analytically complex, costly or controversial. Note that the panel did not specify thresholds of complexity, cost or controversy that would trigger an external review. It did recognize the possibility for disagreements with a decision on internal versus external review. Therefore, either the Administration (perhaps from the director of the OMB or the CEQ) or Congress (through congressional resolution or other legislative action) would be able to request an external review of a Corps planning study.
If an external review were called for, the administrative group staff would enter into a contract with an independent organization for the conduct of the review. Examples of organizations that might lead these independent reviews include professional science or engineering societies, the National Academies, and the National Academy for Public Administration. External reviewers should not be selected by the Corps or include Corps staff or others with a conflict of interest. With these external reviews, selection of the reviewers would be at the discretion of the independent body, but the administrative group can recommend reviewers.
Internal reviews are adequate for Corps planning studies that are less complex, which have modest cost, and are unlikely to generate significant controversy. In these instances, staff of the administrative group would select internal review panels that consist of a balance between Corps staff and external experts.
I want to stress the importance of independence of the reviewers in external reviews. In discussing the external review process, Dr James Mitchell University Distinguished Professor Emeritus, Virginia Tech , who was the NRC panel chair, noted "The highest degree of credibility will be achieved if responsibility for external review is given to an organization that is independent of the Corps." He added, "There is a strong correlation between the independence of reviewers and the credibility, both real and perceived, of review."
In complex planning studies, there may be several reviews during the study process. The panel recommended that the final independent review commence after the 60-day comment period on the draft feasibility report and EIS.
The administrative group should provide a summary document of the planning report to all review panels. The administrative group would also provide a comprehensive compendium to all review panels, including review comments and responses from any internal reviews, and comments and communications between the Corps and relevant federal agencies, interest groups, and the public. The review itself would commence when the reviewers accepted this document.
Once an external, independent review is underway, it is imperative that the review be completed as expeditiously as possible. Therefore, the administrative group would be responsible for communication between those who prepared the planning study and the reviewers. This role calls for the administrative group to expedite getting answers to questions of clarification and explanation raised by the reviewers, while also enhancing the administrative groups understanding of the reviewers’ concerns.
The review comments would be delivered to the Chief of Engineers. After considering each comment, the Chief may send the report back to the field for further analysis or may support authorization of some action. Without regard to the Chief’s decision, the Chief’s report would include responses to each of the review comments. The Chief should either agree with a point and explain how it was addressed or the point should be rebutted with an explanation of the reason for rejecting the comment. The report from the review panel, with the written responses from the Chief, would be included in the report that passes to the ASA(CW), the OMB and then to the Congress.
The NRC panel believed that for the more complex planning studies, it may be appropriate to conduct an initial review in the study’s early phases, with more comprehensive later, relying on separate reviewers at each stage. In this way the review results can improve the study and lend credibility to the process. In the event that review is initiated in a planning study’s early stages, the results of the review should be submitted to the Corps’ District Engineer. The District Engineer would then be responsible for preparing a written document that explains how the Corps intends to incorporate the review results into the study. That set of comments and questions would be part of the public record and would be included with the final feasibility report.
Let me comment briefly on the nature of the review itself. The reviewers will not comment on whether a project should be constructed or whether a particular operations plan should be implemented; the administration and the Congress are ultimately responsible for this final decision. The reviewers will evaluate whether a full range of technically feasible and plausible alternatives was considered. While there may be compelling policy and other reasons why certain alternatives may not be considered, these reasons would need to be made clear in the Chief’s response to review. Also, the review panel will identify, evaluate, and comment on key assumptions that underlie the technical, economic, and environmental analysis in the planning studies. In this regard the panel received advice early in its study that the review should focus on matters of technical evaluation and avoid making statements on policy. After much deliberation, the NRC panel concluded that the boundary between the technical and the policy realms was often unclear. Rather that have the reviewers linger over questions of whether a topic was one of policy or method, the NRC panel concluded that the reviewers should be free to comment on any aspect of the report that they felt was relevant to the analysis. If the Chief feels that a comment is on a policy and not technical matter, this can be indicated in the written response to the review.
As a final thought, let me respond to one comment I have heard about this report. Some who have had the opportunity to study the report have noted the great emphasis on organizational structure and have wondered why such a structure is needed. This structure was carefully considered by the committee and represents their collective judgment on how to best secure essential independence, accountability and a way to secure a separation of technical from policy issues. There may be other ways to organize a review of Corps of Engineers Planning studies and still secure these goals. However, the ability to secure independence, accountability and separation of technical from policy issues should be the test of any review process.
I thank you again for the opportunity to speak with you today and would be pleased to discuss questions you may have about this panel’s recommendations regarding expert review within the Corps of Engineers’ project planning.
LEONARD SHABMAN is a resident scholar at Resources for the Future in Washington, D.C. Prior to joining RFF, Dr. Shabman was a professor of agricultural and applied economics at Virginia Tech and director of the Virginia Water Resources Research Center. Dr. Shabman earned his Ph.D. in resource and environmental economics from Cornell University. During his career he has served as staff economist at the United States Water Resources Council, as Scientific Advisor to the Assistant Secretary of Army, Civil Works August and as Visiting Scholar at the National Academy of Sciences National Research Council. His present research includes a review of the history and consequences of Section 404 of the Clean Water Act, strategies for water quality standard setting under the Clean Water Act, design of incentive-based systems for water quality management and development of evaluation protocols for large scale ecosystem restoration projects. Dr. Shabman is currently a member of the National Research Council’s Water Science and Technology Board and has served on several National Research Council committees.[<br>]
JAMES K. MITCHELL, Chair, Virginia Polytechnic Institute and State University, Blacksburg
MELBOURNE BRISCOE, Office of Naval Research, Arlington, Virginia
STEPHEN J. BURGES, University of Washington, Seattle
LINDA CAPUANO, Honeywell, Inc., San Jose, California
DENISE FORT, University of New Mexico, Albuquerque
PORTER HOAGLAND, Woods Hole Oceanographic Institution, Woods Hole, Massachusetts
DAVID H. MOREAU, University of North Carolina, Chapel Hill
CRAIG PHILIP, Ingram Barge Company, Nashville, Tennessee
JOHN T. RHETT, Consultant, Arlington, Virginia
RICHARD E. SPARKS, Illinois Water Resources Center, Urbana
BORY STEINBERG, Steinberg and Associates, McLean, Virginia