"Transportation Research Priorities: Maximizing Return on Investment of Taxpayer Dollars”
Professor Emeritus, Pennsylvania State University
Senior Visiting Scientist, American Association for the Advancement of Science
Member, Committee on National Research Frameworks: Application to Transportation
Studies and Special Programs Division
Transportation Research Board
The National Academies
Subcommittee on Technology and Innovation
Committee on Science, Space, and Technology
U.S. House of Representatives
June 14, 2011
Mr. Chairman and Members of the Subcommittee,
It is an honor to be invited to testify before the House Committee on Science, Space, and Technology, Subcommittee on Technology and Innovation.
Over several decades and under the leadership of several distinguished committee and subcommittee chairs, the House Science, Space and Technology Committee and its Subcommittees have played a distinctive and essential role in ensuring that the Federal Government’s investments in research and development are allocated to those ends and performed in ways that maximize the taxpayer’s benefits, be these in the form of increased standards of living, safer, healthier lives, or comparable monetary and non-monetary returns. Spanning the R&D, technology transfer and related educational activities of individual Federal agencies and departments, it has assumed responsibility for monitoring the performance of major components of the larger Federal R&D enterprise. It thus has contributed and continues to significantly contribute to the vitality, productivity, and efficiency—and thus taxpayer returns—of this enterprise.
The questions that I have been asked to respond to in today’s testimony on Transportation Research Priorities are specific applications of the Committee’s larger purview. Correspondingly, my answers are specific applications to the field of surface transportation research of overarching principles and findings about how best to allocate Federal R&D funds to achieve maximum benefits and how to assess the performance of those receiving these funds.
These answers distill findings from a career as a researcher into technology transfer, evaluation of Federal and state government science and technology programs, performance measurement, and the organization of university research centers. This research has been enriched, and enlivened, by service as a member and chair of numerous Federal agency advisory and evaluation panels, related experiences on several National Research Council committees charged with studying the effectiveness of the Federal R&D programs, and similar international experiences, including advisory and consulting work for the European Commission, the Organization for Economic Development and Cooperation and several countries.
My work on technology transfer was the basis for an invitation to become a member of the Transportation Research Board’s (TRB) Research and Technology Coordinating Committee (RTCC), on which I served between September 1, 1997, and August 31, 2003. Likewise, my work on comparative national science and innovation policies has led to my current membership on the National Research Council‘s Committee on National Research Frameworks: Application to Transportation.
My answers to Questions (2)-(5) in the Committee’s invitation thus blend facts and findings that span most of the Federal Government’s domestic R&D activities, with working knowledge of specific Federal surface transportation R&D program and policy issues. My expertise regarding the technical contents of specific R&D programs and projects though is limited. My answer to Question (1), which requests a brief overview of the Transportation Board’s roles and responsibilities is taken from the TRB’s website and communications with TRB staff.
In key respects, my answers restate long recognized, well documented, and articulately expressed concerns about the shortcomings of the current system of Federal funding of surface transportation research that can be found in numerous independent, expert reports and in previous testimony before this Committee and other Congressional committees. These shortcomings include the excessive earmarking of transportation funds that limits the ability of Department of Transportation agencies to shape a coherent, sustainable national transportation R&D program and the dilution of the impact of Federal R&D research dollars associated with having to disburse them through an unduly large number of University Transportation Centers. (UTCs). Each condition drives the transportation research system to short-term, incremental research undertakings at the expense of the higher priority, longer term, more fundamental, more collaborative and thereby more impactful research topics that could be funded with the same research dollars. Simply put, current arrangements constitute systemic obstacles to garnering the maximum benefits from Federal surface transportation R&D outlays.
If there is a value-added to be found and generated by my testimony, it rests perhaps in two things. First, as evidenced by enactment of the Government Performance Results Act and recent salutary reforms already taken to reduce earmarking across the swathe of Federal government expenditures, Congress has demonstrated an increasingly unwillingness to accept inefficient budget practices and ineffective programs. Thus, on this Committee, as well as hopefully on related authorization and appropriations committee, old words may be heard by new ears.
Second, viewing transportation R&D from the enlarged cross-agency perspective that weaves through my answers offers additional, new insights into why current arrangements for funding and organizing transportation R&D are inconsistent with basic principles for justifying Federal government investments in domestic R&D. Equally importantly, as my answers detail, current arrangements are inconsistent with the internal dynamics of scientific discovery and technological innovation. It is not that some beneficial results do not emerge from current surface transportation research programs. Of course, they do. Rather, it is that the benefits are small relative to what is needed and what is possible.
(1) TRB in the National Academies
TRB is one of six major divisions of the National Research Council–a private, nonprofit institution that is the principal operating agency of the National Academies in providing services to the government, the public, and the scientific and engineering communities. The National Research Council is jointly administered by the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
TRB was created in the 1920s to be an intermediary between newly formed state highway departments and research programs and the federal government, then the Bureau of Public Roads. Since the 1980s TRB has also convened committees under the auspices of the National Research Council which advise Congress and federal agencies on transportation policy issues and evaluate and advise agency research programs.
TRB is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. Since its inception, TRB has provided opportunities for dialogue, information exchange, and sharing of research activities (to avoid duplication of effort) and research results. For more than 40 years TRB has been a multimodal research organization with activities in all modes. TRB provides an extensive portfolio of services, including:
- Information exchange on current transportation research and practice,
- Management of cooperative research and other research programs,
- Analyses of national transportation policy issues and guidance on federal and other research programs, and
- Publications and access to research information from around the world.
These activities annually engage more than 7,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest by participating on TRB committees, panels, and task forces.
Earmarks are a tax on the rate of return to the national investment in transportation R&D. Earmarks constrain the publically funded transportation research system’s ability to produce other than incremental solutions to incremental problems. Legitimate objectives to insure that the national transportation R&D portfolio take into account geographical, climatic, and demographic differences while simultaneously providing geographically dispersed opportunities for the education and training of technically trained transportation personnel do not require earmarks. These objectives can be met by far simpler, more cost-effective policies and programs.
Indeed, to someone who has only recently migrated into the domain of transportation R&D, its most striking landscape feature, as documented in several TRB reports and the overview article, “Earmarking in the US Department of Transportation Research Programs”, by Ann Brach and Martin Wachs (Transportation Research Part A 39 (2005), 501-522), is the pervasiveness of earmarking in the allocation of transportation R&D programs. Both in terms of total DOT transportation research and the allocation of UTC funds, earmarking has gone viral. Its causes have moved from the opportunistic actions of influential members of Congress to designate funds for selected purposes and/or performers within their jurisdictions to what I have termed a race to the bottom among academic administrators and faculty who increasingly “view earmarks….as an acceptable way to get financial support for projects and facilities that might not survive review procedures based on merit” (“Research Subverted by Academic Greed”, Chronicle of Higher Education, January 16, 2004; B6ff).
Only in my earlier work on the U.S. agricultural research and technology transfer system have I encounter such pervasive earmarking. This comparison, I believe, is an especially telling one in considering the future. Earmarking contributed to “same old/ same old” research agendas —some of it highly productive, some of it mundane—of agricultural researchers in the USDA’s Agricultural Research Service and university faculty receiving funds via State Agricultural Experiment Stations. It thus was a factor in the limited role of these research communities in the genetics-biotechnology scientific and technological revolution that has since transformed plant and animal research. The major scientific and technological advances that spawned this revolution came from researchers in the life sciences whose work was funded by other Federal agencies and other sources. Indeed, the backwater nature of mainstream agricultural research even gave rise to proposals, some of them originating in this Committee, for moving funding for agricultural research from USDA to other Federal agencies. Even today, the mix of formula funding and competitive awards in the funding of agricultural research remains at issue, but at least it is one openly debated, leading to some acceptance, albeit at times grudgingly, that some provision for all-comer, competitively awarded grants is needed to insure the continuing vitality and thus productivity of the underlying research.
Earmarking, by definition, is a method for circumventing the quality control contained within competitive, merit review processes. The result is not necessarily that earmarked projects do not produce useful findings. Research by definition involves uncertainty; not inconceivably, as in selecting single entries from two sample distributions, one may find some earmarked awards producing results that are at least as good or better by some standard than those allocated by competitive merit review processes. But one has every reason to expect that a comparison of overall statistics—mean, mode, lower and upper tails—will document the greater return on the public’s investment to those allocated via competitive, merit review processes.
Beyond the quality control issue, there are increasingly compelling reasons to expect this difference. These differences derive from the dynamics of faculty and institutional behavior and the dynamics of scientific and technological discovery.
Earmarking of academic R&D funds is enervating. It provides researchers and their institutions with assured funding that need not necessarily require their best efforts. It reduces the incentives faculties have to respond to new scientific or technological advances or new sources of funding. As a dean of engineering once lamented during an interview I conducted during a study of the determinants of university research competitiveness, he had little ability to motivate his faculty to seek larger, more technically challenging but competitively awarded grants because they were “comfortable” with the support for summer salaries and graduate research assistants they received from a congressional earmark.
Insulation from competitive, merit review processes also removes opportunities for constructive learning. Painful and idiosyncratic at times that it can be, and here I speak from considerable experience, running the gauntlet of competitive merit reviews can contribute significantly to the improvement of one’s work. The judgment and advice of peers about ways to reconceptualize a problem or to attend to otherwise overlooked analytical techniques or data sources are low cost ways of increasing the yield from specific projects. These opportunities are missed or diluted when earmarking occurs.
My current research into the dynamics of scientific and technological advances across many fields of endeavor highlights yet another debilitating effect of earmarking on the returns to Federal investments in transportation research. As cross-disciplinary, cross-sector/collaborative research becomes an increasingly essential element in generating significant/transformative/impactful discoveries, other Federal R&D agencies have responded by increasing their support for multi-year, multi-institutional R&D awards. A key requirement in this new mode of funding academic science is the requirement that the research program involve participation by faculties in multiple disciplines, departments, and colleges.
Cross-fertilization of ideas, techniques and discoveries are increasingly the seedbed of significant advances, not only in “basic” science but also in mission-oriented/problem-focused research. To cite but one of numerous contemporary examples, the University of California-Santa Cruz’s Center for Adaptive Optics research efforts to improve the precision of telescopes used in astronomical research also have yielded important advances in vision science that ultimately will enter into practical applications in optometry.
Existing earmarking arrangements limit these possibilities in transportation R&D. In particular, earmarking serves to isolate the transportation research community from researchers in cognate disciplines. Isolation occurs because the recipients of earmarks have little incentive to seek out or engage colleagues in disciplines whose work may enhance their own. Further contributing to this isolation is that faculties outside of the transportation field have little opportunity to extend their techniques or findings to transportation related problems. Isolation in turn contributes to making transportation R&D an academic research backwater.
Indeed, writing now clearly as a lay person and not a technical expert, I continue to be struck by the opportunities for collaboration and cross fertilization of ideas and techniques—all pointing to higher returns on the nation’s investments in R&D—in the thematic areas I have encountered during my participation in a recent assessment by the American Association for the Advancement of Science of the National Science Foundation’s Science and Technology Center’s program and those identified in several TRB reports about opportunities and needs in transportation R&D. New materials, nanotechnology, remote sensing, optics, computer software, and more, are but a few such examples of the research being conducted by these centers that appear to connect directly to the larger transportation R&D agenda. A more open, flexible and competitive transportation system that fosters such connections would contribute both to more impactful findings and by narrowing the gap between discovery-oriented and mission-oriented work also serve accelerate the incorporation of new scientific and technological discoveries into socially beneficial practices.
The essential point to my answer about earmarking is that it is mainly a restatement of views already expressed by key performers and users of transportation R&D. One of the most striking, and indeed gratifying experiences, I had as a member of the RTCC that prepared the 2001 report, The Federal Role in Highway Research and Technology (Transportation Research Board, Special Report 261) was to observe its members—representatives from industry, state government, universities, not-for-profit organizations, and professional association—advance as a core recommendation that:
“University transportation research funded under the UTC program should be subject to the same guidelines as FHWA’s R&T program—open competition, merit review, stakeholder involvement, and continuing assessment of outcomes—to ensure maximum return of the funds invested (p.9)”
Similarly, the 2008 RTCC report, The Federal Investment in Highway Research 2006-2009, offers as a recommendation:
“To the maximum extent practical, research funding should be awarded through competition and merit review” (TRB 295, p.4).
Far more important than may be my views, it is the stakeholder community that is asking for relief from current earmarking arrangements. It is they who are asking to be allowed to be all that they can be.
(3) This is a multi-part question, with my answers limited to the two areas—measuring returns to Federal investments in research and technology transfer—on which I have conducted research and professional activities. My answer to the question, “How is the value of Federal transportation measured”, is based on a general impression, augmented by a review of FHWA’s Office of Research Development and Technology’s 2007 “Synthesis of R&D Benefits Case Studies”, which reports on findings from 3 contractor studies.
At present, assessment of the value of transportation R&D appears to be based heavily on expert review panels, augmented, as above, with infrequent contractor studies. Expert review is a mainstream technique, widely used by and for Federal agencies to assess R&D programs. But it is not state-of-the art. Increasingly, both Congress and the Executive Branch are demanding or requiring that expert judgment be augmented and/or supported by “evidence”, typically of a quantitative nature.
As the author of several recent review articles on measuring the returns to Federal government R&D and as an active participant in recent workshops and forums relating to the “science of science policy research”, I recall no participation of anyone whose work bore upon transportation R&D. Noting that I have not had time to conduct a full review of the above cited contractor studies or the larger published literature, I am not presently aware of any studies related to transportation research that have employed the concepts—e.g., knowledge spillovers; social savings—or employed the methodological techniques—e.g., network analyses, patent analysis, bibliometric analysis–that are becoming standard components of efforts to measure the value of Federal R&D in other domestic domains.
The cause and consequences of this lag are circular. Without the type of evidence now being demanded of research programs in budget reviews, advocates for Federal support of transportation research are at a competitive disadvantage relative to those in other fields who have advanced beyond review panels and are tackling the admittedly formidable challenges of deriving valid and credible estimates of outcomes and impacts from what is inherently a long-term, circuitous and probabilistic process. Lacking funds, and especially discretionary funds to support policy oriented research, which is not cheap, DOT and its subunits cannot gather the type of evidence needed to make a “convincing” case about the value its research activities have generated.
My answer to the question, “ Is technology transfer from federally-funded research and development effective”, and “How could it be strengthened” is an indirect one, in part again because of what I perceive to be the limited availability of quantitative evidence that would permit program level assessments along cost-effectiveness or benefit-cost lines, and in part because the processes of technology transfer are so variable and context dependent that it is difficult to generalize from one or a few cases—successes or not—to a program level assessment.
Certainly, one can point to notable successes in technology transfer. Moving beyond the justifiably oft-cited example of SuperPave, my favorite example based on personal experience as a taxpaying consumer is the increased adoption of roundabouts. According to TRB 295, the diffusion of roundabouts was spurred by an FHWA 2000 report, Roundabouts: An Informational Guide, which is described as having lent “legitimacy and credibility to an alternative intersection design…” (p. 78) that has considerable safety benefits. I now encounter roundabouts on Route 15 crossing over between Virginia and Pennsylvania and most especially, and thankfully, on Route 179 between Oak Creek and Sedona, when my wife and I spend time there in the winter. I also believe that I am about to get a roundabout in my local neighborhood as construction continues on a new intersection between Old Gatesburg Road and Pine Hall Road in Ferguson Township, Pennsylvania.
DOT’s existing technology transfer programs consists of information dissemination, technical assistance, and demonstration projects. These are the tried and true technology transfer techniques of most Federal agencies. Thus FHWA’s Priority, Market-Ready Technologies and Innovations List which offers clear, concise, and informative information about “vetted” new technologies is one means of reducing the technical and regulatory uncertainties associated with trying new things, thereby making them more attractive to potential adopters.
What needs to be considered here is less the present than the future. The design and operation of a technology transfer system must be based on the design, operation and outputs of its parent R&D system. Whatever may be the current level of effectiveness of DOT’s technology transfer activities, a new, expanded Federal role and set of techniques will be required if the recommendations relating to the restructuring of the direction and conduct of transportation research contained in the other answers are adopted.
In particular, a shift to a system directed at longer-term, more exploratory research, especially one predicated heavily on the participation of universities, requires a broader conceptualization of meaning and implementation of technology transfer. Only sketching here the elements of such a system, added emphasis would need to be given on how the Federal government could assist in the development of university-industry-state and local government cooperative agreements or research centers that provide for closer, upfront connections between research agendas and user needs. Also, under a similar revamping of the research agendas of UTCs, additional attention would need to be given to policies and terms relating to patent and licensing arrangements between Federal labs and universities and private sector firms. Again, focusing on technology transfer from the UTCs, added attention would need to be given to the role that the placement and mobility of graduates of university research centers or of other university degree programs plays in disseminating new practices into the agencies and firms in which they work.
(4) University Transportation Centers
I am aware that in the interval between the Subcommittee’s invitation to me to address this topic and submission of my written testimony, important administrative actions have been taken by DOT to modify the program. Specifically, it is my understanding that the Research and Innovative Technology Administration (RITA) has decided to end funding for all 59 UTCs (including those selected through competition) and hold a new competition that will select a total of 20 UTCs.
Recognizing then that I address a situation much in flux, my answers relate to the previous setting, while in the process being consistent with the general thrust of RITA’s recent actions.
My recommendations for improving the University Transportation Center program essentially extend the above answers about the need to curtail the earmarking of transportation research funds to specific projects and performers, with the added observation that specific provisions of the UTC program further sap its potential to be a significant contributor to a to a vibrant national transportation R&D program. In particular, the requirement that UTCs match their federal funding with nonfederal funding on a dollar-for-dollar basis and the peanut butter spreading of program funds among such a large number of recipients cannot but serve to drive research agendas to short-term, applied projects. Basing my answer on my experience as a social
science researcher accustomed to the modest size awards offered by NSF but also as the director for 25 years of a social science research institute in which single investigators received competitive multi-year awards for several hundred thousand dollars, the $500,000 annually awarded to the earmarked Tier II schools is below the threshold needed on average to engage in a substantial, sustained research program.
The program’s 4 tier categorization serves little purpose but to insure that each state has 1 center, each doing what it states it can do best, with little regard for an integrated, priority-driven national transportation R&D agenda. Moreover, quality control checks on the program’s performance are reported as weak or lacking. According to the 2008 TRB, The Federal Investment in Highway Research, only 38 percent of the Title V UTCs are awarded their funds competitively (p. 77). More strikingly, in contrast to the increasingly rigorous evaluations already undertaken or being planned for the R&D programs of agencies such as NIH, NSF, NIST, DOE, and USDA, “There is little program oversight…for the earmarked universities” (TRB, 2008; p. 73).
Overall, whatever its initial merits as both a research and educational program, at present the UTC program is poorly designed to produce substantial returns. The program requires fundamental re-engineering based on the design principles of providing adequate funds for some smaller number of competitively selected universities so that they can engage in longer term, more fundamental research. The original design principal of one UTC in each of 10 districts should be retained, with the collective university research agenda closely linked to a clearly articulated set of national transportation R&D priorities. Rather than relying on earmarking to insure participation of other universities (and political jurisdictions), one of the selection criteria used to competitively select host institutions should be the extent to which the proposed host institution can demonstrate partnership relationships with other universities and stakeholders within the region.
The above answers not surprisingly lead to this final answer about recommended changes in the highway bill’s reauthorization of its research titles. My overarching recommendation is to deregulate transportation R&D. Existing provisions are overly restrictive, prescriptive, and inflexible. New titles should be based on setting forth broad national transportation objectives—economic productivity/competitiveness/efficiency; safety and the like, as have been identified in earlier national reports; funding for these objectives should reflect mutually arrived at agreement among Congress, the Executive, and stakeholders about the relative priorities to be assigned among these objectives along with the assessments of existing and newly consulted relevant research communities about the feasibility and the opportunities predicted for research and development; funding should be provided for a modest number of multi-year research centers in order to foster longer-term, interdisciplinary research, with awards made on the basis of competitive merit review; funding also should be provided for all-comer, unsolicited proposals directed at stated research priorities, with awards again based on competitive, merit review; and procedures should be put in place for systematic, independent, expert assessment of the quality of research and of subsequent impacts.
In one sense, these are very modest recommendations. They integrate best R&D organizational design, management practices, and evaluation procedures from across Federal agencies. They are clearly grounded in the oft expressed views of transportation R&D leaders and users across levels of government as well as the private, public and not-for-profit sectors. In another sense though they clearly are stretch goals for the Congress and for the relevant stakeholder and performer communities for they represent far reaching changes in the status quo.
They are presented here today in the view that this Subcommittee is in a unique position to substantially increase the national return on Federal investments in transportation R&D by catalyzing long recognized and much needed changes.
There are more specific recommendations to the transportation research title made in the 2008 report of the Research and Technology Coordinating Committee referenced earlier. Although I was no longer a member of the committee when this report was developed, it is relevant to your work and I recommend that the Subcommittee request a briefing on it from the Transportation Research Board.
Thank you, Mr. Chairman
An archived webcast of the hearing can be found on the House Science, Space and Technology Committee's Web site.