Skip to main content

Evaluation of the Advanced Research Projects Agency – Energy (ARPA-E)

Completed

A committee under the auspices of the Board on Science, Technology, and Economic Policy undertook a detailed and in-depth examination of the Advanced Research Projects Agency – Energy (ARPA-E). The resulting report, An Assessment of ARPA-E, appraises the appropriateness and effectiveness of ARPA-E's processes and operations to achieve its mission and goals. It also presents a retrospective and technical assessment of ARPA-E. The study’s committee of experts relied on quantitative and qualitative analyses to develop a set of findings and recommendations to the agency and to Congress.

Description

The Advanced Research Projects Agency-Energy (ARPA-E) was established in 2009 for the stated purpose of funding energy technology projects by “identifying and promoting revolutionary advances in fundamental and applied sciences; translating scientific discoveries and cutting-edge inventions into technological innovations; and accelerating transformational technological advances in areas that industry by itself is not likely to undertake because of technical and financial uncertainty.”

An ad hoc committee will review and evaluate the progress that ARPA-E has made toward achieving its goals. The committee will develop a methodology and framework for an operational assessment as well as a retrospective and technical assessment of ARPA-E and then conduct both. The operational assessment will appraise the appropriateness and effectiveness of ARPA-E's structure to position it to achieve its mission and goals. Specifically, the committee will:

  • Evaluate ARPA-E's methods and procedures to develop and evaluate its portfolio of activities;
  • Examine appropriateness and effectiveness of ARPA-E programs to provide awardees with non-technical assistance such as practical financial, business, and marketing skills;
  • Assess ARPA-E's recruiting and hiring procedures to attract and retain qualified key personnel;
  • Examine the processes, deliverables, and metrics used to assess the short and long term success of ARPA-E programs;
  • Assess ARPA-E's coordination with other Federal agencies and alignment with long-term DOE objectives;
  • Evaluate the success of the focused technology programs to spur formation of new communities of researchers in specific fields;
  • Provide guidance for strengthening the agency's structure, operations, and procedures;
  • Evaluate, to the extent possible and appropriate, ARPA-E’s success at implementing successful practices and ideas utilized by the Defense Advanced Research Projects Agency (DARPA), including innovative contracting practices; and
  • Reflect, as appropriate, on the role of ARPA-E, DOE, and others in facilitating the culture necessary for ARPA-E to achieve its mission.

The committee's retrospective assessment and technical evaluation will describe the most significant accomplishments and/or impacts of the ARPA-E program to date and any unique features of the program that may have contributed to these accomplishments, and consider how well ARPA-E's activities have supported the agency's goals. Specifically, the committee will:

  • Analyze specific focused technology programs and/or awardee projects to determine whether ARPA-E is focused on truly “transformational” technologies;
  • Examine funded projects to assess and whether ARPA-E solicitations attract the best and brightest to the relevant program areas;
  • Identify where ARPA-E projects have made an important difference; and
  • Assess performance of the portfolio of awards and evaluate whether and how much the award portfolio has helped ARPA-E achieve its goals.

Developing new energy technologies often requires several decades, while ARPA-E has operated for five years. This will likely limit the quantity and quality of available data and other evidence of long-term outcomes such as commercialization of new technologies. If and where the committee finds a paucity of data or other evidence, the committee will assess progress to date and also identify the most valuable data and information that ARPA-E could gather going forward.

The committee will issue a final report detailing its findings, conclusions, and recommendations, including suggestions for improvements in ARPA-E, lessons learned from the operation of ARPA-E that may apply to other DOE programs, and factors that Congress should take into consideration in determining the future of ARPA-E.

Collaborators

Committee

Chair

Vice Chair

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Member

Download all bios

Committee Membership Roster Comments

Peidong Yang added on 7/20/2015.
There has been a change in committee membership with the resignation of Dr. Jay Whitacre (9/20/2016)

Sponsors

Department of Energy

Staff

Paul Beaton

Lead

Gail Cohen

David Dierksheide

Subscribe to Email from the National Academies
Keep up with all of the activities, publications, and events by subscribing to free updates by email.