A Review of NASA Human Research Program's Scientific Merit Assessment Processes: Letter Report (2012)
Chapter: Appendix A: Workshop Agenda and List of Participants
A
Workshop Agenda and List of Participants
INSTITUTE OF MEDICINE
National Academy of Sciences
Committee on the Review of NASA Human Research Program’s Scientific Merit Assessment Processes
Workshop on Evaluating the Scientific Merit of Directed Research* Proposals
March 28, 2012
20F Conference Center
Washington, DC
Focus: This workshop is focused on scientific merit assessment processes, and specifically those processes used to assess directed research.
*Directed research is defined for the purposes of this workshop as commissioned or noncompetitively awarded research that is not competitively solicited because of specific reasons, such as time limitations or highly focused or constrained research topics. The research topic may be identified by the sponsor or by submission of an unsolicited proposal from external researchers.
Workshop Format: Institute of Medicine (IOM) committee members will facilitate roundtable discussions. Invited speakers, National Aeronautics and Space Administration (NASA) staff, and committee members will be at the table and will use preselected discussion questions as a starting point for discussion. Invited speakers will be asked to provide information on the directed research and scientific merit processes used in their organization prior to the workshop.
AGENDA
| 8:30 a.m. |
Welcome, Introductions, and Organization of the Workshop |
|
|
James Pawelczyk, Chair |
||
| 9:00 |
NASA Human Research Program’s Scientific Merit Assessment Processes |
|
|
John Charles, NASA |
||
| 9:30 |
Roundtable Discussion 1: Identifying Directed Research |
|
|
Facilitator: Carol Scott-Conner |
||
|
Discussion Questions: |
||
|
• Does your agency or organization fund directed research for any part of its research portfolio? |
||
|
• How does your organization determine what projects constitute directed research projects? What are the criteria? Who makes the decision that it should be directed research (“authorization to review”)? |
||
|
• Do you consider directed research concepts from both internal and external stakeholders? |
||
| 10:30 |
Break |
| 10:45 a.m.-12:15 p.m. |
Roundtable Discussion 2: Scientific Merit Assessment |
|
|
Facilitator: Ian Graham |
||
|
Discussion Questions: |
||
|
• What type(s) of scientific merit assessment review does your organization use to assess directed research proposals? How does it differ from your process for assessing scientific merit for other types of research? |
||
|
• What are the strengths and weaknesses of each type of review? |
||
|
• What metrics are used in the review to evaluate scientific merit? |
||
|
• Who decides that a research project should be approved and that the research should be initiated (“authorization to proceed”)? |
||
| 12:15 |
Lunch |
|
| 1:00 |
Roundtable Discussion 3: Evaluation and Quality Improvement |
|
|
Facilitator: Kathie Olsen |
||
|
Discussion Questions: |
||
|
• Monitoring progress and evaluating project results: |
||
|
◦ How is the progress of a directed research project monitored over the course of the project? How is the success of each project evaluated once it is complete? |
||
|
◦ What metrics are used and over what period of time? (e.g., Did the researcher deliver the product on time? Did the result meet the needs of the requestor?) |
||
|
• Evaluating the directed research process: |
||
|
◦ How is the overall quality of directed research process evaluated? |
|
o What metrics are used and with what frequency? Who conducts the evaluation? |
||
|
o Do researchers provide feedback about the process and if so, how is that feedback incorporated? Are other quality improvement processes used? |
||
|
o What is done to ensure transparency of the process? |
||
| 2:30 |
Break |
|
| 2:45 |
Roundtable Discussion 4: Dialogue with NASA Sponsors |
|
|
Facilitator: James Pawelczyk, Chair |
||
|
Discussion about NASA’s directed research processes |
||
| 4:15 |
Closing Remarks |
|
|
James Pawelczyk, Chair |
||
| 4:30 p.m. |
Adjourn |
LIST OF PARTICIPANTS
IOM Committee Members
James A. Pawelczyk, Chair
Associate Professor of Physiology, Kinesiology and Medicine
Pennsylvania State University
Michelle H. Biros
Professor and Vice Chair of Research
Department of Emergency Medicine
Hennepin County Medical Center
University of Minnesota Medical School
Divya Chandra
Principal Technical Advisor
Volpe National Transportation Systems Center
Department of Transportation
Ian D. Graham
Associate Professor
University of Ottawa
Chavonda Jacobs-Young
Associate Administrator
Agricultural Research Service
U.S. Department of Agriculture
Kathie L. Olsen
Founder, Managing Director
ScienceWorks, LLC
Terry M. Rauch
Program Director for Defense Medical Research and Development
Office of the Assistant Secretary of Defense for Health Affairs
Department of Defense
Sally J. Rockey
NIH Deputy Director for Extramural Research
Office of the Director
National Institutes of Health
Carol E. H. Scott-Conner
Professor of Surgery
University of Iowa Hospitals and Clinics
Peter Suedfeld
Professor Emeritus of Psychology
Department of Psychology
University of British Columbia
Invited Participants
Kenneth Baldwin
Professor of Physiology and Physics
University of California, Irvine, School of Medicine
Wayman Wendell Cheatham
Director of the Naval Medical Research and Development Center
U.S. Navy
David Dinges
Professor of Psychology in Psychiatry
University of Pennsylvania School of Medicine
Karl Friedl
Director of the Telemedicine and Advanced Technology Research Center
U.S. Army Medical Research and Materiel Command
Paul Krois
Manager of the Human Factors Research and Engineering
Division
Federal Aviation Administration
James Lightbourne
Senior Advisor
National Science Foundation
Adrian Mota
Manager of Knowledge Translation Initiatives
Canadian Institutes of Health Research
Tim O’Leary
Director of the Clinical Science Research and Development Service
Department of Veterans Affairs
Sonia Ortega
Program Director, Graduate STEM Fellows in K-12 Education Program
National Science Foundation
Alfred M. Powell, Jr.
Director of the Center for Satellite Applications and Research
National Oceanic and Atmospheric Administration
Jay Schnitzer
Director of the Defense Sciences Office
Defense Advanced Research Projects Agency
Sharlene Weatherwax
Associate Director of Science for Biological and Environmental Research
Department of Energy
Edward Zambraski
Division Chief of the Military Performance Division
U.S. Army Research Institute of Environmental Medicine
Sponsor Participants
Gale Allen
Science Mission Directorate
NASA
John Charles
Chief Scientist of the Human Research Program
NASA
Marisa Covington
Wyle Integrated Science and Engineering Group
Craig Kundrot
Deputy Chief Scientist of the Human Research Program
Manager of the Human Research Program Science Management Office
NASA
Marc Shepanek
Deputy Chief of Medicine of Extreme Environments
NASA
This page intentionally left blank.
