Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs (2013)
Chapter: Appendix D: Program Evaluation Information Provided by the Agencies
Visit
NAP.edu/10766
to get more information about this book, to buy it in print, or to
download it as a free PDF.
Previous Chapter:
Appendix C: Workshop Participants
Next Chapter: Appendix E: Biographical Sketches of Committee Members
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| USGS National Cooperative Geologic Mapping Program (EdMap) | • Train students in geologic mapping • Help fund academic research • Prepare students for real-world careers in the geosciences • Create collaborations between academic institutions, state geological surveys, and the USGS • Contribute to national geologic mapping efforts |
• Survey of participants and providers | • Number of students educated in geologic mapping techniques | • Program benefits participants |
| USGS/NAGT Cooperative Summer Field Training Program | • Provide an opportunity to partner a highly able intern with a quality science mentor to work on a meaningful research project • Present research results at professional meetings or in refereed publications |
• Surveys of participants and providers | • Number of participants • Success of participants in earth science careers |
• Excellent interns are placed in research units • Program quality is recognized by USGS and earth science academic community |
| USGS Youth Internship Program | • Stimulate ongoing interest in science among college undergraduates • Build a pool of well-prepared new college graduates who could fill vacancies in the USGS |
• Reports by participants and providers | • Fraction of participants that obtained permanent jobs • Program satisfaction |
• Most available slots are filled • Many interns accept positions at USGS |
| USGS Hydrologic Technician Internship Program | • Stimulate ongoing interest in water science among college undergraduates • Build a pool of well-prepared new college graduates who could fill vacancies in the USGS |
• Reports by participants and providers | • Fraction of participants that obtained permanent jobs • Program satisfaction |
• Most available slots are filled • Many interns accept positions at USGS |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| NSF Earth Sciences Research Experience for Undergraduates (REU) Program | • Support research participation by undergraduate students | • External, formal evaluation in 2006 • Reports by providers |
• Number of applicants to REU sites • Number of former participants in an earth science career |
• Interest of undergraduate students |
| NSF Opportunities for Enhancing Diversity in the Geosciences (OEDG) Program | • Increase participation in the geosciences by African Americans, Hispanic Americans, Native Americans, Native Pacific Islanders, and persons with disabilities • Increase the perceived relevance of the geosciences among broad and diverse segments of the population |
• External evaluation by an expert consulting firm using a logic model • External reviews every 3 years by a visiting committee • Reports by providers |
• None specified | • New degree programs at minority-serving institutions (MSIs) • New partnerships between MSIs and 4-year and research universities • Better understanding how to recruit and retain minorities in earth science • Slight increase in number of undergraduate majors |
| NSF Earth Sciences Postdoctoral Fellowships | • Recognize beginning investigators of significant potential • Provide these individuals with research and education experience that will establish them in positions of leadership in the scientific community |
• External reviews every 3 years by a visiting committee • Reports by participants on research advances • Information on career development of former fellows |
• None stated | • A high fraction of participants are in tenure-track positions • Participants are active scientifically • Some participants have become mentors |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| NSF Geoscience Education (GeoEd) Program | • Improve the quality and effectiveness of formal and informal geoscience education at all educational levels • Increase the number of students pursuing geoscience education and career paths • Broaden participation of traditionally underrepresented groups in the geosciences • Promote public engagement in Earth system science |
• External reviews every 3 years by a visiting committee • Reports by providers |
• Widespread use of models for understanding geoscience content and strategies for developing spatial and systems-thinking skills • Number of peer-reviewed publications • Continued research support |
• Incubation of the Digital Library for Earth System Education • Incorporation of big ideas that all citizens should know into the next-generation science standards • Creation of geoscience educators who use best practices from STEM education research to strengthen formal and informal education |
| NSF/NOAA/NASA Global Learning and Observations to Benefit the Environment (GLOBE) Program | • Improve student understanding of environmental and Earth system science across the curriculum • Contribute to scientific understanding of Earth as a system • Build and sustain a global community of students, teachers, scientists, and citizens • Engage the next generation of scientists and global citizens in activities to benefit the environment |
• Internal evaluation (not specified) | • None specified | • A large number of K–16 students are engaged worldwide • A large community of teachers with improved pedagogy and content knowledge |
| NSF Geoscience Teacher Training (GEO-Teach) Program | • Improve the quality of geoscience instruction, primarily at middle and high school levels | ESSEA project: • Internal evaluation based on data from course site, participant surveys and products, and provider surveys and products |
ESSEA project: • Resources developed and how much they have been used • Effect of courses on learner’s knowledge of Earth system science and how to |
ESSEA project: • Institutionalization of effective professional development and training methods over many universities • Funding to create more modules |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| teach it • Effect of the program on faculty and their institutions |
||||
| TESSE project: • External evaluation planned based on participant surveys, focus groups, phone interviews, observation of activities, and standardized teacher assessment |
TESSE project: • Extent to which project goals, activities, and outcomes are understood by providers and participants • Extent to which program activities met stated goals • Extent to which teacher opportunities improved their understanding of earth science and ability to teach it • Success of processes and structures to improve collaboration |
TESSE project: • Identification of challenges in extending a professional development model from research institutions to HBCUs |
||
| DOE Office of Science Graduate Fellowship (SCGF) Program | • Encourage the development of the next generation of scientific and technical talent in the United States who will pursue careers in research critical to the Office of Science mission | • Program goals, outputs, and outcomes defined in a logic model • Reports by providers and participants • Longitudinal tracking of participants • External review of WDTS portfolio in 2010 • External reviews every 3 years by a visiting committee |
Long-term outcomes include • Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry |
• Program is too young to determine. The first cohort is nearing the end of the 3-year support period |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| DOE Summer of Applied Geophysical Experience (SAGE) Program | • Introduce students in geophysics and related fields to “hands-on” geophysical exploration and research • Apply research and geophysical exploration methods to solve specific problems |
• Reports by participants and providers | • Rate of continuation to graduate school and careers in geophysics • Support and feedback from industry participants and visitors • Presentations at professional meetings and publications |
• None specified |
| DOE Science Undergraduate Laboratory Internships (SULI) Program | • Encourage undergraduate students to pursue STEM careers | • Program goals, outputs, and outcomes defined in a logic model • Pre- and postparticipation surveys of participants • Longitudinal tracking of participants • External review of DOE laboratories’ execution of the program • External review of WDTS portfolio in 2010 • External reviews every 3 years by a visiting committee |
Near-term outcomes include • Completion of STEM degree • Pursuit of advanced degree in STEM • Preparedness for STEM career • Knowledge of DOE mission science and technology Long-term outcomes include • Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry |
• More than 90 percent of participants report that the internship helped prepare them for a STEM career • More than three-quarters of participants (2003-2011) are pursuing a career in STEM |
| DOE Community College Internship (CCI) Program | • Encourage community college students to enter technical careers relevant to the DOE mission | • Program goals, outputs, and outcomes defined in a logic model • Pre- and postparticipation surveys of participants • Longitudinal tracking of participants • External review of DOE laboratories’ execution of the program |
Near-term outcomes include • Completion of STEM degree • Pursuit of certification in STEM • Preparedness for STEM career • Knowledge of DOE mission science and technology |
• None specified |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| • External review of WDTS portfolio in 2010 • External reviews every 3 years by a visiting committee |
Long-term outcomes include • Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry |
|||
| NASA Gravity Recovery and Climate Experiment (GRACE) programs | None specified. Proposed outcomes are: • Conduct GRACE activities with 500 teachers with priority given to educators from underrepresented areas or with underrepresented audiences • Increase the knowledge of 500 secondary teachers about the Earth’s history, Earth system science, and global climate change • Increase the knowledge of 500 students about earth science and global climate change • Make available selected activities of “changing mass = changing earth” programs through the University of Texas Center for Space Research Web site • Incorporate NASA satellite data and resources into the activities |
• Reports by providers • External evaluations are planned |
• Number of teacher and student participants • Field trip opportunities for students • Increase in knowledge of global climate change • Student interest in STEM careers |
• None specified |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| USDA 4-H Environmental Education/Earth Science programs | • Increase awareness of science among youth • Improve science skills and knowledge among youth • Increase awareness of opportunities to contribute to society using science skills • Increase life skills among youth |
• Internal evaluation (not specified) | • None specified | • 4-H enrollment • Professional development has positive impact on performance and satisfaction • High-quality programs • Creation of a new research database • Participants excel beyond their peers |
| USDA Agriculture and Food Research Initiative (AFRI) projects | • Increase technical competency in priority area(s) to ensure that the United States remains globally competitive in the knowledge age | • None yet | • Number of participants | • Participants complete their degree |
| USDA National Institute of Food and Agriculture Fellowships Grant Program | • Strengthen the ability of the nation’s scientific community to meet the current and future challenges facing agriculture, forestry and food systems • Develop the technical and academic competence of doctoral candidates • Develop the research independence and teaching credentials of postdoctoral scientists |
• Reports by providers | • None specified (project dependent) | • None specified (project dependent) |
| EPA Greater Research Opportunities (GRO) Undergraduate Fellowship Program | • Support undergraduate study in environmentally related fields | • External review by a visiting committee in 2009 • Reports by participants |
• Number of awards • Number of completed degrees • Topic area distribution • Professional success of former fellows |
• None specified |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| EPA Science to Achieve Results (STAR) Graduate Fellowship Program | • Defray the cost of study leading to advanced degrees in environmental science fields | • External review by a visiting committee in 2009 • Reports by participants • NRC review of STAR in 2003 |
• Number of awards • Number of completed degrees • Topic area distribution • Professional success of former fellows |
• None specified |
| NOAA Educational Partnership Program (EPP) With Minority Serving Institutions | Cooperative Science Centers: • Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission • Increase collaborative research efforts between NOAA scientists and researchers at NOAA EPP cooperative science centers |
Cooperative Science Centers: • External review of cooperative science centers in their third year • NRC review of education program in 2010 • Surveys of providers and participants |
Cooperative Science Centers: • Annual number of students from underrepresented communities who are trained and graduate in NOAA mission sciences • Annual number of students who are trained and graduate in NOAA mission sciences • Number of students completing experiential opportunities at NOAA facilities • Number of students who are hired by NOAA; NOAA contractors; other environmental, natural resource, and science agencies; academia; and the private sector • Number of collaborative research projects undertaken between NOAA and EPP cooperative science centers in support of NOAA operations • Number of students and faculty who participate in and complete postdoctoral-level research programs in support of the NOAA mission |
Cooperative Science Centers: • None specified |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| • Number of peer-reviewed papers published in NOAA mission sciences by faculty, postdoctoral fellows, and students sponsored by NOAA EPP • Funds leveraged with NOAA EPP funds • Number of participants engaged in NOAA mission-relevant learning opportunities |
||||
| Graduate Sciences Program: • Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission |
Graduate Sciences Program: • NRC review of education program in 2010 • Surveys of providers and participants |
Graduate Sciences Program: • Number of graduate sciences program students hired by NOAA |
Graduate Sciences Program: • None specified |
|
| Undergraduate Scholarship Program: • Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission |
Undergraduate Scholarship Program: • NRC review of education program in 2010 • Surveys of providers and participants |
Undergraduate Scholarship Program: • Number of undergraduate scholarship students who are educated, trained, and graduate in NOAA mission sciences • Number of undergraduate scholarship students who pursue graduate work in NOAA mission sciences |
Undergraduate Scholarship Program: • None specified |
|
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| • Number of undergraduate scholarship students who are hired by NOAA, NOAA contractors, and other natural resources and science agencies | ||||
| NPS/GSA Geoscientists-inthe-Parks Program | • Provide on-the-job earth science training for America’s youth • Build technical capacity for parks • Enhance the public’s understanding of earth science |
• Reports by providers | • Completion of unmet earth science projects in NPS units • Communication of earth science to the public and park staff who are not geologists • Program growth |
• Program has grown from 20 positions per year in 2007 to more than 100 positions per year in 2012 |
| NPS/NAGT Geoscience-Teachers-in-Parks Program | • Provide cooperative exchange of learning and scientific research between the park, local earth science teachers, and communities • Advance educational and interpretive opportunities at the park • Develop a lifelong network with local communities, schools, and the park |
• Participant feedback | • None specified | • Teachers gain career enhancement opportunities • Program increases the recognition of NAGT as a leader in promoting earth science education • Teachers sign up as park volunteers and return to the park after completing their internship • Products developed during the internship can be used by park staff for other educational purposes • Teachers continue using the park as an educational tool or destination • Students become more familiar with the park as an educational resource and more aware of the need to protect park resources • Teachers and students become stewards of parks and the environment |
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
| Program | Goals | Evaluation | Criteria for Success | Successes |
| NPS National Fossil Day | • Promote public awareness and stewardship of fossils • Foster a greater appreciation of the scientific and educational value of fossils |
• Survey of participants | • None specified | • Rapid expansion of partners embracing the mission • Positive feedback from partners and the media |
| Smithsonian Leadership and Assistance for Science Education Reform (LASER) Program | • Improve K–12 science education | • External reviews of the program and its components are planned | • Increased student achievement | • Statewide implementation of program (e.g., Washington state LASER) |
NOTE: DOE = Department of Energy; EPA = Environmental Protection Agency; ESSEA = Earth System Science Education Alliance; GSA = Geological Society of America; HBCUs = historically Black colleges and universities; MSI = minority-serving institution; NAGT = National Association of Geoscience Teachers; NASA = National Aeronautics and Space Administration; NOAA = National Oceanic and Atmospheric Administration; NPS = National Park Service; NSF = National Science Foundation; STEM = science, technology, engineering, and mathematics; TESSE = Transforming Earth Systems Science Education; USDA = U.S. Department of Agriculture; USGS = U.S. Geological Survey; WDTS = Office of Workforce Development for Teachers and Scientists.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Suggested Citation:
"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press.
doi: 10.17226/18369.
Subscribe to Email from the National Academies
Keep up with all of the activities, publications, and events by subscribing to free updates by email.