Evaluation of the Minerva Research Initiative (2020)
Chapter: Appendix E: Survey of Minerva Grantees
Appendix E
Survey of Minerva Grantees
DATA COLLECTION METHODS
The grantee survey was a census of principal investigators (PIs) for the 102 Minerva Research Initiative grants awarded between 2008 and 2017. In the case of grants with co-PIs, one PI was selected to participate in the survey per grant. The survey was conducted by the National Opinion Research Center (NORC) at the University of Chicago between August 17 and October 1, 2018. Grantees received an email invitation to complete the survey via web. Reminders were sent by email and FedEx. Of the 102 grantees, 76 answered the survey questions, and an additional 3 answered them partially, which resulted in a 77 percent completion rate. As part of the same request, grantees were also asked to submit lists of their research outputs based on the Minerva grant. Of the 79 grantees who answered survey questions, 67 (85%) provided a list of outputs. For further discussion of the grantee survey, see Chapter 2.
FREQUENCY DISTRIBUTIONS
All results reported here show percentages among all respondents, unless otherwise noted. SKP refers to items skipped by the respondent. The survey was designed to allow respondents to skip any item they did not wish to answer. In the case of questions with a series of “Yes/No” items, missing responses were recoded as “No” in cases in which the respondent selected at least one “Yes” response (see Chapter 2 for additional detail).
Q1. Prior to applying for a Minerva grant, did you learn about the Minerva grant program in any of the following ways?
| Yes | No | SKP | |
|---|---|---|---|
| University research office | 29 | 71 | - |
| Department of Defense (DoD) website | 37 | 63 | - |
| National Science Foundation | 22 | 78 | - |
| At a conference | 34 | 66 | - |
| From a colleague | 79 | 21 | - |
| Mailing List | 21 | 79 | - |
| Other, please specify | 11 | 89 | - |
N=76
Q2. Do you have any experience as an investigator with federal social science grant programs other than Minerva?
| Yes | 71 |
| No | 29 |
| SKP | - |
N=76
If Yes to Q2
Q3. Do you have experience as an investigator with any of the following federal social science grant programs?
| Yes | No | SKP | |
|---|---|---|---|
| National Science Foundation grants | 76 | 24 | - |
| Department of Homeland Security grants | 17 | 83 | - |
| Grants, other than Minerva, from the DoD service branches (e.g., Air Force, Army, or Navy) | 35 | 65 | - |
| Other federal social science grants, please specify | 46 | 54 | - |
N=54
If Yes in Q3a
Q4. How satisfied are you with the following aspects of the Minerva grant program compared to National Science Foundation grants?
[HALF SAMPLE ASKED RESPONSE OPTIONS IN REVERSE ORDER; “MUCH/SOMEWHAT” COLUMNS COMBINE RESPONSES FROM THE SEPARATE “MUCH” AND “SOMEWHAT” COLUMNS]
| Much/Somewhat less satisfied | Much less satisfied | Somewhat less satisfied | About the same | Somewhat more satisfied | Much more satisfied | Much/Somewhat more satisfied | Unable to compare this aspect | SKP | |
|---|---|---|---|---|---|---|---|---|---|
| Selection of important topics for research | 10 | 2 | 7 | 37 | 24 | 29 | 54 | - | - |
| White paper process | 12 | 2 | 10 | 32 | 17 | 20 | 37 | 20 | - |
| Full Proposal submission process and requirements | 12 | 2 | 10 | 56 | 15 | 15 | 29 | 2 | - |
| Communication during the proposal stage | 12 | 2 | 10 | 51 | 12 | 22 | 34 | 2 | - |
| Post-award grant management (e.g., incremental funding, modifications, no cost extensions, compliance with terms and conditions, etc.) | 15 | 5 | 10 | 54 | 12 | 12 | 24 | 7 | - |
| Institutional Review Board requirements | 22 | 10 | 12 | 56 | 10 | 2 | 12 | 10 | - |
| Financial and narrative grant reporting requirements | 15 | 2 | 12 | 68 | 10 | 7 | 17 | 2 | - |
| Post-award communication | 15 | 5 | 10 | 37 | 22 | 22 | 44 | 5 | - |
| Assistance with dissemination or translation of research findings | 7 | 5 | 2 | 56 | 7 | 15 | 22 | 15 | - |
N=41
If Yes in Q3b
Q5. How satisfied are you with the following aspects of the Minerva grant program compared to Department of Homeland Security grants?
[HALF SAMPLE ASKED RESPONSE OPTIONS IN REVERSE ORDER; “MUCH/SOMEWHAT” COLUMNS COMBINE RESPONSES FROM THE SEPARATE “MUCH” AND “SOMEWHAT” COLUMNS]
| Much/Somewhat less satisfied | Much less satisfied | Somewhat less satisfied | About the same | Somewhat more satisfied | Much more satisfied | Much/Somewhat more satisfied | Unable to compare this aspect | SKP | |
|---|---|---|---|---|---|---|---|---|---|
| Selection of important topics for research | 11 | - | 11 | 22 | 44 | 11 | 56 | 11 | - |
| White paper process | 11 | - | 11 | 11 | 33 | 22 | 56 | 22 | - |
| Full Proposal submission process and requirements | 22 | - | 22 | 22 | 33 | 11 | 44 | 11 | - |
| Communication during the proposal stage | 22 | 11 | 11 | 33 | 22 | 11 | 33 | 11 | - |
| Post-award grant management (e.g., incremental funding, modifications, no cost extensions, compliance with terms and conditions, etc.) | - | - | - | 33 | 33 | 11 | 44 | 22 | - |
| Institutional Review Board requirements | 33 | 11 | 22 | 33 | 11 | - | 11 | 22 | - |
| Financial and narrative grant reporting requirements | 11 | - | 11 | 44 | 22 | - | 22 | 22 | - |
| Post-award communication | 11 | - | 11 | 22 | 44 | - | 44 | 22 | - |
| Assistance with dissemination or translation of research findings | 11 | - | 11 | 33 | 11 | - | 11 | 44 | - |
N=9
If Yes in Q3c
Q6. How satisfied are you with the following aspects of the Minerva grant program compared to grants from other DoD service branches (e.g., Air Force, Army, or Navy)?
[HALF SAMPLE ASKED RESPONSE OPTIONS IN REVERSE ORDER; “MUCH/SOMEWHAT” COLUMNS COMBINE RESPONSES FROM THE SEPARATE “MUCH” AND “SOMEWHAT” COLUMNS]
| Much/Somewhat less satisfied | Much less satisfied | Somewhat less satisfied | About the same | Somewhat more satisfied | Much more satisfied | Much/Somewhat more satisfied | Unable to compare this aspect | SKP | |
|---|---|---|---|---|---|---|---|---|---|
| Selection of important topics for research | 5 | - | 5 | 58 | 11 | 26 | 37 | - | - |
| White paper process | 11 | - | 11 | 58 | 11 | 21 | 32 | - | - |
| Full Proposal submission process and requirements | 16 | - | 11 | 58 | 11 | 16 | 26 | - | - |
| Communication during the proposal stage | 11 | - | 11 | 74 | 5 | 11 | 16 | - | - |
| Post-award grant management (e.g., incremental funding, modifications, no cost extensions, compliance with terms and conditions, etc.) | 11 | - | 11 | 53 | 16 | 5 | 21 | 16 | - |
| Institutional Review Board requirements | 11 | - | 11 | 68 | 11 | 5 | 16 | 5 | - |
| Financial and narrative grant reporting requirements | 11 | - | 11 | 53 | 16 | 16 | 32 | 5 | - |
| Post-award communication | 16 | 5 | 11 | 68 | 5 | 5 | 11 | 5 | - |
| Assistance with dissemination or translation of research findings | 5 | - | 5 | 58 | 11 | 5 | 16 | 21 | - |
N=19
Q7. For each of the following activities, did the Minerva program greatly increase, somewhat increase, or not increase at all your opportunities?
[HALF SAMPLE ASKED RESPONSE OPTIONS IN REVERSE ORDER; “MUCH/SOMEWHAT” COLUMNS COMBINE RESPONSES FROM THE SEPARATE “MUCH” AND “SOMEWHAT” COLUMNS]
| Greatly increased opportunities | Somewhat increased opportunities | Did not increase at all opportunities | Not applicable | SKP | |
|---|---|---|---|---|---|
| Pursuing research in new directions related to the national security topics funded by the Minerva program | 76 | 20 | 1 | 3 | - |
| Expanding networks with other researchers interested in national security research | 49 | 46 | 4 | 1 | - |
| Participating in interdisciplinary and cross-disciplinary research | 46 | 37 | 16 | 1 | - |
| Providing training opportunities for students and postdoctoral scholars/fellows | 53 | 36 | 5 | 7 | - |
| Interacting with service branch staff (e.g., Air Force, Army, or Navy) interested in integrating basic research insights into their work | 26 | 42 | 28 | 4 | - |
| Interacting with other DoD staff | 26 | 38 | 30 | 5 | - |
| Interacting with national security policy staff in other federal agencies | 22 | 37 | 34 | 7 | - |
| Interacting with policymakers in the legislative branch (e.g., through congressional testimony, meetings with staff or members) | 13 | 29 | 50 | 8 | - |
N=76
Q8. How many students or fellows were actively involved in your Minerva grant(s) for at least one academic quarter or semester? Please enter a number.
[OPEN-ENDED RESPONSE]
| Median | SKP | |
|---|---|---|
| Undergraduate students | 3 | 9 |
| Graduate students | 4 | 4 |
| Postdoctoral fellows/scholars | 1 | 14 |
N=76
Q9a. Did your Minerva grant(s) result in any products such as publicly available software, websites, databases, patents, licenses, or training materials? Please do not include publications or presentations as we will ask about those later.
For discussion of the results, see report text.
If Yes on Q9a
Q9b. Please list any products such as publicly available software, websites, databases, patents, licenses, or training materials that resulted from your Minerva grant(s).
[OPEN-ENDED RESPONSE]
For discussion of the results, see report text.
Q10. Has your support from the Minerva proçgram led to additional funding for research that builds on your Minerva funded work from any of the following sources?
| Yes | No | SKP | |
|---|---|---|---|
| Received additional funding from the Minerva program | 38 | 61 | 1 |
| Received additional non-Minerva funding from DoD | 17 | 82 | 1 |
| Received additional funding from another source | 41 | 58 | 1 |
N=76
If Yes on Q10c
Q11. Did you receive additional funding for research that builds on your Minerva funded work from any of the following sources?
| Yes | No | SKP | |
|---|---|---|---|
| National Science Foundation | 23 | 77 | - |
| Department of Homeland Security | - | 100 | - |
| Other, please specify | 65 | 35 | - |
N=31
Q12. Would you say the Minerva grant program has had a positive impact, no impact, or a negative impact on each of the following. . .
[HALF SAMPLE ASKED RESPONSE OPTIONS IN REVERSE ORDER]
| Positive impact | No impact | Negative impact | Unable to say | SKP | |
|---|---|---|---|---|---|
| The amount of dialogue between DoD and the social science research community as a whole? | 87 | 5 | - | 8 | - |
| The number of social science researchers with interest in national security research? | 82 | 8 | - | 11 | - |
| The amount of collaboration among researchers working on different national security research topics? | 76 | 11 | - | 13 | - |
N=76
Q13. What challenges do you face in conducting unclassified research relevant to national security that are different from the challenges you face in conducting research in other areas?
[OPEN-ENDED RESPONSE]
| Coded Open-Ended Responses | |
|---|---|
| No challenges | 22 |
| Other | 17 |
| Onerous IRB/human subject review process | 12 |
| Criticism from academic colleagues due to DoD funding | 12 |
| Access to data/ability to collect data | 8 |
| Lack of interest/understanding among other scholars | 7 |
| Reluctance of research subjects to work with American researcher or DoD grantee | 7 |
| Lack of access to classified materials | 5 |
| Lack of adoption of findings by military leaders and national security stakeholders | 5 |
| Lack of interest/understanding of social science by military leaders and national security stakeholders | 4 |
| Lack of dissemination opportunities | 4 |
| Politicization of work | 1 |
| Lack of time to conduct research | 1 |
| SKP | 21 |
N=76
Q14. Describe any changes you would like to see to the Minerva program. [OPEN-ENDED RESPONSE]
| Coded Open-Ended Responses | |
|---|---|
| Other | 21 |
| Increased funding/longer grant cycles | 14 |
| Better visibility of work/dissemination opportunities | 14 |
| More cross-project collaboration | 12 |
| None | 9 |
| Opportunities for applied research | 9 |
| Better funding allocation/timing | 8 |
| Lower administrative burden | 5 |
| Feedback on initial concepts, white papers | 5 |
| More notice for call for papers | 4 |
| Funding of ongoing projects post-award | 4 |
| Less intensive human subject and IRB requirements | 4 |
| Dedicated Minerva office staff in DoD | 4 |
| Standardization of reporting requirements across program officers | 1 |
| More demographically diverse awardees | 1 |
| SKP | 22 |
N=76
Q15. How could DoD cultivate greater interest among young scholars in working with DoD on unclassified social science research relevant to national security?
[OPEN-ENDED RESPONSE]
| Coded Open-Ended Responses, Detailed Coding Scheme | |
|---|---|
| Other | 21 |
| General outreach | 18 |
| Already cultivates enough interest | 14 |
| Category specifically for junior scholars | 13 |
| Visibility at conferences | 13 |
| Increase funding for junior scholars | 9 |
| General increase of funding | 9 |
| Award more smaller grants | 5 |
| Advertise outputs/products | 5 |
| Establish grants that pair senior and junior PI’s | 5 |
| Help with translating research into policy | 4 |
| Additional funding for things like websites | 3 |
| SKP | 20 |
N=76
| Coded Open-Ended Responses, Summary Coding Scheme | |
|---|---|
| Other | 46 |
| Outreach | 32 |
| Opportunities for junior scholars | 21 |
| SKP | 20 |
N=76
OUTPUTS. Learning about the outputs that resulted from your Minerva funded research is an important aspect of this evaluation. These outputs include peer-reviewed publications, any other publications (e.g., papers, manuscripts, reports, op-ed pieces), and presentations (e.g., conference presentations, briefings, or testimony). Although you may have been asked to provide similar information to DoD, we would like to ask you to assist our evaluation by providing up-to-date information.
To make this as convenient for you as possible, this information can be provided in three different ways. In which way would you like to submit this information?
For discussion of the results, see report text.
Q16. Did you have outputs of the following type that resulted from your Minerva grant(s)?
For discussion of the results, see report text.
If Yes in Q16a
Q17. Please list your peer-reviewed publications that resulted from a Minerva grant.
[OPEN-ENDED RESPONSE]
For discussion of the results, see report text.
If Yes in Q16b
Q18. Please list any other publications (e.g., papers, manuscripts, reports, op-ed pieces) that resulted from a Minerva grant?
[OPEN-ENDED RESPONSE]
For discussion of the results, see report text.
If Yes in Q16c
Q19. Please list any presentations (e.g., conference presentations, briefings, or testimony) that resulted from a Minerva grant.
[OPEN-ENDED RESPONSE]
For discussion of the results, see report text.
UPLOAD. Please highlight peer-reviewed publications, any other publications (e.g., papers, manuscripts, reports, op-ed pieces), and any presentations (e.g., conference presentations, briefings, or testimony) that resulted from your Minerva grant(s) on your CV and upload below.
For discussion of the results, see report text.
This page intentionally left blank.
