Assessment of NIH Minority Research and Training Programs: Phase 3 (2005)
Chapter: Appendix D Interview Instruments for Training Program Administrators
Appendix D
Interview Instruments for Training Program Administrators
Program Administrator at Recipient Institutions (PARI)
Introduction
Thank you for agreeing to participate in the Assessment for NIH Minority Research/Training Programs project. The goal of this study is to assess and analyze NIH minority trainee educational and career outcomes, and to recommend improvements to the NIH coordinated tracking/information system of minority research/training programs and their participants.
In this interview, we are going to discuss various aspects of the training program. The results will help us determine:
-
The ways and extent to which NIH minority research/training programs work
-
Which features of minority programs have been the most successful in helping individual students and faculty members move a step forward toward productive careers as research scientists
-
What programmatic, environmental, or other factors increase the likelihood of minority training programs and their participating trainees achieving success
-
How to assess better NIH minority training programs
In this interview, we would like to discuss the ______________ program, which is funded though the __________ mechanism(s).
Do you know if this is a targeted or non-targeted program? Just to confirm, by targeted NIH is referring to mechanisms and programs designed for underrepresented minority trainees only. By non-targeted, NIH is referring to mechanisms and programs not specifically designed for underrepresented minority trainees.
A. Introductory Questions
A1: Briefly describe the history and evolution of the program at your institution or university.
A2: How does this program fit within the department at your institution or university?
A3: What are the stated program goals?
PROBE: How have the program goals changed over time?
A4: What is your role in the program?
PROBE: What percentage of your time at the institution, department, or university relates to the program?
PROBE: How long have you been in your current position?
PROBE: How long have you been administering this mechanism?
B. Trainee Recruitment
B1: How do you attract or recruit individuals to the training program?
PROBE: How does your department communicate the availability of the program to individuals?
PROBE: What is your role in the communication process?
PROBE: What is your role in announcing the availability of this research/training grant?
B2: What kinds of candidates do you wish to attract with this program?
PROBE: Describe the characteristics of the typical candidate
PROBE: Describe the characteristics of the ideal candidate.
B3. Do you take any special steps to recruit different kinds of trainees?
PROBE: From different gender groups?
PROBE: From different ethnic groups?
PROBE: Describe what steps you take.
B5: Do you think that any potential minority trainees are missed by the current recruitment strategy?
PROBE: What have you seen that helps you form this opinion?
B6: In your opinion, how could the dissemination of information about minority-targeted training programs be improved?
PROBE: Have there been changes in the way the program has been announced in the past?
PROBE: What would need to happen to implement the changes you suggest?
B7: What are you looking for when you select candidates?
PROBE: What criteria are used most often?
PROBE: What are the cues you see in candidates that lead you to recommend they be selected?
B8: How has your recruitment or selection approach changed since the program was started at your institution, university, or department?
C. Trainee Experience within the Program
C1: What challenges and issues confront students in the program?
C2: Describe a specific example of a student who was successful in the program.
PROBE: What factors about different trainees distinguish successful experiences from less successful experiences?
PROBE: Where would this student be, in terms of his/her career, if he/she did not participate in the program?
C3: Describe a specific example of a student who had difficulty in the program.
PROBE: What about the student, the program, and the surrounding context made it difficult for him/her?
PROBE: How were the issues resolved?
C4: How do you know when a student is having trouble in the program?
PROBE: What measures are in place to help students who begin to have trouble in the program?
D. Program Administration
D1: Describe how you interact with the trainees’ sponsors or mentors at recipient institutions.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions/qualities?
D2: Describe how you interact with the granting agency—NIH.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions/qualities?
D3: From your experience, what can you tell me about the difference in contact between you as the Program Administrator and trainees with individual versus institutional awards?
PROBE: In what way is the nature of the contact different?
D4: From your experience, what can you tell me about the difference in contact between you as the Program Administrator and trainees in targeted programs versus nontargeted programs?
PROBE: In what way is the nature of the contact different?
D5: What would make the administration of the program more efficient or more effective?
PROBE: From an administrative perspective, what NIH policies if any, hinder the administration of this program?
PROBE: Are there any NIH policy changes that you would recommend?
PROBE: How would these policy changes make administration of the program more efficient or more effective?
D6: Has your institution, department, or university’s focus on minority training changed over time?
PROBE: How has this focus changed?
PROBE: What has been the cause of these changes?
PROBE: How do these changes further or hinder NIH’s goals related to minority research/training grant mechanisms?
PROBE: In your opinion, what is your institution, department, or university’s level of commitment to minority research training programs?
E. Program Evaluation
E1: Tell me about the program and trainee outcomes.
PROBE: What percentage of potential applicants actually apply to the program?
PROBE: Of all applicants, what percentage is accepted?
PROBE: Of accepted applicants, what percentage actually enters the program?
PROBE: Of those who enter the program, what percentage completes the program?
E2: Is your program evaluated?
PROBE: Would you say this was a formal or informal evaluation?
PROBE: When was this?
PROBE: What were the findings?
PROBE: How else is the program evaluated?
PROBE: How did the findings influence the administration of the program?
PROBE: What other changes were made as a result of the evaluation?
E3: Describe any other major changes that have been implemented in this mechanism over time.
PROBE: What were the instigating reasons for these changes?
PROBE: What were the outcomes of having made these changes?
E4: Describe how you measure the attainment of the program’s goals?
PROBE: What are the specific performance measures used?
F. Final Words
F1: Are there any other aspects of your training program that you think we should know about?
F2: Are there any issues that we haven’t covered that should be discussed as part of our evaluation?
Conclusion
Thank you for participating.
Program Administrators at NIH Institutes and Centers (PAIC)
Individual Awards
Introduction
Thank you for agreeing to participate in the Assessment for NIH Minority Research/Training Programs project. The goal of this study is to assess and analyze NIH minority trainee educational and career outcomes, and to recommend improvements to the NIH coordinated tracking/information system of minority research/training programs and their participants.
In this interview, we are going to discuss various aspects of the training program. The results will help us determine:
-
The ways and extent to which NIH minority research/training programs work
-
Which features of minority programs have been the most successful in helping individual students and faculty members move a step forward toward productive careers as research scientists
-
What programmatic, environmental, or other factors increase the likelihood of minority training programs and their participating trainees achieving success
-
How to assess better NIH minority training programs
Just to confirm, by targeted NIH is referring to mechanisms and programs designed for underrepresented minority trainees, only. By non-targeted, NIH is referring to mechanisms and programs not specifically designed for underrepresented minority trainees.
In this interview, we would like to discuss the ______________ mechanism(s).
A. Introductory Questions
A1: Briefly describe the purpose of the mechanism, its history, and how it has changed over time.
PROBE: In your opinion what are the goals of the mechanism?
A2: Briefly describe your current role and how this mechanism is a part of that role.
PROBE: How long have you been in your current position?
PROBE: How long have you been administering this mechanism?
B. Trainee Recruitment
B1: How does your Institute or Center communicate the availability of training mechanisms to individuals?
B2: How do you communicate the availability of training mechanisms to institutions?
PROBE: What is your role in the communication process?
PROBE: What is your role in announcing the availability of this research / training grant?
B3. What kinds of candidates do you wish to attract with this mechanism?
PROBE: Describe the characterizes of the typical candidate
PROBE: Describe the characteristics of the ideal candidate.
B4: Do you take any special steps to recruit different kinds of trainees?
PROBE: From different gender groups?
PROBE: From different ethnic groups?
PROBE: Describe what steps you take.
B5: Do you think that any potential minority trainees are missed by the current recruitment strategy?
PROBE: If YES, Why?
B6: In your opinion, how could the dissemination of information about minority-targeted training mechanisms could be improved?
PROBE: Have there been changes in the way the mechanism has been announced in the past?
PROBE: What would need to happen to implement the changes you suggest?
C. Selection Criteria
C1: What are you looking for when you select candidates?
PROBE: What criteria are used most often?
PROBE: What are the cues you see in candidates that leads you to recommend they be selected?
D. Retention
D1: Describe how you interact with the trainees’ sponsors or mentors at recipient institutions.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions/qualities?
D2a—INDIVIDUAL AWARDS ONLY: Describe how you interact with individual trainees.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions/qualities?
D2b—INSTITUTIONAL AWARDS ONLY: Describe how you interact with recipient institutions.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions / qualities?
D3: From your experience, what can you tell be about the difference in contact between you as the Program Administrator and trainees with individual versus institutional awards?
PROBE: In what way is the nature of the contact different?
D4: From your experience, what can you tell be about the difference in contact between you as the Program Administrator and trainees in targeted programs versus nontargeted programs?
PROBE: In what way is the nature of the contact different?
D5: How does your Institute address trainee retention?
PROBE: What about trainees who do not complete their grant?
PROBE: What about trainees who leave the field or do not get a second grant?
D6: How do you learn about trainee attrition?
PROBE: Are you aware when a trainee leaves a training program without graduating, or quits the lab in which they were working?
PROBE: Is there any formal notification, such as a written or electronic form?
E. Program Evaluation
E1: Is your program evaluated?
PROBE: Would you say this was a formal or informal evaluation?
PROBE: When was this?
PROBE: What were the findings?
PROBE: How else is the program evaluated?
PROBE: How did the findings influence the administration of the program?
PROBE: What other changes were made as a result of the evaluation?
E2: Describe how you measure the attainment of the mechanism’s goals?
PROBE: What are the specific performance measures used?
E2a—INSTITUTIONAL AWARD ONLY: Tell me about a recipient institution that in your opinion has been very successful in achieving the goals of this mechanism.
PROBE: What factors distinguish this successful case from other, less successful efforts at other locations?
E2b—INSTITUTIONAL AWARD ONLY: Tell me about a recipient institution that was less effective at achieving the goals of the mechanism.
PROBE: What strategies, if any, do you use to raise the performance of what might be considered less successful programs?
E3: Describe any other major changes that have been implemented in this mechanism over time.
PROBE: What were the instigating reasons for these changes?
PROBE: What were the outcomes of having made these changes?
F. Program Administration
F1: What would make the administration of the program more efficient or more effective?
PROBE: From an administrative perspective, what NIH policies if any, hinder the administration of this program?
PROBE: Are there any NIH policy changes that you would recommend?
PROBE: How would these policy changes make administration of the program more efficient or more effective?
F2: At your Institute or Center, who decides which minority-targeted mechanisms the Institute or Center will sponsor?
PROBE: To what extent are you Program Administrators included in the decision making process?
PROBE: To what extent are Program Administrators in general included in the decision making process?
F3: Has your Institute or Center’s focus on minority training changed over time?
PROBE: How has this focus changed?
PROBE: What has been the cause of these changes?
PROBE: How do these changes as further or hinder NIH’s goals related to minority research / training grant mechanisms?
F4: In your opinion, what is your Institute’s level of commitment to minority research training programs?
G. Final Words
G1: Are there any issues that you would like to see this evaluation study address?
G2: Are there any other questions about this program that you think that we should be asking?
Conclusion
Thank you for participating.
Program Administrators at NIH Institutes and Centers (PAIC)
Institutional Awards
Introduction
Thank you for agreeing to participate in the Assessment for NIH Minority Research / Training Programs project. The goal of this study is to assess and analyze NIH minority trainee educational and career outcomes, and to recommend improvements to the NIH coordinated tracking/information system of minority research/training programs and their participants.
In this interview, we are going to discuss various aspects of the training program. The results will help us determine:
-
The ways and extent to which NIH minority research/training programs work
-
Which features of minority programs have been the most successful in helping individual students and faculty members move a step forward toward productive careers as research scientists
-
What programmatic, environmental, or other factors increase the likelihood of minority training programs and their participating trainees achieving success
-
How to assess better NIH minority training programs
Just to confirm, by targeted NIH is referring to mechanisms and programs designed for underrepresented minority trainees, only. By non-targeted, NIH is referring to mechanisms and programs not specifically designed for underrepresented minority trainees.
In this interview, we would like to discuss the ______________ mechanism(s).
A. Introductory Questions
A1: Briefly describe the purpose of the MECHANISM, its history, and how it has changed over time.
PROBE: In your opinion what are the goals of the MECHANISM?
A2: Briefly describe your current role and how this MECHANISM is a part of that role.
PROBE: How long have you been in your current position?
PROBE: How long have you been administering this MECHANISM?
B. Trainee Recruitment
B1: How does your Institute or Center communicate the availability of training mechanisms to individuals?
B2: How do you communicate the availability of training mechanisms to institutions?
PROBE: What is your role in the communication process?
PROBE: What is your role in announcing the availability of this research / training grant?
B2. What kinds of candidates do you wish to attract with this MECHANISM?
PROBE: Describe the characterizes of the typical candidate
PROBE: Describe the characteristics of the ideal candidate.
B3: Do you take any special steps to recruit different kinds of trainees?
PROBE: From different gender groups?
PROBE: From different ethnic groups?
PROBE: Describe what steps you take.
B4: Do you think that any potential minority trainees are missed by the current recruitment strategy?
PROBE: If YES, Why?
B5: In your opinion, how could the dissemination of information about minority-targeted training mechanisms could be improved?
PROBE: Have there been changes in the way the mechanism has been announced in the past?
PROBE: What would need to happen to implement the changes you suggest?
C. Selection Criteria
C1: What are you looking for when you select candidates?
PROBE: What criteria are used most often?
PROBE: What are the cues you see in candidates that leads you to recommend they be selected?
D. Retention
D1: Describe how you interact with the trainees’ sponsors or mentors at recipient institutions.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions?
D2: Describe how you interact with individual trainees.
PROBE: Frequency?
PROBE: Duration?
PROBE: Depth?
PROBE: Other dimensions?
D3: Describe how you interact with recipient institutions.
PROBE: Frequency? Duration? Depth? Other dimensions?
D3: Is the level of contact with trainees different with awards made to individuals, as opposed to institutions?
IF YES, PROBE: In what way is the level of contact different? (frequency? duration? depth? All of these dimensions?)
D4: Of all the trainees, which do you have more contact with: those in targeted programs or those in non-targeted programs?
D5: How does your Institute address trainee retention?
PROBE: What about trainees who do not complete their grant?
PROBE: What about trainees who leave the field or do not get a second grant?
D6: How do you learn about trainee attrition?
PROBE: Are you aware when a trainee leaves a training program without graduating, or quits the lab in which they were working?
IF YES, PROBE: Is there any formal notification, such as a written or electronic form?
E. Program Evaluation
E1: Is your program evaluated?
PROBE: Formally?
PROBE: When was this?
PROBE: What were the findings?
PROBE: How did the findings influence the administration of the program?
PROBE: What other changes were made as a result of the evaluation?
PROBE: Informally?
PROBE: When was this?
PROBE: What were the findings?
PROBE: How did the findings influence the administration of the program?
PROBE: What other changes were made as a result of the evaluation?
E2: Describe how you measure the attainment of the MECHANISM’S goals? PROBE: What are the specific performance measures used?
E4: Tell me about a recipient institution that in your opinion has been very successful in achieving the goals of the {MECHANISM}.
PROBE: What factors distinguish this successful case from other, less successful efforts at other locations?
E5: Tell me about a recipient institution that was less effective at achieving the goals of the {MECHANISM}.
PROBE: What strategies, if any, do you use to raise the performance of ‘less successful’ programs?
E6: Describe any other major changes that have been implemented in this {MECHANISM} over time.
PROBE: What were the instigating reasons for these changes?
PROBE: What were the outcomes of having made these changes?
F. Program Administration
F1: What would make the administration of the program more efficient or more effective?
PROBE: Are there any NIH policy changes that you would recommend?
IF YES, PROBE: How would these policy changes make administration of the program more efficient or more effective?
F2: From an administrative perspective, what NIH policies if any, hinder the administration of this program?
F3: At your Institute, who decides which minority-targeted mechanisms the Institute will sponsor?
F4: To what extent are Program Administrators included in the decision making process?
F5: Has your Institute’s focus on minority training changed over time?
IF YES, PROBE: How has this focus changed?
F6: In your opinion, what is your Institute’s level of commitment to minority research training programs?
G. Final Words
G1: Are there any issues that you would like to see this evaluation study address?
G2: Are there any other questions about this program that you think that we should be asking?
G3: How difficult was it to compile the documents we asked for?
Conclusion
Thank you for participating.
