An Assessment of the Small Business Innovation Research Program: Project Methodology (2004)
Chapter: Annex F Research Matrix
ANNEX F:
Research Matrix
|
SBIR Review Objectives |
||||||||
|
|
Assessment of the program |
Consideration of operational improvements to the program |
||||||
|
|
quality of research |
commercialization of SBIR funded research/economic and non-economic benefits |
small business innovation/growth (incl.minority owned) |
use of small business research to advance agency missions |
program design |
service to participants |
efficiency of SBIR program administration |
program evaluation |
|
Questions |
* How does the qualityof SBIR funded research compare that of other government funded R&D? |
* What is the over all economic impact of SBIR funded research? * What fraction of that impact is attributable to SBIR funding? |
* How to broaden participation and replenishing contractors? * Possibility of link SBIR with state/regional programs? |
* How to increase agency up take while continuing to support high-risk research? |
* Time and award limits? * Review process and criteria? * Intra- and inter-agency interface? * New funding model to build collaborative networks? |
* How can SBIR better serve technology entrepreneurs? * Lessons from angels? Corporate R&D? |
* How to leverage economies of scale in program administration while allowing each agency full management autonomy? |
* How to evaluate SBIR success: Appropriate measures? Impact of skew in returns? |
|
Measures |
peer review scores, publication counts, citation analyses |
sales; follow-up funding (e.g. angel, VC, corporate); drug development progress; IPOs |
patent counts and other IP/employment or job growth, number of new technology firms |
agency procurement of products resulting from SBIR work |
? |
? |
speed of response to applicants; administrative cost per dollar awarded |
[all] |
|
Key researchissues /challenges |
* Difficulty both of measuring quality and of identifying proper reference group |
* Skew of returns: estimates of returns invariably dominated by a few big "winners"` * Significant inter-agency and inter-industry differences |
* Measures of actual success failure (both project and firm level) * Relationship of Federal and state programs in this context |
Major inter-agencies differences in use of SBIR to meet agency missions |
Documenting/addressing the problem of "backwater" status of SBIR programs within agencies |
* Response time * Ease of application * Outreach activities |
* Shift from paper to electronic program management * Possibility of using some funds for program administration |
* Appropriate implementation of GPRA in this context? |
|
Desired outcomes |
* SBIR isworking better* Projects are beingtracked* Measures exist of long-term impacts |
|||||||
|
|
Assessment of the program |
Consideration of operational improvements to the program |
||||||
|
|
quality of research |
commercialization of SBIR funded research/economic and non-economic benefits |
small business innovation/growth (incl. minority owned) |
use of small business research to advance agency missions |
program design |
service to participants |
efficiency of SBIR program administration |
program evaluation |
|
Phase I survey |
|
|
X |
|
X |
X |
|
X |
|
Phase II survey |
|
X |
X |
|
X |
X |
|
X |
|
survey of "program managers" |
|
X |
|
X |
X |
|
X |
X |
|
case studies |
X |
X |
X |
X |
|
X |
|
X |
|
agency program studies |
X |
|
|
X |
X |
X |
X |
X |
|
proposed study of repeat winners |
X |
X |
X |
X |
|
X |
X |
X |
|
proposed bibliometric analysis comparing SBIR firms with (similar) non-SBIR firms |
X |
|
X |
|
|
|
|
X |
