SBIR at the Department of Defense (2014)
Chapter: Appendix G: Comparison Group Analysis
Appendix G
Comparison Group Analysis
The 2011 large-scale survey of SBIR recipients at the Department of Defense (DoD) was sent to two distinct populations: all principal investigators who received DoD Phase II SBIR awards between fiscal year (FY) 1999 and FY2008 inclusive; and all principal investigators who received a Phase I SBIR award and whose company did not receive a Phase II award during the same period.
We sought to develop a useful comparison group for the data collected through the 2011 NRC Survey, but encountered substantial difficulties in finding matching firms – similar in their demographics, market orientation, industry sector, age, size etc.—that have not received SBIR funding.
We then sought to develop a comparison group from among Phase I awardees that had not received a Phase II award from the three surveyed agencies (DoD, the National Science Foundation, and NASA) during the award period covered by the survey (1999-2008). After considerable review, we concluded that the Phase I-only group was also not appropriate for use as a statistical comparison group.
In the interests of providing researchers with a full view of the data collected, this appendix includes tables showing both the Phase I and Phase II survey responses for questions where both groups were surveyed. Some questions that addressed only Phase II respondents are excluded from this Appendix. These tables can be found in the body of the report, especially in Chapter 6.
TABLE APP G-1 Sales (percent of respondents)
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| No sales to date, no sales expected | 44.6 | 28.1 |
| No sales to date, but sales expected | 27.7 | 26.4 |
| Any sales to date | 27.7 | 45.5 |
| 100.0 | 100.0 | |
| N= | 390 | 765 |
NOTE: Data collected 2011. Data covers awards 1998-2007 inclusive.
SOURCE: 2011 NRC Survey, Question 35.
TABLE APP G-2 Distribution of Total Sales, by Range and Phase
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Under $100,000 | 34.4 | 23.4 |
| $100,000-$499,999 | 38.7 | 33.2 |
| $500,000-$999,999 | 12.9 | 13.2 |
| $1,000,000 or more | 14.0 | 30.2 |
| 100.0 | 100.0 | |
| N= | 93 | 325 |
SOURCE: 2011 NRC Survey, Question 36, B1.
TABLE APP G-3 Markets for DoD SBIR Products and Services (percentage of total sales)
| Phase I Only (Percent) | Phase II (Percent) | |
| DoD | 33.7 | 37.3 |
| Domestic private sector | 31.9 | 21.4 |
| Primes for DoD | 15.5 | 21.6 |
| Export markets | 5.2 | 4.5 |
| Other federal agencies | 3.5 | 4.1 |
| NASA | 1.6 | 2.4 |
| State or local governments | 0.5 | 1.6 |
| Prime contractor for NASA | 0.6 | 1.3 |
| Agency that awarded the Phase II (if not NASA or DoD) | 0.0 | 0.9 |
| Other (specify) | 7.5 | 5.0 |
| 100.0 | 100.0 | |
| N= | 108 | 348 |
NOTE: For this question, each respondent reported a percentage distribution. Values above were calculated by deriving the mean value for all the responses received for each category. N=firms reporting sales >$0.
SOURCE: 2011 NRC Survey, Question 37.
TABLE APP G-4 Employment at Time of Award
| Number of Employees | DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) |
| Under 5 | 28.1 | 19.0 |
| 5 to 9 | 16.9 | 17.6 |
| 10 to 19 | 15.9 | 16.8 |
| 20 to 49 | 16.1 | 23.7 |
| 50 to 99 | 10.7 | 11.4 |
| 100 or more | 12.2 | 11.5 |
| 100.0 | 100.0 | |
| Mean | 40 | 41 |
| Median | 12 | 17 |
| BASE: TOTAL RESPONDENTS ANSWERING | 378 | 727 |
SOURCE: 2011 NRC Survey, Question 18A.
TABLE APP G-5 Employment at Time of Reporting
| Number of Employees | DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) |
| Under 5 | 23.1 | 16.0 |
| 5 to 9 | 16.5 | 14.5 |
| 10 to 19 | 13.6 | 15.2 |
| 20 to 49 | 22.0 | 25.1 |
| 50 to 99 | 10.5 | 13.4 |
| 100 or more | 14.2 | 15.8 |
| Mean | 63 | 66 |
| Median | 15 | 24 |
| N= | 366 | 739 |
SOURCE: 2011 NRC Survey, Question 18B.
TABLE APP G-6 Additional Investment after SBIR Award
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Yes | 48.5 | 61.0 |
| No | 51.5 | 39.0 |
| 100.0 | 100.0 | |
| N= | 390 | 765 |
SOURCE: 2011 NRC Survey, Question 33.
TABLE APP G-7 Additional Investments by Non-SBIR Federal Sources by Phase and Amount
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Under $100,000 | 19.5 | 17.8 |
| $100,000-$499,999 | 31.2 | 31.2 |
| $500,000-$999,999 | 27.3 | 15.0 |
| $1,000,000 or more | 22.1 | 36.0 |
| 100.0 | 100.0 | |
| N= | 77 | 253 |
NOTE: N=Those reporting additional funds >$0.
SOURCE: 2011 NRC Survey, Question 34.1.
TABLE APP G-8 Sources of Additional Funding
| Phase I Only (Percent) | Phase II (Percent) | |
| Own company | 49.7 | 54.6 |
| Federal non-SBIR funding | 40.7 | 54.2 |
| Other companies | 11.6 | 16.3 |
| Personal funds | 15.9 | 10.7 |
| State/local government | 6.9 | 6.4 |
| Private equity/angels | 5.3 | 3.6 |
| Venture capital | 4.2 | 2.8 |
| Foreign private | 2.1 | 3.0 |
| Universities/colleges | 2.6 | 2.4 |
| N= | 189 | 467 |
NOTE: Responses do not sum to 100 percent because respondents could select more than one response.
SOURCE: 2011 NRC Survey, Question 34.
TABLE APP G-9 Company-Level Changes as a Result of the SBIR Funding
| Phase I Only (Percent) | Phase II (Percent) | |
| Established one or more spin-off companies | 11.5 | 16.1 |
| Been acquired by/merged with another company | 6.5 | 3.5 |
| Made an IPO | 1.3 | 1.4 |
| Planning to make an IPO in 2011-2012 | 0.5 | 0.9 |
| None of the above | 81.3 | 80.1 |
| N (unique companies) = | 250 | 386 |
| N (unique respondents) = | 359 | 659 |
NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2011 NRC Survey, Question 10.
TABLE APP G-10 Market-Oriented Activities—Finalized Agreements with U.S. Companies and Investors
| Phase I Only (Percent) | Phase II (Percent) | |
| R&D agreement(s) | 58.5 | 49.2 |
| Customer alliance(s) | 25.6 | 27.0 |
| Licensing agreement(s) | 22.0 | 24.2 |
| Manufacturing agreement(s) | 18.3 | 18.9 |
| Marketing/distribution agreement(s) | 13.4 | 16.8 |
| Joint venture agreement | 7.3 | 7.4 |
| Sale of technology rights | 6.1 | 6.6 |
| Sale of company | 4.9 | 2.9 |
| Company merger | 3.7 | 3.3 |
| Partial sale of company | 1.2 | 2.9 |
| Other | 8.5 | 4.9 |
| N= | 82 | 244 |
NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2011 NRC Survey, Question 38.1.
TABLE APP G-11 Participation in Commercialization Training, by Phase
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Yes | 21.4 | 30.2 |
| No | 78.6 | 69.8 |
| 100.0 | 100.0 | |
| N= | 388 | 761 |
SOURCE: 2011 NRC Survey, Question 17.
TABLE APP G-12 Full-Time Marketing Staff (one or more)
| Phase I Only (Percent) | Phase II (Percent) | |
| Yes | 35.4 | 39.0 |
| No | 64.6 | 61.0 |
| Total | 100.0 | 100.0 |
| N (unique respondents) = | 361 | 670 |
SOURCE: 2011 NRC Survey, Question 12.
TABLE APP G-13 Number of Patents Related to All Company SBIR Awards
| Phase I Only (Percent) | Phase II (Percent) | |
| 0 | 47.3 | 42.0 |
| 1 or 2 | 21.4 | 24.7 |
| 3 or 4 | 12.4 | 12.1 |
| 5 to 9 | 12.1 | 11.3 |
| 10 or more | 6.9 | 10.0 |
| Total | 100.0 | 100.0 |
| At least 1 | 52.7 | 58.0 |
| N (unique companies) = | 245 | 374 |
| N (unique respondents) = | 348 | 643 |
NOTE: N=348 statistical tests were run on responses weighted by company.
SOURCE: 2011 NRC Survey, Question 11.
TABLE APP G-14 Number of Patents Related to Surveyed Project
| Phase I Only (Percent) | Phase II (Percent) | |
| 0 | 72.9 | 68.2 |
| 1 | 17.1 | 17.3 |
| 2 | 4.3 | 9.2 |
| 3 | 3.3 | 2.6 |
| More than 3 | 2.4 | 2.6 |
| Total | 100.0 | 100.0 |
| At least 1 | 27.1 | 31.8 |
| N= | 210 | 422 |
SOURCE: 2011 NRC Survey, Question 39.1.2.
TABLE APP G-15 Number of Scientific Publications Related to the Surveyed Project
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| 0 | 48.6 | 25.6 |
| 1 | 21.4 | 18.0 |
| 2 | 14.5 | 17.8 |
| 3 | 6.4 | 12.7 |
| More than 3 | 9.1 | 25.8 |
| Total | 100.0 | 100.0 |
| At least 1 | 51.4 | 74.4 |
| N= | 220 | 488 |
SOURCE: 2011 NRC Survey, Question 39.4.2.
TABLE APP G-16 Links to Universities
| Phase I Only (Percent) | Phase II (Percent) | |
| Faculty member(s) or adjunct faculty member(s) worked on this project in a role other than PI | 16.0 | 18.8 |
| A university or college was a subcontractor on this project | 13.4 | 19.7 |
| Graduate students worked on this project | 10.2 | 17.7 |
| The technology for this project was originally developed at a university or college by one of the participants in this project | 8.9 | 7.7 |
| The PI for this project was an adjunct faculty member | 5.2 | 3.3 |
| The technology for this project was licensed from a university or college | 4.2 | 3.1 |
| The PI for this project was a faculty member | 3.4 | 1.7 |
| Any of the above | 30.4 | 34.8 |
| N= | 381 | 750 |
SOURCE: 2011 NRC Survey, Question 59.
TABLE APP G-17 Long-Term Impacts of SBIR Funding on Recipient Companies
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Had a transformative effect | 20.2 | 20.2 |
| Had a substantial positive long-term effect | 44.6 | 56.8 |
| Had a small positive effect | 27.2 | 17.2 |
| Had no long-term effect | 7.0 | 3.8 |
| Had a negative long-term effect | 1.1 | 1.9 |
| Total | 100.0 | 100.0 |
| N (unique companies) = | 263 | 416 |
| N (unique respondents) = | 387 | 762 |
SOURCE: 2011 NRC Survey, Question 45.
TABLE APP G-18 Number of Founders
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| 1 | 37.6 | 41.8 |
| 2 | 33.2 | 30.8 |
| 3 | 14.5 | 16.1 |
| 4 | 7.6 | 7.7 |
| 5 or more | 7.2 | 3.5 |
| 100.0 | 100.0 | |
| N (unique companies) = | 244 | 382 |
| N (unique respondents) = | 350 | 660 |
NOTE: The NRC survey received responses from multiple respondents per company. For company-level questions, these responses were weighted equally, so that, taken collectively, the responses from each company had equal impact on statistical analysis. This approach was used for tables based on Questions 3-12 inclusive in the survey questionnaire (see Appendix D).
SOURCE: 2011 NRC Survey, Question 4A.
TABLE APP G-19 Number of Previous Companies Started by Founders
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| 0 | 56.4 | 62.0 |
| 1 | 21.1 | 17.8 |
| 2 | 11.1 | 12.9 |
| 3 | 6.9 | 4.2 |
| 4 | 1.4 | 1.3 |
| 5 or more | 3.0 | 1.8 |
| 100.0 percent | 100.0 percent | |
| N (unique companies) = | 244 | 382 |
| N (unique respondents) = | 350 | 660 |
SOURCE: 2011 NRC Survey, Question 4B.
TABLE APP G-20 Number of Founders with Business Backgrounds
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| None | 48.6 | 51.7 |
| 1 | 37.0 | 32.2 |
| 2 | 9.2 | 11.0 |
| 3 | 2.3 | 2.9 |
| 4 | 2.5 | 1.4 |
| 5 or more | 0.4 | 0.7 |
| Total | 100.0 | 100.0 |
| At least 1 | 51.4 | 48.2 |
| N (unique companies) = | 246 | 382 |
| N (unique respondents) = | 352 | 660 |
NOTE: Statistical tests were run on responses weighted by company.
SOURCE: 2011 NRC Survey, Question 4C.
TABLE APP G-21 Number of Founders with Academic Backgrounds
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| None | 31.4 | 38.8 |
| 1 | 42.5 | 39.9 |
| 2 | 14.9 | 12.2 |
| 3 | 6.5 | 4.2 |
| 4 | 3.9 | 3.6 |
| 5 or more | 0.9 | 1.4 |
| Total | 100.0 | 100.0 |
| At least 1 | 67.8 | 61.3 |
| N (unique companies) = | 246 | 382 |
| N (unique respondents) = | 351 | 660 |
SOURCE: 2011 NRC Survey, Question 4D.
TABLE APP G-22 Prior Employment of Founders
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Other private company | 67.8 | 73.1 |
| College or university | 29.8 | 27.6 |
| Government | 12.2 | 9.2 |
| Other | 7.2 | 4.8 |
| N (unique companies) = | 250 | 391 |
| N (unique respondents) = | 358 | 671 |
SOURCE: 2011 NRC Survey, Question 5.
TABLE APP G-23 Company Founded Because of SBIR Program
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Yes | 15.5 | 17.9 |
| In part | 16.5 | 17.5 |
| No | 68.1 | 64.7 |
| Total | 100.0 | 100.0 |
SOURCE: 2011 NRC Survey, Question 6.
TABLE APP G-24 Distribution of Responses by Sector and Phase
| Technology sector | DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) |
| Aerospace | 31.6 | 46.3 |
| Defense-specific products and services | 68.1 | 71.6 |
| Energy and the environment | 11.8 | 10.7 |
- Sustainable energy production (solar, wind, geothermal, bio-energy, wave) | 2.8 | 2.2 |
- Energy storage and distribution | 2.8 | 2.0 |
- Energy saving | 2.1 | 2.5 |
- Other energy or environmental products and services | 2.3 | 3.0 |
| Engineering | 39.1 | 51.9 |
- Engineering services | 10.3 | 14.3 |
- Scientific instruments and measuring equipment | 7.5 | 12.6 |
- Robotics | 4.1 | 5.5 |
- Sensors | 15.7 | 24.0 |
- Other engineering | 7.5 | 10.6 |
| Information technology (IT) | 19.3 | 18.2 |
- Computers and peripheral equipment | 3.6 | 4.2 |
- Telecommunications equipment and services | 4.1 | 3.8 |
- Business and productivity software | 4.1 | 3.3 |
- Data processing and database software and services | 4.6 | 5.9 |
- Media products (including web-, print- and wireless-delivered content) | 1.5 | 1.3 |
- Other IT | 2.8 | 5.4 |
| Materials (including nanotechnology for materials) | 26.2 | 23.1 |
- Medical technologies | 2.6 | 2.1 |
- Pharmaceuticals | - | 0.7 |
- Medical devices | 5.4 | 5.9 |
- Other biotechnology products | 2.6 | 2.2 |
- Other medical products and services | 2.8 | 1.6 |
| Other (please describe) | 7.5 | 12.6 |
NOTE: Answers do not sum to 100 percent because respondents could select more than one response.
SOURCE: 2011 NRC Survey, Question 20.
TABLE APP G-25 Current Status of Surveyed Projects
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Project has not yet completed [SBIR] funded research | 8.5 | 2.4 |
| Efforts at this company have been discontinued | 43.6 | 30.7 |
Discontinued because no sales or additional funding resulted from this project | 36.4 | 20.8 |
Discontinued—The project did result in sales, licensing of technology, or additional funding | 7.2 | 9.9 |
| Project is continuing post-award technology development | 22.1 | 26.9 |
| Commercialization is under way | 12.1 | 12.4 |
| Products/Processes/Services are in use | 13.8 | 27.6 |
In use by target customers | 9.0 percent | 21.7 percent |
In use by customers not anticipated at the time of the award | 4.9 percent | 5.9 percent |
| Total (primary categories) | 100.0 | 100.0 |
N= | 390 | 765 |
SOURCE: 2011 NRC Survey, Question 30.
TABLE APP G-26 Primary Reason for Project Discontinuation
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Not enough funding | 61.2 | 24.8 |
| Project goal was achieved (e.g. prototype delivered for federal agency use) | 8.8 | 16.5 |
| Market demand too small | 4.7 | 13.0 |
| Company shifted priorities | 5.9 | 2.6 |
| Level of technical risk too high | 3.5 | 2.6 |
| Other reason mentioned: | 15.9 | 40.4 |
| 100.0 | 100.0 | |
| N= | 170 | 230 |
NOTE: N = Respondents with awards no longer active.
SOURCE: 2011 NRC Survey, Question 32.
TABLE APP G-27 Company Annual Revenues by Phase (most recent fiscal year)
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Less than $100,000 | 12.1 | 10.0 |
| $100,000-$499,999 | 19.1 | 13.9 |
| $500,000-$999,999 | 13.8 | 12.2 |
| $1,000,000-$4,999,999 | 30.6 | 37.6 |
| $5,000,000-$19,999,999 | 15.0 | 18.3 |
| $20,000,000-$99,999,999 | 7.1 | 6.7 |
| $100,000,000 or more | 2.3 | 1.4 |
| 100.0 | 100.0 | |
| N (unique companies) = | 243 | 385 |
| N (unique respondents) = | 350 | 659 |
SOURCE: 2011 NRC Survey, Question 8.
TABLE APP G-28 Percentage of R&D Effort expended on SBIR projects (most recent fiscal year)
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| 0% | 26.5 | 17.3 |
| 1%-10% | 10.8 | 14.5 |
| 11%-25% | 14.6 | 15.9 |
| 26%-50% | 13.5 | 19.5 |
| 51%-75% | 19.2 | 17.6 |
| 76%-100% | 15.4 | 15.2 |
| 100.0 | 100.0 | |
| N (unique companies) = | 249 | 387 |
| N (unique respondents) = | 357 | 660 |
SOURCE: 2011 NRC Survey, Question 7.
TABLE APP G-29 Percentage of Company Revenues Related to SBIR Project (most recent fiscal year)
| Phase I Only (Percent) | Phase II (Percent) | |
| 0% | 30.8 | 21.3 |
| 1%-10% | 10.8 | 16.3 |
| 11%-25% | 16.5 | 16.9 |
| 26%-50% | 14.3 | 16.8 |
| 51%-75% | 12.2 | 15.3 |
| 76%-99% | 9.6 | 10.3 |
| 100% | 5.6 | 3.1 |
| Total | 100.0 | 100.0 |
| N (unique companies) = | 247 | 385 |
| N (unique respondents) = | 355 | 654 |
NOTE: Companies reporting $0 have completed their previous awards and do not have current awards.
SOURCE: 2011 NRC Survey, Question 9.
TABLE APP G-30 Number of Prior SBIR or STTR Phase I Awards Related to the Surveyed Project
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| 0 | 26.1 | 20.0 |
| 1 | 35.6 | 34.4 |
| 2 | 18.7 | 17.3 |
| 3 | 8.3 | 11.5 |
| 4 | 5.2 | 6.5 |
| 5 or more | 6.0 percent | 10.2 percent |
| Total | 100.0 | 100.0 |
| 1 or more | 73.9 | 80.0 |
| N= | 348 | 704 |
SOURCE: 2011 NRC Survey, Question 40.1.1.
TABLE APP G-31 Sources of Funding, Prior to SBIR Award, for Related Technology
| DoD SBIR Phase I Only (Percent) | DoD SBIR Phase II (Percent) | |
| Internal company investment (including borrowed money) | 45.4 | 52.7 |
| Prior [SBIR] (excluding the Phase I that preceded this Phase II) (a) | NA | 53.3 |
| Prior non-[SBIR] federal R&D | 27.3 | 30.0 |
| Prior SBIR Phase I funding (b) | 45.8 | NA |
| Other private company | 8.8 | 14.0 |
| Private investor (including angel funding) | 15.4 | 9.5 |
| State or local government | 8.5 | 7.7 |
| Venture capital | 3.5 | 5.7 |
| College or university | 3.8 | 3.7 |
| Other (please specify) | 8.5 percent | 6.1 |
| N= | 260 | 493 |
NOTES: (a) Only asked for Phase II awards. (b) Only asked for SBIR Phase 1 awards. Totals do not sum to 100 percent because respondents could select more than one funding source.
SOURCE: 2011 NRC Survey, Question 21.
