Summary

Created in 1982 through the Small Business Innovation Development Act, the Small Business Innovation Research (SBIR) program remains the nation’s largest innovation program for small business. The SBIR program offers competitive awards to support the development and commercialization of innovative technologies by small private-sector businesses. At the same time, the program provides government agencies with technical and scientific solutions that address their different missions.

Seeking to bridge the gap between basic science and commercialization of resulting innovations, the Small Business Technology Transfer (STTR) program, created in 1992 by the Small Business Research and Development Enhancement Act of 1992, seeks to expand joint venture opportunities for small businesses and nonprofit research institutions. Under STTR, a small business receiving an award must collaborate formally with a research institution.

Adopting several recommendations from a 2008 National Research Council (NRC) study of the SBIR Program, Congress reauthorized the SBIR/STTR programs in December 2011 for an additional 6 years. As a part of this reauthorization, Congress called for further studies by the National Academies of Sciences, Engineering and Medicine. In turn, the Department of Energy (DoE) requested the National Academies to provide a subsequent round of analysis, focused on operational questions with a view to identifying further improvements to the program.

FOCUS ON LEGISLATIVE OBJECTIVES

This report assesses the performance of the DoE SBIR and STTR programs against the broad congressional objectives for the programs.

For SBIR, these objectives were reiterated in the 2011 program reauthorization and elaborated in the subsequent policy directive of the Small Business Administration (SBA). Section 1c of the SBA SBIR Directive states program goals as follows:

The parallel language from the SBA’s STTR Policy Directive is as follows:

SCOPE OF THE ASSESSMENT

The SBIR/STTR programs are unique in terms of scale and mission focus. In addition, the evidence suggests that there are no truly comparable programs in the United States, and those in other countries operate in such different ways that their relevance is limited. Further, as in the 2008 NRC study, the objective of this second-round study is “not to consider if SBIR should exist or not.” Rather, this study is charged with “providing assessment-based findings of the benefits and costs of SBIR [and STTR] . . . to improve public understanding of the program, as well as recommendations to improve the program’s effectiveness.”

It is important to note at the outset that this volume—and this study—do not seek to provide a comprehensive review of the value of the SBIR/STTR programs, in particular measured against other possible alternative uses of federal funding. Such a review is beyond the committee’s scope. Rather, the committee’s work is focused on assessing the extent to which the DoE SBIR/STTR programs have met the congressional objectives set for the programs, in particular whether recent initiatives have improved program

outcomes, and to provide recommendations for further improvements to the program.

STUDY METHODOLOGY AND LIMITATIONS

The committee’s findings are based on a complement of quantitative and qualitative tools including a survey, case studies of award recipients, agency data, public workshops, and agency meetings. The methodology is described in Chapter 1 and Appendix A of this report. In reviewing the findings, it is important to note that the National Academies’ 2014 Survey—hereafter referred to as the 2014 Survey—was sent to every principal investigator (PI) who won a Phase II award from DoE, FY 2001-2010 (not the registered company points of contact [POC] for each company.) Each PI was asked to complete a maximum of two questionnaires, which as a result excludes some awards from the survey. The preliminary population was developed by taking the original set of SBIR and STTR Phase II awards made by DoE during the study period and eliminating on a random basis awards in excess of two to PIs who received more than two awards (to limit the burden on respondents). The resulting preliminary population was 1,077 awards. PIs for 583 of these awards were determined to be not contactable at the SBIR/STTR company listed in the DoE awards database. The remaining 494 awards constitute the effective population for this study. From the effective population, we received 269 responses. As a result, the response rate in relation to the preliminary population was 25.0 percent and in relation to the effective population was 54.5 percent.

The committee acknowledges that the study lacks an experimental or quasi-experiment study design that allows a randomly based comparison of the outcomes of companies that applied and did not apply and of those that received SBIR/STTR awards and those that did not—a design that would allow testing of the award’s impact and the effect of gender and ethnicity on applications, awards, and success rates. As is typically the case in studies of competitively based grant programs, study designs that allowed comparisons only of the application or grant effect was impossible because the populations of applications and non-applications and of award recipient and non-recipients differ in many more ways than whether or not they applied and in whether or not they received an award; also, the program has criteria for making awards that are not randomly based.

The committee acknowledges that because information from non-respondent PIs was lacking, and because the agencies also have minimal information about PIs which could be used to track potential non-respondent biases, we can conclude only that the data are likely to be biased. Two potential biases are expected with regard to PIs participating in the survey: A bias toward PIs who are working at companies that are still in business as corporate entities (i.e. have not failed or been acquired), and a bias toward PIs who have received multiple awards because they are in the system multiple times and they may tend to have a greater reliance on the SBIR program, a more favorable view of

it, and a greater willingness to complete the survey; furthermore, they may have greater recall about the program from working with it multiple times. Another potential bias results from the fact that the body of data is skewed, such that companies showing successful commercialization are rarer than companies having less commercial success. A random draw from the database would be less likely to produce a commercial success than not. Box A-1 in Appendix A presents a more complete discussion of the potential sources of biases that can skew the results in different directions.

The committee chose to focus the survey on Phase II awards rather than Phase I awards because Phase II-funded projects are expected to have business plans and to have progress toward commercialization. Thus, it is reasonable to expect a survey based on Phase II to show more evidence of commercial activity than one based on Phase I or a combination of both phases. The focus on Phase II awards reflects the effects of a “weeding out” of projects which were either not pursued by the companies for further SBIR/STTR funding or which were deemed not worthy of additional funding by the SBIR/STTR funding process. The focus on Phase II seems reasonable given the interest in commercialization.

In addition to information from this survey, the committee has drawn on company case studies, discussions with agency staff, and other documentation. In interpreting the findings and recommendations set out below, the reader needs to keep in mind the size of the survey population and response rates, and the overall potential sources of bias.

KEY FINDINGS

The SBIR program at the DoE is having a positive overall impact. It is meeting three of the four legislative objectives of the program with regard to stimulating technological innovation, using small businesses to meet federal research and development (R&D) needs, and increasing private-sector commercialization of innovations derived from federal R&D. However, the committee finds that more needs to be done to “foster and encourage participation by socially and economically disadvantaged small businesses (SDBs), and by woman-owned small businesses (WOSBs), in technological innovation.” The STTR program at DoE is also meeting the program’s statutory objectives, defined above, in that it is encouraging and supporting linkages between small business corporations (SBCs) and research institutions (RIs).

The findings are organized according to the legislative goals for SBIR/STTR plus findings on the management of the program.

Program Management

• DoE has substantially improved its SBIR/STTR programs since 2008 (the publication year of the previous National Academies report on the

• DoE SBIR program). A number of recommendations from the 2008 report have been adopted. (Finding I-A)

• DoE has adopted a number of other initiatives and pilot programs, which collectively have improved the program. (Finding I-A)
• DoE is seeking ways to improve its data collection and tracking. (Finding I-E)

Commercialization

• Nearly half of the respondents to the National Academies’ 2014 Survey reported some sales, and a further 23 percent reported anticipating future sales. Of those respondents reporting some sales, 25 percent had sales less than $100,000. Six percent had sales over $10 million, and an additional 26 percent had sales over $1 million. The large number of companies with small-scale revenues suggests that although many companies reach the market, few can be described as successful in commercial terms. This finding reflects a deeper understanding of the limitations of the available data on successful commercialization. (Finding II-A)
• Subsequent investment in DoE SBIR/STTR projects is an indicator that they are seen as having the potential for commercial value even if they have not yet reached the market. The 2014 Survey shows that seventy-eight percent of 2014 Survey respondents reported receiving additional investment funding in the technology related to the surveyed project. (Finding II-C)
• SBIR/STTR funding makes a substantial difference in determining project limitation, scope, and timing. The 2014 Survey data show that seventy-one percent of respondents reported that the project probably or definitely would not have proceeded without SBIR/STTR funding. (Finding II-E)

Fostering the Participation of Women and Other Underserved Groups in the SBIR/STTR Programs

• Current data show that the objective of fostering the participation of women and underserved minorities has not been met by the DoE SBIR/STTR programs. (Finding III-A)
• Woman-owned firms accounted for less than 9 percent of Phase I SBIR and STTR awards in FY 2005-2015. The average success rates for Phase I applications by firms owned by woman and white males were 15.7 percent and 18.9 percent, respectively, during this period. (Finding III-A)
• Minority-owned firms accounted for less than 7 percent of Phase I SBIR and STTR awards during FY 2005-2015. (Finding III-A)

• Among respondents to the 2014 Survey, the vast majority of “minority” firms were in fact owned by Asians. Firms owned by Blacks, Hispanics, and American Indians accounted for 2 percent of all responses (including zero Black-owned and American-Indian owned firms). (Finding III-A)
• DoE is making efforts to understand the patterns of woman and minority participation in the SBIR program, but more is needed. (Finding III-C)

Stimulating Technological Innovation and Meeting Agency Mission Needs

• The DoE SBIR/STTR programs support the development and adoption of technological innovations that advance the agency’s mission. (Finding IV-A)
• The DoE SBIR/STTR programs connect companies to universities and research institutions. Among SBIR awardees responding to the 2014 Survey, 43 percent reported a link to a research institution related to the surveyed project; 26 percent reported that faculty worked on the project (not as a PI); 21 percent employed graduate students for the project; and 29 percent used universities and research institutions as subcontractors for the surveyed project. (Finding IV-B)

Fostering Innovative Companies

• The DoE SBIR/STTR programs encourage new firm start-up. Forty-five percent of companies responding to the 2014 Survey indicated that the company was founded entirely or in part because of the SBIR/STTR programs. (Finding V-A)
• Sixty-one percent of respondents to the 2014 Survey indicated that the DoE SBIR/STTR programs “had a highly positive or transformative effect” on their company. Another 35 percent said that it “had a positive effect.” (Finding V-C)

STTR

• STTR is meeting the program objectives defined in the Small Business Administration’s Policy Guidance for STTR. (Finding VI-A)
• Analysis of STTR in particular suggests that National Laboratories generally do not make good formal partners for small business concerns: their administrators do not prioritize SBIR/STTR because the funding amounts are small; and small businesses have limited leverage if the Laboratories fail to meet their obligations. (Finding VI-E)
• The DoE SBIR and STTR programs have not made sufficient efforts to enhance collaborations between the National Laboratories and small innovative firms. (Finding VI-E)

KEY RECOMMENDATIONS

Although the DoE SBIR/STTR programs generate substantially positive outcomes, the committee has identified a series of recommendations to improve their processes and outcomes. The order of these recommendations reflects the relative emphasis of the committee.

Improving Monitoring, Evaluation, and Assessment

• Although DoE recognizes the need for better data and is working to improve tracking mechanisms, more remains to be done in this area. (Recommendation I)
• DoE should improve current data collection approaches and methodologies. (Recommendation I-A)
• DoE should ensure that the outcomes data it now collects are systematically employed to guide program management. (Recommendation I-B)
• DoE should prepare a comprehensive SBIR/STTR Annual Report that replaces current reporting requirements and provides a clear picture of program operations to the Secretary of Energy, Congress, and the public. (Recommendation I-C)

Addressing Underserved Populations

• DoE should immediately extend past and current efforts to foster the participation of underserved populations in the SBIR/STTR programs, develop an outreach and education program focusing on these populations, and create benchmarks and metrics to relate the impact of such activities. (Recommendation II)
• While DoE should strive to increase participation of under-represented populations in the SBIR/STTR programs, it should not develop quotas for that purpose. (Recommendation II-A)
• DoE should accelerate its efforts to develop new benchmarks and metrics. (Recommendation II-B)
• DoE should develop an outreach and education program focused on expanding participation of underserved populations. DoE should provide significant management resources, because these outreach efforts are likely to be difficult and long term, and should consider designating a senior staff member to work exclusively on outreach to women and minorities to improve reporting and deployment of the new initiatives. (Recommendation II-C)
• DoE should review selection procedures and remove any identified biases in the selection process. DoE should ensure that patterns of

Improving Commercialization Outcomes

• DoE should support the commercialization of SBIR and STTR supported technologies beyond the completion of Phase II. (Recommendation III-A)
• DoE should review the effectiveness of its commercialization support and training initiatives. (Recommendation III-B)

Improving Linkages to National Laboratories

• DoE should seek to develop programs linking Laboratories’ procurement actions with relevant SBIR/STTR projects. (Recommendation IV-A)
• DoE should seek ways to ensure that Laboratories fully understand and respect the intellectual property (IP) provisions of SBIR/STTR. (Recommendation IV-B)
• DoE should examine from a strategic perspective how the relationship of SBIR/STTR with the National Laboratories works today. (Recommendation IV-C)

Improving Program Management

• DoE should improve its topic development process. (Recommendation V-A)
• DoE should change the balance of funding to better reflect innovation and commercialization opportunities in the private sector. (Recommendation V-B)
• DoE should review and possibly rethink the relationship between the National Laboratories and SBIR/STTR. (Recommendation V-D)
• DoE should improve its application review system and monitor the profile of applicants. (Recommendation V-E)
• DoE should consider whether its current requirements for Phase I commercialization plans are appropriate. (Recommendation V-E)
• DoE should ensure that the selection criteria are fully transparent. (Recommendation V-E)
• DoE should monitor the percentage of multiple awards and the composite age of company applicants (e.g., ratio of startups to mature companies) who are applying for and receiving awards. Careful monitoring and study should inform the question of whether “small” or “young” companies are more effective in generating state-of-the-art technology and innovation in the context of SBIR. This evidence can

• be used by Congress to determine if encouraging participation by younger firms furthers the missions of the SBIR program. (Recommendation V-E)

• DoE should further address the funding gap between Phase I and II awards. (Recommendation V-F)

This page intentionally left blank.