Summary

Created in 1982 through the Small Business Innovation Development Act, the Small Business Innovation Research (SBIR) program remains the nation’s single largest innovation program for small business. The SBIR program offers competitive awards to support the development and commercialization of innovative technologies by small private-sector businesses. At the same time, the program provides government agencies with technical and scientific solutions that address their different missions.

Adopting several recommendations from the 2008 National Research Council (NRC) study of the SBIR program, Congress reauthorized the program in December 2011 for an additional 6 years. In addition, Congress called for further studies by the Academies. In turn, the National Aeronautics and Space Administration (NASA) requested the Academies to provide a subsequent round of analysis, focused on operational questions with a view to identifying further improvements to the program.

FOCUS ON LEGISLATIVE OBJECTIVES

The SBIR programs are unique efforts designed by Congress to provide funding via government agencies in pursuit of four key objectives. These objectives are described in the Small Business Administration (SBA) Policy Directive that guides program implementation at all agencies. Section 1c of the Directive lists the program objectives as follows:

SCOPE OF THE ASSESSMENT

This study recognizes that the NASA SBIR program is distinctive in terms of scale, integrity, and mission focus. It does not purport to benchmark the NASA SBIR against SBIR programs at other agencies or non-SBIR programs in the United States or abroad. Further, the study does not consider if the NASA SBIR should exist or not; rather, it assesses the extent to which the SBIR program at NASA has met the congressional objectives set for the program, examining the extent to which recent initiatives have improved program outcomes and providing recommendations for further improving the program to meet its objectives.

It is also important to note at the outset that this study does not seek to provide a comprehensive review of the value of the SBIR program, in particular measured against other possible alternative uses of federal funding. This is beyond the study scope. Our work is focused on assessing the extent to which the SBIR program at NASA has met the congressional objectives set for the program, to determine in particular whether recent administrative initiatives have improved program outcomes, and to provide recommendations for improving the program further.

STUDY METHODOLOGY AND LIMITATIONS

The committee’s findings are based on a complement of quantitative and qualitative tools including a survey, case studies of award recipients, agency data, public workshops, and agency meetings. The methodology is described in Chapter 1 and Appendix A of this report. In reviewing the findings below, it is important to note that the Academies’ 2011 Survey—hereafter referred to as the 2011 Survey—was sent to every principal investigator (PI) who won a Phase II award from NASA, FY1998-2007 (not the registered company points of contact [POC] for each company. Each PI was asked to complete a maximum of two questionnaires, which as a result excludes some awards from the survey. The preliminary population was developed by taking the original set of SBIR Phase II awards made by NASA during the study period and eliminating on a random basis awards to PIs who received more than two awards (to limit the burden on respondents). The resulting preliminary population was 1,131 awards. PIs for 641 of these awards were determined to be not contactable at the SBIR company

listed in the NASA awards database. The remaining 490 awards constitute the effective population for this study. From the effective population, we received 179 responses. As a result, the response rate in relation to the preliminary population was 15.8 percent and in relation to the effective population response was 36.5 percent.

The committee acknowledges that because it was not possible to collect information from non-respondent PIs and because the agencies have minimal information about PIs which could be used to track potential non-respondent biases, we do not have data on which to develop an analysis of non-respondent bias. The committee has concluded that the data are likely to be biased toward PIs who are still working at companies that are still in business as corporate entities (i.e. have not failed or been acquired).

The absence of usable quantitative outcomes data from NASA further limits the conclusions that can be drawn from this assessment. Although the 2011 Survey provides quantitative data on NASA outcomes agency-wide, the number of responses is too limited to permit definitive conclusions. Similarly, although the limited data provided by NASA and that provided by Department of Defense (DoD) on NASA projects recorded in the DoD Company Commercialization Record database are helpful, neither is comprehensive.

Given the size of the survey population and response rates and overall potential sources of survey bias, the following findings and recommendations rely more heavily on company case studies, discussions with agency staff, and other documentation than we would have preferred. The committee’s findings are accordingly qualified.

KEY FINDINGS

Although more and better data would improve the grounding for these findings, it is our judgment that the NASA SBIR program is encouraging the expansion of technical knowledge. And although the limited data available from the 2011 Survey indicates limited infusion of SBIR technologies into NASA Mission Directorates for awards made in FY1998-2007, the program has since then become increasingly aligned with NASA Mission Directorate needs. NASA SBIR projects commercialize at a level similar to that of comparable SBIR programs at DoD, although the small size of the NASA market limits the scale of commercialization. However, with regard to the third program objective, we conclude that the NASA SBIR program is not adequately fostering and encouraging participation by women and minorities and socially and economically disadvantaged small businesses.

The findings are organized in terms of the four legislative objectives of the SBIR program plus findings on the management of the program.

Commercialization

• In the main…it appears that many NASA SBIR companies are affected by the small size of the NASA marketplace and sometimes very long lags as technology matures and large scale programs evolve toward completion. In some ways, they also suffer from the NASA SBIR program’s focus on NASA’s specialized needs. (Finding I)
• NASA SBIR projects commercialize at a substantial rate. Forty-six percent of respondents to the 2011 Survey reported some sales. An additional 26 percent reported that they anticipate future sales. (Finding I-A)

Meeting the Mission Needs of the Agency

• The lack of comprehensive quantitative data concerning the agency uptake of SBIR-funded technologies prevented effective determination of the program’s impact within NASA. NASA was unable to provide comprehensive data on follow-on contracts after Phase II. The new data collection mechanism may provide better data in the future. (Finding II-A)
• Responses to the 2011 Survey shows limited uptake of SBIR projects within NASA. An average of about 20 percent of reported project sales were to NASA or NASA primes. (Finding II-B)
• There is qualitative evidence on uptake of SBIR-funded technologies within NASA. Company case studies provide examples of technologies that were used on NASA missions or that made important contributions to NASA operations. (Finding II-C)

Fostering the Participation of Women and Other Under-represented Groups in the NASA SBIR Program

• The levels of participation by minority-owned and woman-owned firms in the NASA SBIR program are low and in some areas falling. Data from NASA indicate that approximately 8 percent of Phase I awards in FY2014 went to Woman-owned Small Businesses (WOSBs). Approximately an equal share went to Minority-owned Small Businesses (MOSBs). (Finding III-A)
• Phase I success rates (awards/applications) for MOSBs were lower than those for non-MOSBs every year since FY2005. In FY2014 the gap was more than 20 percentage points. (Finding III-A)
• Phase I success rates for WOSB were lower than those for non-WOSBs in all the years studied. The gap was largest in FY2014, about 20 percentage points. (Finding III-A)

• MOSB shares of Phase II awards fell substantially: MOSB firms received 19 Phase II awards in FY2009 and 5 in FY2014. Their share declined by over half after FY2006. The WOSB share of Phase II awards declined to below 8 percent in FY2011 and FY2012. (Finding III-A)
• MOSB Phase II success rates in every year FY2005-2012 were lower than those for non-MOSB firms. Overall, MOSB success rates were 13 percentage points lower than those for non-MOSBs. (Finding III-A)
• Phase II success rates for WOSBs were lower than those for non-WOSBs in every year of the study period (FY2005-2012) except for FY2005. (Finding III-A)
• NASA has not made sustained efforts to “foster and encourage” the participation of WOSBs and MOSBs. (Finding III-B)
• NASA does not report on or sufficiently track participation by WOSBs and MOSBs. NASA provided no separate data on Black-owned and Hispanic-owned small businesses or on minority or female principal investigators (PIs). NASA does not maintain data on woman and minority PIs. (Finding III-C)

Enhancing Science and Technology

• The SBIR program at NASA supports the development and adoption of technological innovations. However there is growing misalignment between the enhancement of science and technology and the demands of meeting specific agency mission needs. (Finding IV-A)
• The NASA SBIR program continues to connect companies and universities. Just over 30 percent of respondents reported a link to a university. About 21 percent of respondents reported that a research institution was a subcontractor; about 15 percent, reported that university faculty worked on the project (not as PI); and 14 percent reported employing graduate students. (Finding IV-B)
• NASA SBIR projects generate substantial knowledge-based outputs such as patents and peer reviewed publications. More than 80 percent of respondents reported at least one resulting peer-reviewed publication related to the surveyed project. (Finding IV-C)

Fostering Innovation Companies

• The NASA SBIR program fosters the formation of innovative small companies. Forty percent of respondents said that the company was founded entirely or in part because of the SBIR program. (Finding V-A)

Program Management

• NASA’s SBIR program is not sufficiently driven by metrics. NASA lacks sufficient evidence on the operations of its SBIR program. (Finding VI-A)
• Many of NASA’s commercialization initiatives are potentially promising but are too recent to provide an outcome assessment. (Finding VI-B)
• NASA’s monitoring and evaluation of the SBIR program is insufficient. While NASA has initiated a tracking system focused on commercialization starting in 2012, participation is key. NASA has not provided metrics against which program improvement could be measured. (Finding VI-C)
• Some NASA program management practices do not reflect best practices. NASA’s SBIR contracts management is unnecessarily rigid. Funding gaps between Phase I and Phase II have not been effectively addressed. (Finding VI-D)

KEY RECOMMENDATIONS

The following recommendations, which are organized in terms of four sets of leading actions needed to improve the SBIR program at NASA, can help improve outcomes.

Furthering a Culture of Monitoring, Evaluation, and Assessment Predicated on Enhanced Information Flows

• The NASA SBIR program should improve current data collection approaches and methodologies. NASA should make it a top priority to develop and implement appropriate metrics for assessing program outcomes. (Recommendation I-A)
• NASA should better use the data it collects on the SBIR program. These data should be utilized to systematically guide program management. (Recommendation I-C)
• NASA should improve its reporting on the SBIR program. The NASA SBIR program should provide a comprehensive annual report to Congress and the public on its operations. (Recommendation I-D)

Addressing Under-represented Populations

• NASA should immediately enhance efforts to address the Congressional mandate to foster the participation of under-represented populations in the SBIR program. (Recommendation II)

• Quotas are not necessary. While NASA should strive to increase participation of under-represented populations in the SBIR program, it should not develop quotas for that purpose. (Recommendation II-A)
• NASA should develop outreach and education programs focused on expanding participation of under-represented populations. NASA should develop a coherent and systematic outreach strategy that provides for cost-effective approaches to enhanced recruitment, developed in conjunction with other stakeholders and with experts in the field. This may in part build on existing efforts at some Field Centers, notably those at the Johnson Space Center as well as other efforts to enhance diversity at NASA. (Recommendation II-C)

Commercialization

• NASA should improve support for the commercialization of SBIR technologies by: (Recommendation III-A)
  • Identifying and applying best practices. Potentially, different approaches adopted by various NASA Field Centers could provide valuable data on more and less effective commercialization support strategies. Such analysis would of course require better data and a commitment to this kind of analysis and subsequent follow-up. (Recommendation III-A)
  • Leveraging existing programs and opportunities. For example NASA should explore more systematic ways to connect with DoD SBIR commercialization efforts, particularly given the significant overlap between NASA and DoD revealed by the 2011 Survey and the DoD CCR database. (Recommendation III-A)

Improving Program Management

• NASA should improve the application process. NASA should improve connections with applicants prior to application. (Recommendation IV-A)
• NASA should adopt more flexible contracting practices to encourage firm participation in the program. (Recommendation IV-B)

This page intentionally left blank.