Summary Report – Evaluation of the NRC’s Industrial Research Assistance Program

About the program

This evaluation of the National Research Council of Canada's Industrial Research Assistance Program (NRC IRAP) covered the period of FY 2017-18 to FY 2021-22. The program is evaluated every 5 years in accordance with the Financial Administration Act (FAA), and was last evaluated in Fiscal Year (FY) 2016-17.

NRC IRAP is an innovation assistance program designed to help Canadian SMEs grow through innovation and technology. The program is one of the flagship platforms through which the federal government supports small and medium-sized businesses (SMEs) to stimulate wealth creation in Canada.

NRC IRAP helps SMEs build their innovation capacities and take their ideas to market. It supports client firms through financial assistance for research and development (R&D), advisory services and fostering linkages to an extensive network of regional, national and international expertise, programs and partners.

NRC IRAP is delivered across Canada by five regional teams. The program is mainly delivered by Industrial Technology Advisors (ITAs) who offer technical and business advice, referrals and other services from concept to commercialization.

• $1.3 billion were disbursed to clients by NRC IRAP over the evaluation period
• 267 ITAs assist SMEs across the country in a variety of sectors
• 29% of clients are in the information and communication technology (ICT) or digital technology sector

Key findings

Reach

NRC IRAP is reaching high-potential and innovative SMEs as intended. It reached more clients than it did in the previous evaluation period (FY 2013-2017).

The program funded a higher proportion of SMEs owned by diverse Canadians than the proportion of SMEs owned by Canadians overall, based on an analysis of available program and national data (i.e. 2017 Statistics Canada data).

More youth, and specifically more youth from underrepresented groups, were reached by the program. More than ever before, youth identifying as female, visible minorities, Indigenous and persons with disabilities were placed on client projects.

The quality of program reach improved. The introduction of Client Engagement Advisors (CEAs) and the continued utilization of a portfolio approach positively impacted the quality of clients reached by improving focus on high-potential firms.

NRC IRAP reach among government partners increased. The business and technical assessment services provided by ITAs to government partners also continued to expand.

• Total reach (funded and unfunded): 22,458 supported firms (+20% over previous period)
• Advisory services only: 13,699 firms (+21% over previous period)
• Youth reach: 7,232 youth placements (+48% over previous period)

Effectiveness

NRC IRAP facilitated increased access of SMEs to an ecosystem of support. Clients reported that engaging with the program de-risked their projects, making them more attractive to other providers of innovation support services.

The program enhanced client innovation capacity and productivity. The full spectrum of support provided by NRC IRAP allows SMEs to achieve their innovation objectives and puts them on a better path towards growth.

NRC IRAP accelerated SME growth. Participation in the program enabled funded clients to not only increase their revenues but also their workforce. Firms led by women and visible minorities grew at a similar or better rate.

The program supported sustainable wealth creation in Canada. The majority (64%) of clients were profitable following the completion of their NRC IRAP projects. Funded clients supported over 62,000 Canadian jobs.

The net benefit of the program to the Canadian economy is estimated to be 5.3 times greater than the program costs.

NRC IRAP clients are supporting the fight against COVID-19. The program has funded clients to develop vaccines and therapeutics and produce personal protective equipment, sanitization solutions and medical devices.

Compound annual growth rate of funded firms in revenue and employment exceeded targets

Figure – Text version

An overlapping horizontal bar chart representing the compound annual growth rate of firms in revenue and employment of NRC IRAP clients for the evaluation period. The figure shows that the compound annual growth rate target for both revenue and employment were exceeded for the evaluation period by 12% and 3% respectively.

• 90% of funded firms reported enhanced ability to conduct R&D (+39% over previous period)
• Net benefit value over program costs: $16 billion

Efficiency

NRC IRAP has generally been delivered in an efficient and effective manner. The vast majority of clients (91%) are satisfied with their interactions with the program and with the services provided. The program was found to be flexible, agile and well-designed to meet the needs of intended clients. It was able to rapidly respond to evolving government priorities, and introduce new services and programs as client needs arose.

NRC IRAP continued to serve client needs during the COVID-19 pandemic while simultaneously implementing and supporting new temporary pandemic-related programs. Both the program and its clients were operating in a highly uncertain and challenging context during the last two years of the five-year evaluation period (FY 2017-2022).

Opportunities to maximize program efficiency exist. The evaluation identified the following areas for improvement: enhanced solutions to human resources challenges (e.g. surge capacity and ITA recruitment and hiring), refinements to Large Value Contributions (LVC) and IRAP International project development and approval processes, better communications and integration of GBA Plus considerations in programming and new tools to strengthen performance measurement.

Relevance

NRC IRAP is responsive to government efforts to foster an entrepreneurial society and support global science excellence. The program also aligns with government efforts to increase innovation clusters, the ease of doing business with Canada and SME competitiveness in a digital world. It additionally supports government efforts to accelerate clean growth.

NRC IRAP proved to be a key and increasingly relevant player in Canada's innovation ecosystem. It fills a gap in the SME innovation support ecosystem and for government partners through the provision of technical and business expertise.

Clients and stakeholders perceive NRC IRAP as unique among government innovation support programs in the combination and quality of services that it offers to SME and government partners. Services offered by funded organizations are considered to be valuable and important to the enhancement of SMEs' innovation capacities.

Progress towards full alignment with government priorities around Gender-based Analysis Plus (GBA Plus) has been positive, although it could be better communicated to staff. The program continues to evolve from data collection to the actual use of data.

Recommendations

1. NRC IRAP should explore alternative solutions to tackle current human resource challenges, including options for improving surge capacities, reviewing hiring and promotion practices and better aligning work planning with available resources. Issues particularly affecting staff in the ITA, CEA and Regional Contribution Agreement Officer (RCAO) roles should be prioritized.
2. NRC IRAP should further refine project development and approval processes and tools, particularly those related to LVC and IRAP International, to enhance overall service delivery and the integration of these components in regular programming.
3. NRC IRAP should ensure that its existing inclusiveness strategy and initiatives are better communicated to and operationalized by staff in order to ensure enhanced integration of GBA Plus considerations into all aspects of the program from planning to execution and reporting.
4. NRC IRAP should strengthen its performance measurement system and further its use of data analytics to inform program management processes.

About the evaluation

The evaluation assessed the program's reach, effectiveness, efficiency and relevance. It considered all NRC IRAP activities and expenditures, except for its collaborative programs with Global Affairs Canada and the Innovation Assistance Program (IAP). IRAP International—though included in the evaluation to some extent—was not a primary focus. The evaluation was conducted by the NRC's Office of Audit and Evaluation using a mixed-methods approach that included data and document reviews, internal and external interviews, case studies of funded firms, a survey and focus groups with ITAs, journey mapping of two unfunded firms, an international comparison study with four institutions and a partial benefit-cost analysis based on a selection of high impact firms funded by the program.

The full evaluation report, including recommendations to NRC IRAP and the management response and action plan, is available on the NRC website: https://nrc.canada.ca/en/corporate/planning-reporting/evaluation