Evaluation of the Strategic Innovation Fund (SIF)

From: Innovation, Science and Economic Development Canada

Evaluation of the Strategic Innovation Fund – PDF

April 2021

Table of Contents

• Background
• Methodology
• Findings: Relevance
• Findings: Performance
• Findings: Design and Delivery
• Findings: Efficiency
• Conclusions
• Annex A: SIF logic model
• Annex B: Technology Readiness Levels (TRL) Scale
• Annex C: Case Studies
• Annex D: Legacy Programs
• Annex E: Summary Statistics for SIF Projects
• Footnotes

Background

SIF Overview

The Strategic Innovation Fund (SIF) is an Innovation, Science and Economic Development Canada (ISED) program delivered by Innovation Canada. The SIF was created in 2017 and involved the consolidation of four legacy ISED programs (refer to Appendix D for details). The program is a claims-based contribution program that supports large-scale, transformative and collaborative projects that help position Canada to prosper in the global knowledge-based economy. SIF projects promote the long-term competitiveness of Canadian industries, clean growth, and the advancement of Canada's strategic technological advantage.

The SIF covers all sectors of the economy and is available to for-profit and not-for profit organizations with the goal of supporting the Canadian innovation ecosystem. The objectives for each of the five SIF streams^{Footnote 1} are outlined below.

Business Innovation and Growth

• Stream 1: Encourage R&D that accelerate technology transfer and commercialization of innovative products, processes and services.
• Stream 2: Facilitate the growth and expansion of firms in Canada.
• Stream 3: Attract and retain large-scale investments to Canada.

Collaborations and Networks

• Stream 4: Advance industrial research, development and technology demonstration through collaboration between the private sector, researchers and non-profit organizations.
• Stream 5: Support large-scale, national innovation ecosystems through high impact collaborations across Canada.

As of March 31, 2020, 66 projects have been announced, with a total of $2.1 billion in SIF contributions committed to projects under five streams:

61 projects under streams 1, 2, and 3
4 projects under stream 4
1 project under stream 5

SIF in the Canadian innovation ecosystem

The SIF is one of the flagship platforms within the Government of Canada's business innovation funding programming, along with the Industrial Research Assistance Program (IRAP) of the National Research Council (NRC) and the innovation programs of the Regional Development Agencies (RDAs)^{Footnote 2}. The SIF complements these other funding platforms by supporting large-scale projects and high-growth potential firms, whereas programs delivered by IRAP and the RDAs support smaller-scale, regional innovation projects undertaken primarily by small and medium enterprises (SMEs).

IRAP and RDAs funding threshold is of up to $10 million per project and SIF funding is for projects over $10 million.

SIF Supports large scale projects, leverages private investments, secures economic, innovation and public benefits for Canadians and offers different repayment terms^{Footnote 3}.

Methodology

About the evaluation

Evaluation objectives and scope

This is the first evaluation of the SIF. The objectives of this evaluation were to assess the relevance, performance and efficiency of the program, which included issues identified by SIF management. The evaluation was conducted in accordance with the Treasury Board Policy on Results and Directive on Results and covered the period of April 1, 2017, to March 31, 2020. The evaluation focused on streams 1 to 3 of the SIF, as the projects in streams 4 and 5 were newer streams, thereby limiting the ability to assess these related outcomes. For streams 1 to 3, the evaluation assessed the immediate outcomes and, to the extent possible, the preliminary results pertaining to the intermediate outcomes. See Annex A for the SIF logic model.

Evaluation approach

ISED's Audit and Evaluation Branch (AEB) conducted the evaluation of the SIF. There is a Financial Administration Act requirement to evaluate the Automotive Supplier Innovation Program by the 2020-21 fiscal year. In lieu of conducting an evaluation of the ASIP, the evaluation of the SIF is being conducted in the 2020-21 fiscal year because the ASIP was subsumed under the SIF.

The evaluation was calibrated to consider both the complexity (size, design and delivery) and the evaluation context (logic model and management information needs) of the SIF, which was informed by an evaluability assessment of the SIF conducted in 2019. Specifically, the scope of the evaluation was calibrated to consider:

• Program implementation and outcomes;
• Data availability; and
• Scope and scale of the SIF.

Evaluation Questions

Relevance

1. To what extent is there a demonstrable need for the SIF?

Performance

Immediate Outcomes

2. To what extent has the SIF contributed to investments in R&D, commercialization, and industrial or technological facilities or projects in Canada?
3. To what extent has the SIF contributed to investment in industrial research and large-scale technology demonstration projects?
4. To what extent has the SIF supported collaboration between universities, colleges, research institutes, not-for-profit organizations and the private sector?

Preliminary Progress for Intermediate Outcomes

5. To what extent has the SIF contributed to the skills and technological capacity of recipients?
6. To what extent has the SIF contributed to the demonstration and development of innovative products, services or processes?
7. To what extent has the SIF contributed to the attraction, retention and growth of business investments?

Program design and delivery

8. To what extent has the program's outreach activities raised awareness about the program among targeted groups.
9. To what extent has the program design and delivery model allowed for flexible and streamlined services to businesses through a single window?

Efficiency

10. To what extent does the program demonstrate operational efficiency?

Data Collection Methods

The evaluation relied on multiple lines of evidence to address program relevance, performance and efficiency, including both qualitative and quantitative research methods.

Interviews: Interviews were conducted via MS teams and teleconference with 48 key stakeholders, including: ISED officials managing the SIF; SIF recipient firms; consultants that support applicants; intermediary projects for networks and consortiums; other federal and non-federal partners; universities, colleges and research institutions; industry associations; and industry experts.

Case Studies: Case studies were conducted for 10 SIF projects (see Annex C for details). The case studies were selected to be representative of the funding streams under the SIF and sector-specific funding allocations (e.g., steel and aluminum sector funding), recipient type (sole recipient projects, partnership projects, networks, and consortiums), industry sector, region, and business size.

Environmental Scan: An environmental scan was conducted to identify information on the following key areas: innovation funding programs in other countries; the Canadian innovation landscape; demonstrable need among Canadian industry for direct government support for innovation; funding available to SIF recipients from other federal and provincial programs; and gender and diversity considerations in the context of innovation funding.

Document Review: A document review was conducted to gain a thorough understanding of the SIF. The document review was comprised of key program and government priority-setting documents to support the assessment of the program relevance, design and delivery model and performance.

Data Review: Data related to the SIF projects was reviewed to assess program need and performance. This included administrative data, Annual Performance Benefits Report (APRB) data (for the 2018-19 reporting period only) and financial data.

Survey: An online survey was distributed to 56 recipients of SIF funded projects to assess the implementation of the SIF in terms of program need, design and delivery, as well as outcomes.

Limitations and Mitigation Strategies

The evaluation encountered some limitations and applied the mitigation techniques outlined below.

Response burden on Canadian firms: Recipients have various reporting requirements under the SIF (progress reports, project management reviews, financial reports, Annual Performance Benefits Report (APBR), etc.). The COVID-19 pandemic has also placed additional strain on the capacities of recipients, potentially affecting their responsiveness to participate in case studies, interviews and surveys. To minimize the response burden on firms, the evaluation tried to maximize the use of existing program data (e.g., APBR) in lieu of collecting new data from stakeholders and coordinated data collection activities with the SIF program area, where feasible.

Time factor: While offering the advantage of providing early insights on program implementation and outcomes, the timing of the evaluation limited the availability of information on forward-looking outcomes. The intermediate and long-term outcomes of the SIF tend to accrue in the later stages of project implementation. The first projects funded under the SIF were announced in January 2018. As a result, there were approximately two years of project activities undertaken for earlier projects. As most SIF projects have lifespans of 3-5 years, not enough time has elapsed for most SIF projects to have fully realized the intermediate impacts. Further, the longer-term impact of large-scale projects, such as those funded under the SIF, may extend many years beyond the project life span. Only a few projects were completed at the time of the evaluation. Lastly, for streams 4 and 5, the Contribution Agreements had not been executed within the period covered by the evaluation (April 1, 2017 to March 31, 2020). To address this limitation, anecdotal evidence for intermediate outcomes was collected, where possible.

Attribution: The SIF is one of many factors that could have contributed to the success of funding recipients, making it challenging to directly attribute the impact on industry to SIF support. The presence of other partners (e.g., recipient firms, other federal programs^{Footnote 4}, other levels of government, etc.), who may have funded different phases or components of SIF funded projects, presents limitations in measuring the full impact of the SIF's contribution. To mitigate the impact of this challenge, interview and survey questions focused on identifying the SIF's incremental contribution to the projects.

Data reliability: Participating firms self-report, on an annual basis, performance-related information for their projects. In some cases, when exact figures were not available recipients were asked to provide estimates. By using a self-reporting mechanism, data reliability could be called into question. This was mitigated by the extensive efforts undertaken by the SIF to validate the information provided by recipients. Further, the evaluation was conducted while the funded projects were still ongoing. There was only one year of self-reported data available for the assessment of results and limited data available for intermediate and longer-term outcomes (e.g., innovation and intellectual property). Another challenge was that recipient projects have different timeframes and different distributions of expenditures over their project lifecycle. Consequently, a single year of project data may not capture a representative sample of a project's expenditures over its lifecycle. To help mitigate these methodological limitations, interviews with firms, case studies and a survey were used to validate and contextualize the performance information.

GBA+: There was limited data available pertaining to the demographic characteristics of recipients, such as gender, age, minority status, and indigenous status. Gender disaggregated data is not collected in the APBR, nor was gender and diversity data collected from program applicants. While the SIF has recently begun collecting some of this data in its application forms, this data was not available for projects covered in this evaluation. This consequently limits the extent to which GBA+ analysis can be incorporated into the evaluation of the program. To help mitigate this methodological limitation, interviews with firms, case studies and a survey were used to supplement the limited quantitative data. Data from these other lines of evidence primarily related to the gender and diversity requirements associated with the SIF funded projects.

Findings: Relevance

Need to support innovation among Canadian industry

Need for direct investment in innovation

The SIF was created to address weaknesses in Canada's industrial competitiveness, including:

• An over-reliance on indirect tax incentives for R&D;
• Low levels of collaboration between large firms, SMEs and academia;
• Commercialization and growth capital shortages for firms;
• A lack of tools to attract investments from multi-national enterprises;
• A lack of funding programs for some sectors of the economy; and
• A fragmented approach to business innovation programming.

Environmental scan affirms there is a need to support innovation, which is the key driver of long-term economic growth. Reports^{Footnote 5} suggest that innovation could be accelerated through increased direct support to business. Direct assistance has the advantage of being more strategic in supporting the development of innovation capacity, especially in emerging sectors and developing regions^{Footnote 6}. When strategic public investments are aligned with market forces and technology trends, they can have a significant impact in developing industrial and innovation capacity in new sectors^{Footnote 7}. Direct support can also be better targeted towards addressing innovation weaknesses specific to Canada^{Footnote 8} (e.g., lower levels of growth capital, commercialization, and collaboration).

The environmental scan found that the SIF's current funding approach is consistent with the approaches and best practices of other international comparator programs (in Germany, U.S., U.K., Switzerland and Finland) with regard to the beneficiaries, project duration, funding and type of contribution, and application process. These comparator programs have similar objectives as the SIF, including increasing commercialization, growing and expanding firms, increasing R&D, and increasing highly-skilled employment.

Did you know? OECD data indicates that as of 2017, direct support accounted for 26% of total government support for business expenditures on R&D in Canada, compared to the Organization for Economic Co-operation and Development (OECD) average of 54%.^{Footnote 9} Further, Canada's Business Expenditures on R&D as a percent of Gross Domestic Product was about 50% lower than the OECD average.^{Footnote 10}

Emerging needs and trends

Emerging needs and trends

• Interviews and case studies found that there were a variety of emerging needs and trends to be addressed by the SIF, with Industry 4.0 (the ongoing automation of traditional manufacturing and industrial practices using smart technology) and cleantech being the most frequently cited:
• Industry 4.0: Interviewees emphasized the need to improve productivity, capacity, and competitiveness through accelerated development and adoption of Industry 4.0 technologies, with advanced robotics and automation, artificial intelligence (AI), machine learning, big data, and digitization cited frequently by stakeholders.
• Cleantech/Clean Growth: Interviewees noted a growing need for the development and adoption of green and renewable technologies, products and processes to reduce the environmental and carbon footprint of industry.

Other emerging needs and trends cited often by interviewees included:

• Inter-industry collaboration (e.g., digital sector collaborations with the resource, agri-food or manufacturing sectors);
• Investment in capital equipment in the manufacturing sector;
• Investments in large-scale R&D and commercialization projects;
• Equity, diversity and inclusion measures in the workplace; and
• Counterbalancing the trend towards increased foreign acquisition of Canadian companies and intellectual property (IP).

Steel and Aluminum (S&A) Highlight: The S&A stream was created in response to U.S. tariffs imposed in 2018. The stream supports investments in new equipment and technologies, workforce retention, and strengthening Canadian capabilities. The SIF was seen as the best venue to provide efficient and timely funding given its experience, flexibility, and governance mechanisms.

Interviews and case studies found this funding allowed the sector to maintain operations and increase investments in modernization during a period of uncertainty, helping to mitigate foreign acquisition of Canadian firms and improve competitiveness.

Needs and trends addressed by the SIF

Industry sector needs addressed

The survey found that the SIF meets recipients' project and organizational needs, as well as the needs of their sector and the region in which the project was located. Interviews and case studies also found that the SIF meets the emerging needs of sectors.

Interviews found that the SIF helped to:

• Catalyze, incentivize, and accelerate innovation and increase productivity in key sectors;
• Support clean growth and environmental objectives;
• De-risk projects and support those with high up-front costs;
• Attract and leverage additional capital investments;
• Support a mandate attraction for multinationals;
• Fill a significant funding gap in the growth and expansion of firms;
• Bring players in different industries together to collaborate; and
• Support job creation and the development of Highly Qualified Personnel.

Equity, Diversity and Inclusion (EDI) measures

The SIF includes EDI criteria in its assessment of project proposals and often includes a requirement in contribution agreements that companies develop and implement EDI plans. The document review and interviews found that the majority of projects had gender and diversity requirements, usually as an EDI plan aimed at increasing representation. The survey found that less than half of recipients (40%) perceived the SIF to meet the needs of underrepresented groups to a great or very great extent. Interviews found that while the SIF acted as a catalyst for companies without an EDI plan, many recipients already had plans in place prior to their SIF project. In many of these cases it was noted that the SIF enhanced existing EDI plans via specific targets included in the Contribution Agreement.

Case Study Highlights:

General Fusion: The SIF reduced the risks associated with large scale demonstration of clean energy technologies.

Smart Grid Atlantic Project: The project allowed Siemens Canada to bridge a market maturity gap and position itself as a market leader.

STEMCELL Technologies: The SIF provided the capital needed to scale and propel the recipient's growth.

Demand and uptake of the SIF

Interest from target groups

The SIF expanded funding to support sectors with untapped potential for innovation not previously targeted by ISED's legacy funding programs. While the legacy programs targeted the automotive and aerospace sectors only, the SIF was expanded to also target the health and bio-sciences, information and communications technology, agri-food, natural resources, and cleantech.

The SIF generated a large amount of interest from industry, with almost 1,100 applications received as of March 31, 2020, which is significantly higher than the target of 100 applications per year. The majority of applications however were ineligible, as the requested contributions were less than the $10 million minimum established in 2018. The SIF supported SMEs and leveraged investment from multi-national enterprises. Almost half of funded projects were for SMEs and 41% were foreign direct investment projects. SIF funding tended to be more concentrated within the manufacturing and digital industries sectors, with the sectors targeted by the legacy programs comprising almost half of these funded projects. There were few projects funded in the clean technology, agri-food, and health/bio sciences sectors, particularly when compared to the number of applications. However, many of the SIF projects (37%) involved cleantech as a secondary industry sector (due to its cross sectoral nature).

Ontario and Quebec represented the largest share of projects and were slightly over-represented relative to the number of applications. In contrast, the Western Region was underrepresented relative to the number of applications received. The document review indicated that the distribution of funding corresponded to the provinces' level of business expenditures on R&D. This suggests that the lower level of representation may be due in part to the distribution of innovation ecosystems and clusters in Canada.

*Figures exclude applicant and recipient projects involving less than $10 million in SIF contributions.

Identified needs

Identified needs and areas for improvement

Sector-specific needs: Interviews found that the SIF could be more effective by providing additional flexibility for specific sectors and placing greater emphasis on the economic impacts of some projects (discussed further under the Design and Delivery section of the report). For example, interviewees found that the SIF is not meeting the needs of the agri-food sector, where program data confirms only one project was funded under the Business Innovation and Growth streams (streams 1-3). This stemmed in part from the way in which innovation is defined by the SIF and the SIF's emphasis on job creation, which runs counter to this sector's goal of automation, aimed at replacing jobs to address labour shortages. This issue does not apply to the Collaborations and Networks streams (streams 4-5), which provided targeted funding of $79.5 million for two network projects in this sector. While Agriculture and Agri-Food Canada's AgriInnovate program provides sector targeted funding, it is designed for smaller projects, offering a maximum of $10 million in funding. For the life sciences sector, it was indicated that few projects have been funded as the SIF generally does not fund projects involving clinical trials.

Support continuum: In Budget 2018, changes to the SIF were announced to focus support on projects requesting over $10 million in contributions. Despite corresponding increases in the maximum funding allowed under the NRC's IRAP and RDA programming from $1 million to $10 million^{Footnote 11}, ISED officials as well as other stakeholders interviewed identified that it was very difficult for companies to acquire funding when their project requires contributions of over $5 million and up to $10 million. The data review indicates that few projects received funding in this range, as for fiscal years 2018-19 to 2020-21 only one RDA had provided contributions above $5 million (8 projects). IRAP only funded 5 projects and only one project was provided more than $6 million in funding^{Footnote 12}. However, for cleantech projects SDTC also provided funding of over $5 million and up to $10 million for 13 projects between 2018-19 and 2020-21^{Footnote 13}.

Agri-Food Highlight: Food and beverage processing is the largest manufacturing industry in Canada (18% of manufacturing employment), with women constituting 41% of total employment.

It is an area of untapped potential because of underinvestment. For every dollar invested per worker in the U.S., Canadian facilities invest only 62 cents.

Food processors experience labour productivity challenges and lower profit margins compared to the overall manufacturing sector.

Findings: Performance

Influence on investment decisions

Influence on investment decisions of recipients

As of March 31, 2020, the SIF has provided funding for 66 projects, with a contribution commitment of $2.1 billion and a total of $43.8 billion leveraged in private sector investment. These projects also included almost $700 million in other government funding (e.g., the Ontario Jobs and Prosperity Fund). The ratio of private sector funding to SIF funding was almost 20:1, substantially higher than the initial target of 3:1 and also much higher than the ratios achieved for the legacy programs. The document review found that research, development and commercialization projects (stream 1) have the largest economic impact in Canada per dollar of project spending, due in part to higher expenditures on hiring new staff relative to capital equipment (which may be imported from outside of Canada). Investment attraction and reinvestment projects (stream 3) have the largest economic impact per dollar of SIF funding, as these projects leveraged the largest amount of private sector investment (due to the size of the investment and the lower cost sharing thresholds used for these types of projects).

Interviews, document review, case studies and the survey found that the SIF has a significant influence on the level of business investment in Canada and that without the SIF, key sectors of the economy would be put at a global disadvantage. The majority of recipients surveyed (60%) indicated that their project would not have occurred without SIF funding. This view was also found among interviewees, who indicated that some projects may not have occurred at all or may not have occurred in Canada. In cases where the project still occurred in Canada, the projects would have been delayed or the scale would have been reduced.

Influence on project location

Multinational investment attraction

Program data found that the SIF contributed to investments in new production mandates and the expansion and improvement of facilities among multinational enterprises. The survey found that for almost half of the recipients (45%), SIF funding had an influence on project location to a great or very great extent. Interviews and case studies found that without the SIF, it would have been more difficult to attract investments to Canada and for subsidiaries to attract investments from their parent companies. Recipients indicated that the SIF funding increased the financial rate of return, thereby making the projects more financially competitive investments relative to other investment options.

Interviews and case studies found that without the financial incentives provided by the SIF, multinational enterprises would invest in other jurisdictions. When a multinational is determining where to expand its mandate or facilities, financial incentives are an important consideration for the global head office. For example, one interviewee noted that their company moved from another jurisdiction to Canada solely because of SIF funding.

Case Study Highlights:

Toyota Motor Manufacturing Canada: The automotive sector is a 'pay-to-play' industry, with a high level of competition with other jurisdictions to attract investments.

Algoma Tubes Inc: If SIF funding was not available, it would have been more difficult to attract investment from the parent company when competing with other subsidiaries.

Elysis Limited Partnership: In the absence of the SIF, the project may have occurred in another jurisdiction or been delayed, disadvantaging it against competitors vying for first-mover advantages in zero carbon aluminum smelting.

Influence on project scope, scale and timing

Acceleration of project scale and timing

Interviews and the survey found that the SIF was most influential on the timing (76% of survey respondents), and to a slightly lesser extent, the scope (71% of survey respondents) and scale (66% of survey respondents) of projects. Interviews found that the SIF helped accelerate projects and created a higher profile for companies – enabling them to attract new internal investment or external investment from other sources.

Interviews and the survey also found that without the receipt of SIF funding to help support their projects, projects would have proceeded at a much slower pace and smaller scale, allowing competitors to catch up or get ahead, an especially crucial aspect when it comes to high-tech companies. Cleantech projects would also not have advanced as quickly, as other sources of funding, such as Sustainable Development Technology Canada, only fund smaller scale cleantech projects.

Without the SIF, there would also be a funding gap for high potential, high growth domestic firms. As a result, these firms would be more reliant on foreign investment – resulting in a dilution of Canadian ownership and putting them at increased risk of foreign acquisition, ultimately resulting in a loss of Canadian IP and talent.

Case Study Highlights:

Smart Grid Atlantic Project: The scope and scale of the project would have been significantly impacted and the project may not have occurred if the funding from the SIF had not been obtained.

General Fusion: The SIF served as a catalyst in attracting private capital, which was critical to the advancement of the project. If this funding was unavailable, the project would have occurred at a slower pace.

Influence on technology demonstration projects

Projects with technology demonstration components

Technology demonstration is a critical phase in the development of a new technology, often referred to as the 'valley of death' due to the difficulty in advancing technologies through this phase. There are significant costs and risks associated with this development phase and as a result projects are often abandoned due to a lack of capital. Interviews noted that a large scale program such as the SIF has a significant impact on de-risking technologies. Program data found that one-third of projects were at the technology demonstration stage when they applied for SIF funding, representing almost $300 million in contributions and $2 billion in total project investment leveraged. Further, the survey found that two-thirds of respondents indicated that their SIF funded projects had technology demonstration components, with the majority (54%) indicating that their project would not have occurred without SIF funding. The survey found the SIF funding was most influential on the scope of these projects, while also having a significant influence on the scale and timing.

The SIF also supports large scale collaborative projects focused on technology demonstration. There were two collaborative technology demonstrations funded by the SIF under the Business Innovation and Growth streams (streams 1-3) – Bell Helicopter and Innovation ENCQOR. These projects represented $116.2 million in contributions and $525.7 million in total project investments leveraged. Interviews, case studies, and the survey found that without the SIF funding, these projects likely would not have occurred in Canada. These projects involve collaboration amongst dozens of partners, including large firms, SMEs, and academic institutions. Program data indicates that various collaborations are occurring through these projects, with some partners reporting increased technological capacities.

Case Study Highlights:

Telesat: The project enables large-scale technology demonstrations to increase the Technology Readiness Level^{Footnote 14} of satellite equipment and enable testing of the new satellite technologies.

Smart Grid Atlantic Project: The funding enabled Siemens Canada to engage in large-scale technology demonstration projects with utility partners.

General Fusion: The large injection of capital is critical to the advancement of the project to technology demonstration and commercialization.

Influence on R&D and commercialization investments

Investments in R&D

Interviews and program data showed that the SIF was effective in leveraging increased investments in R&D. In addition to the $43.8 billion leveraged in project investments, $9.1 billion was specifically committed for R&D activities, with many of the R&D commitments extending beyond the life of the SIF projects. Program data indicates that about half of recipients had increased their businesses expenditures on R&D by at least 25%, which was one of the program's initial long term targets.

Interviews found that some companies moved their R&D facilities to Canada specifically due to the SIF. It was also found that SIF funding led to the increased use of universities to help with R&D, facilitated investments in prototypes, and improved capabilities to test products before beta trials with customers. Interviews and the survey found that high tech companies were more likely to indicate that the SIF led to increased investments in R&D.

Investments in commercialization

Part of the objective of the R&D investments is to ensure that they lead to commercialization. Interviews indicated that bringing an innovation to market is embedded in the structure of the SIF program, with the objective that companies secure IP and retain it in Canada. In terms of commercialization progress, interviews found that it may be too early to draw any conclusions, as most projects are still in the work phase.

R&D Intensity:

During the first year of project expenditures, project R&D constituted approximately half of the recipients' enterprise-wide R&D expenditures.

There was a significant increase in the R&D intensity of the recipient firms in year one of the SIF funded projects in comparison to the three previous years of recipients' R&D expenditures (R&D went from about 6.1% to 8.4% of revenues).

Influence on industrial facility investments

Investments in industrial production and research facilities

Program data showed that the majority of recipients (76%) reported that their project supported a new or retained R&D or production mandate. Program data also showed that the SIF was effective in leveraging capital expenditures, with project capital expenditures constituting approximately two-thirds of recipients' enterprise-wide capital expenditures. Over one-quarter of the projects (11 of 36), which included two steel and aluminum projects, had incurred expenses on upgrading or expanding existing facilities. This indicates that the SIF has made progress towards its initial target of 15 pre-existing facilities expanded by March 31, 2022. These projects also constituted most of the capital expenditures (96%) undertaken by recipients. The survey found that the SIF had a greater influence on increasing investments in industrial production facilities than it did on industrial research facilities. The SIF was also more influential on the retention, expansion, and improvement of facilities than on their establishment.

Interviews and case studies found numerous examples of the expansion, improvement or establishment of facilities, (e.g., the creation of a centre for additive manufacturing, a facility for stem cell research, etc.). It was noted that the SIF has contributed to increased investments in the expansion or improvement of existing industrial facilities for multinational companies. For cleantech companies, it was said that the SIF helps more with establishing production facilities that will lead to commercialization. For the steel and aluminum sector, the funding was intended to counter the effects of the U.S. tariffs. Thus, for this sector, the funding was mainly used for the expansion or improvement of existing facilities.

Foreign Direct Investment in Facilities

31% (19/61) of SIF funded projects involved FDI investments in the expansion or improvement of facilities.

11% (7/61) of SIF funded projects involved FDI investments in new production mandates.

Influence on collaboration

Collaborations

Innovation thrives when companies, investors, and universities collaborate to grow businesses, commercialize inventions and create new ventures. Despite Canada's strong talent base, large firms, SMEs, and research infrastructure, these stakeholders are not collaborating sufficiently, which inhibits commercialization of R&D. In 2017, Canada ranked 18th among OECD countries by university-industry collaboration.

To address this collaboration gap, the SIF includes collaboration requirements in the majority of its contribution agreements. The survey found that the majority of recipients (68%) would have engaged in fewer collaborations in the absence of the SIF project. Program data shows that two-fifths of recipients reported undertaking collaborations in the first year of their SIF funded project. While one project had 162 collaborations, the others had an average of 4 collaborations per recipient directly related to the SIF projects. The majority (59%) of these were with private sector firms, while a smaller percent (41%) were with universities, colleges, and non-profits.

Interviews and case studies found that for those indicating that SIF influenced collaborations, it was particularly the case in terms of working with universities – collaborations that likely would not have happened without the SIF. The SIF also helped open up new collaborations with suppliers in Canada. However, in some cases, projects required few external collaborations – this was more often the case for firms in the high tech sector. In other cases, collaborations were a key part of the projects but they were not necessarily influenced by SIF funding. For these recipients, although collaborations would occur without SIF funding, making them contractual commitments was viewed positively. Others indicated that companies are well suited to do collaborations on their own and that the SIF should encourage collaboration but not mandate it.

Case Study Highlights:

General Fusion:

The recipient increased their focus on collaborations with universities as a result of their SIF funded project.

Nova Scotia Power:

The recipient was already working towards increasing collaborations, but the SIF commitment helped ensure they were undertaken.

Domtar:

The recipient engaged in extensive collaboration with other subsidiary facilities.

Potential areas for improved support to collaboration and influence on networking opportunities

Networking

Most interviewees indicated that their projects led to new networking opportunities, or at the very least, allowed them to expand upon their existing networks. It was noted that having SIF funding increased the profile of companies and enabled them to attract more interest from other companies/suppliers and/or universities. The most prevalent new networking opportunity cited by interviewees was with universities – with some remarking that new opportunities opened up for co-op students from universities to work at SIF-funded firms.

Examples of potential improvements to collaboration

Interview and case study evidence identified a few areas where the SIF could potentially enhance its contribution to supporting the collaboration activities of the project recipients:

• Providing clear guidance early on about the types of collaborations supported and what the SIF is aiming to achieve through its collaboration requirements, as well as helping to facilitate the sharing of collaboration best practices among recipients.
• Providing recipients with more support in identifying and connecting with academic and non-profit organizations and researchers for collaborations on the SIF projects.
• For the Business Innovation and Growth streams (streams 1-3), encouraging companies to collaborate with non-traditional partners, as some sectors (e.g., agri-food, resource extraction) have lower levels of investment in areas such as IT and automation, and therefore would gain more in terms of improved productivity by collaborating in these areas.

Case Study Highlights:

Toyota Motor Manufacturing Canada

Publicity from the project helped spur networking and collaborations with automation companies.

Smart Grid Atlantic Project

The project is expected to result in increased collaboration opportunities with other potential utility company clients.

Due to the high volume of collaborations, challenges were noted in ensuring they contributed to project objectives rather than detracted from them.

Increased technological capacity of recipients and collaborators

Increased technological capacity of recipients

The survey found that SIF projects made a contribution to enhancing the technological capacity of the majority (63%) of recipients. Interviews also found that the SIF contributed to enhancing recipients' technological capacity at an accelerated pace. As well, interviews found that the SIF has facilitated the adoption of new technologies, equipment, and production processes, leading to gains in efficiency as well as new and expanded production capacity. These capacity enhancements will enable recipients to produce a broader range of products as well as more profitable, higher value-added products.

Most interviewees emphasised that technological capacity was enhanced as a result of an increase in the number of highly skilled employees hired and an increase in existing employees technical skills and knowledge (e.g., programming skills, production processes, machinery operation). For example, projects involving the enhancement of facilities resulted in the development of new employee skills as a result of the new equipment and production processes.

Technological capacity of collaborators on network/consortium projects

For the two network/consortium projects (ENCQOR and Bell Helicopter), the data review found that one-third of the collaboration partners (5 of 15) reported improvements in technological capabilities as a result of their participation in SIF-funded projects, indicating that the SIF has made progress towards its initial target of 80% of collaboration partners reporting improved capabilities by March 31, 2025. This included improvements in detection capabilities, software capabilities, technology integration, assembly processes, and various improvements to 5G technologies.

Case Study Highlights:

Toyota Motor Manufacturing Canada: The new production platform is critical to its capacity to produce new vehicle models in the long term.

General Fusion: The funding has significantly increased the recipient's capacity to conduct experimental research and modelling and allowed them to grow their technical staff by 25%.

Algoma Tubes Inc: The project increased the technological capacity to manufacture new and more profitable products, while employees acquired new skills and expertise to operate the new equipment.

Increased employment and skills of recipients and collaborators

Investments in skilled employment

The document and data review found that the SIF investments increased employment in professional and technical occupations and contributed to higher wages relative to the industry average. The proportion of highly skilled employees involved in SIF funded projects (71%) was also higher than at the enterprise-wide level (55%). This is reflective of the fact that the large majority of new employees hired for SIF projects were also highly skilled (80%). Research, development and commercialization projects (stream 1) invested more in employment relative to capital expenditures, which contributed to these projects having the largest jobs impact relative to project expenditures. Due to the higher levels of investment leveraged under investment attraction and reinvestment projects (stream 3), they had the highest jobs impact relative to the level of SIF funding provided.

EDI employment measures

The majority of recipient agreements (62%) had commitments to have an EDI plan in place, usually within one year of the execution of the agreement. The most common measure included as part of the EDI plan pertained to increasing the gender and diversity representation levels within the organization, with program data indicating that most recipients were on track to complete their plans. While interviews and case studies generally perceived these requirements positively, the survey indicated that only a quarter of recipients believed that the EDI requirements would help to gain a greater gender balance and diversity within their firm. Further, only one-fifth of recipients surveyed indicated that the SIF funded project was led by a woman.

Estimated Employment Impact of SIF Project Investments

Increased employment and skills of recipients and collaborators

Investments in training

The survey found that for around half of the recipients, the SIF made a substantial contribution to employee skills and the skills of project collaborators. The interviews and case studies found that the SIF has enabled the creation of new training opportunities leading to the attraction, expansion and retention in the number of highly skilled employees and an increase in technical knowledge. The SIF contribution agreements require the training or re-training of existing staff, as well as the training of students. There were a variety of training opportunities provided, with training on new manufacturing processes cited frequently.

The data review found that further training is required to meet the program target that 80% of employees of SIF supported firms participate in on-the-job training by March 31, 2022. The majority of recipients reported providing training to employees (23 out of 39 projects). Of those recipients, almost half of the training provided (44%) was specifically in relation to their SIF funded project. The total hours and expenditures for project training represented close to 40% of the enterprise-wide total. The data review also found that about half of the recipients' co-op students were employed on the SIF projects.

EDI training measures

In addition to EDI plans, program data indicates that training, outreach and awareness were common measures included as EDI commitments. Interviews and case studies identified a various training and outreach measures included in contribution agreements, such as STEM outreach to indigenous youth, stem educational initiatives, guidance to indigenous start-ups, scholarships, free apprentice training, and awards to indigenous businesses. However, a majority of survey respondents indicated that there would have been no change in their gender and diversity measures in the absence of the SIF funding, indicating that many of these measures would be in place regardless of the SIF.

Development of innovative products, services, or processes

Development of innovative products, services, or processes

The interviews, survey and data review found that firms have adopted new or innovative technologies and developed many new or innovative products, services or processes as part of their SIF funded projects. These innovations range from those which will help the environment (e.g., decarbonizing liquefied natural gas, reducing emissions from cars, etc.) to those which will improve productivity (e.g., equipment to lower steel scrap rates, new packaging process for food, etc.). The survey and interviews indicated that most of these products, services, or processes would have a disruptive impact on a market. One example of this is a project whereby the new technology will reduce the power consumption of large data centres by 50%. Other examples included a project related to cybersecurity, a modular reactor project, development of zero carbon aluminum, and nuclear fusion.

The data review found that the majority of recipients developed new or significantly improved products, services or processes (22 of 37 recipients), indicating substantial progress against the SIF's initial target of having 12 innovative products, services and processes developed by 2022. Process improvements were the most frequently cited (46%), followed by improvements to products (38%) and services (19%). A variety of improved products, services, and processes were identified, such as software and digital platforms, pilot manufacturing facilities, automotive assembly and parts manufacturing processes, high strength steel manufacturing, human cell production, and internal processes and quality controls.

Case Study Highlights:

Toyota Motor Manufacturing Canada: The recipient developed new products in the form of production management and maintenance IT systems, which are now being licensed to other facilities.

Smart Grid Atlantic Project: Siemens Canada noted they were one of the first in the industry to connect a DER management system to an advanced grid distribution system.

Algoma Tubes Inc: The SIF funding allowed the recipient to diversify their product line and enabled the serving of domestic needs by bringing new capabilities to Canada.

Commercialization of innovative products, services, and processes

Commercialization of innovative products, services or processes

Interviews found that for the most part, commercialization outcomes for most projects have yet to be realized as it is too early in the project cycle, although a few examples were provided of completed projects (e.g., re-tooling of a factory to produce hybrid vehicles, new facility for additive manufacturing, etc.). The survey found that the SIF had supported the advancement of most projects to higher Technology Readiness Levels (TRLs), thereby bringing the projects closer to the commercialization stage. The data review found that less than one-fifth of recipients (7 of 39) reported revenues from new or significantly improved products or services, with average revenues of over $9 million per recipient. A small portion of recipients (6 of 39) reported cost savings from improved processes, with an average cost savings of 8.8% attributable to the SIF projects.

Intellectual property and licensing agreements

Interviews noted that the new or innovative products, services and processes would lead to the generation of Intellectual Property (IP). The data review found that a small percentage of respondents (6 of 39) reported having a new or significantly modified IP strategy related to the project. One-third (13 of 39) reported having sought intellectual property protection as a result of activities undertaken for the project, thereby achieving the SIF's initial target of having 10 projects result in IP filings by 2022. The majority (69%) of these applications were for patents and software. However, only two recipients reported IP licensing agreements, with most (89%) consisting of in-licensing from one project partner.

Patent Applications:

One-third of recipients (13 of 39) reported in the APBRs that they had filed 164 applications with various patent offices. The most frequently cited patent offices included:

• U.S. Patent and Trademark Office: 62 (38%)
• European Patent Office: 31 (19%)
• Canadian Intellectual Property Office: 22 (13%)
• China National Intellectual Property Administration: 20 (12%).
• World Intellectual Property Organization: 15 (9%)

Attraction of anchor firm investments and supply chain development

Increased investments by anchor firms and supply chain impacts

The interviews found that the SIF supported increased investments by anchor firms (i.e., large firms with significant market leadership and extensive supply chains), as many recipients themselves are anchor firms (e.g. CAE, Bell Helicopter, Toyota and Stemcell Technologies, Mastercard and Maple Leaf), with foreign multinationals representing a significant share (44% of SIF projects involved FDI). Similarly, the steel and aluminum projects, which represents one-third of FDI projects, involve many anchor firms, as the companies are often key employers in their respective regions. A few other interviewees indicated that the SIF has been partly responsible for their firm partnering with anchor firms (e.g., Bell Canada, Shopify, Bombardier, etc.). It was also explained that when foreign multinationals invest in a Canadian firm, they often take part ownership in the company or IP. It was observed that when SMEs receive funding from the SIF, it reduces their reliance on such investments, thereby supporting the growth of domestic SMEs into anchor firms.

Interviews found that in attracting investments from anchor firms, other smaller firms in the supply chain benefit. Case studies identified that the projects were expected to have significant impacts on the supply chain, but it was too early in the project lifecycle for results to have been realized. Other interviewees noted that the SIF has facilitated the building out of Canadian-based supply chains and has thereby reduced the dependency on foreign suppliers. It was said that one important by-product of having a Canadian supply chain is that it helps maintain IP within Canada, as there is no outsourcing involved. Another point raised was that although the SIF may not have directly impacted the development of supply chains, it had an impact on some of the tools within the supply chain and made it more efficient. For example, it was noted that the SIF has played an important role in helping to develop linkages with companies in other sectors (e.g., between the chemical, oil and gas, and plastic sectors).

Case Study Highlights:

Toyota Motor Manufacturing Canada: The project enhanced the supply chain for the recipient, as the new hybrid models are higher value vehicles, with more component parts.

Algoma Tubes Inc: The SIF investment allowed the recipient to expand domestic manufacturing capabilities, facilitating the Canadian supply chain for steel products.

Domtar: The Espanola mill is an anchor firm for the region and sources the vast majority of raw materials from regional suppliers. The project is expected to improve the reliability of its demand for these raw materials.

Attraction, retention and growth of business investments

Foreign Direct Investment

FDI is associated with increased exports and R&D in the recipient country. Once one of the largest recipients of FDI, Canada was ranked 15th in the world as a destination for FDI in 2017. Further, mergers and acquisitions ('brownfield FDI') accounted for almost half of Canada's FDI, a relatively higher share compared to other countries. One of the key objectives of the SIF is to leverage increased greenfield investments – FDI that expands an existing facility or creates a new facility.

Interviews and case studies found that SIF funding helps influence the FDI decisions of recipient firms. For projects where the recipient was a foreign multinational, SIF funding significantly impacted their FDI decisions, largely by increasing the return on investment relative to foreign jurisdictions. It was remarked that there is a perception that large corporations benefitting from the SIF are already highly profitable. While this may be true, interviewees explained that SIF funding is about creating employment and bringing strategic investments to Canada. Multinational companies have the flexibility to invest in jurisdictions that offer the most support, with other jurisdictions offering significant financial incentives to attract them. The SIF funding also validates and de-risks the work (in the eyes of others) that a company is doing, which helps them attract FDI.

Interviews found that one challenge in attracting FDI through the SIF is that multinationals often want an early funding commitment from the program, but this is not always possible, as application review and due diligence is required. It was said that to better support multinational FDI decisions, the SIF needed to communicate more with applicants (e.g., updates on application status, a rationale when a project is rejected, and guidance on the types of projects applicable to the SIF).

Case Study Highlights:

Algoma Tubes Inc: The recipient attracted an additional $81 million to further invest and expand on the work undertaken for their SIF funded project.

Smart Grid Atlantic Project: The project enhanced the application development capabilities of Siemens Atlantic Canada team, thereby positioning them more competitively relative to other jurisdictions as a hub for future expansion and investment.

Domtar Inc: The SIF investments helped the Espanola Mill compete for internal investment, as the company has nine facilities in the U.S.

Findings: Design and Delivery

Outreach and engagement

Engagement and outreach with targeted stakeholders

Given the high demand experienced for the program, the SIF did not engage in general outreach to raise awareness. Almost all SIF recipients interviewed indicated that they had heard of the SIF and planned on applying before any outreach had taken place. Targeted outreach was undertaken primarily with respect to SIF sub-groups with specific funding allotments, such as steel and aluminum and forestry, and for the launch of the Collaborations and Networks streams (stream 4-5). Interviews identified various ways this outreach and engagement was undertaken:

• Information sessions, Q&A sessions, webinars, stakeholder teleconferences and industry conferences;
• Collaborating with other parts of ISED, namely the Industry sector, as well as the Clean Growth Hub;
• Collaborating with federal partners such as the NRCan, Invest in Canada, Trade Commissioner, and SDTC;
• For stream 4, collaborating with Health Canada and Agriculture and Agri-Food Canada;
• For stream 5, using Business-Led Networks of Centres of Excellence and Center of Excellence for Commercialization and Research's network, since this stream replaced the existing program suite of business accelerators;
• During meetings between CEOs of companies and senior Departmental officials and the Minister.

ISED officials indicated that the SIF's engagement and outreach efforts, with help from its federal partners, has led to various projects. For the forestry sector, the SIF conducted a webinar and got the Forest Products Association of Canada to lead engagement, resulting in a number of applications from its members. Similarly, the Industry Sector led the engagement with the steel and aluminum sector, which helped solicit applications from this sector. Outreach was also used to attract specific projects and secure strategic FDI investments.

Interviews identified an outreach challenge in cases where there is not enough time to bring stakeholders together for consultations. Interviews also indicated that the SIF could benefit from more outreach in a few key areas, including at the regional level to improve regional representation of SIF recipients and for sectors with less experience applying to programs like the SIF. For example, for the food and beverage sector, it was remarked that there was limited outreach and engagement to fit the needs of this sector, particularly during the program design stage for the Business Innovation and Growth streams (streams 1-3).

Guidance, tools and support

Guidance, tools and support

With respect to program guidance for applicants and recipients, the SIF has developed detailed guidance documents, including a statement of interest guide, a comprehensive full application guide, a term sheet punch list (identifies key elements to be negotiated in the Contribution Agreement), a claims cheat sheet, a recipient claim package, and guidance documents and a web-ex session to support completion of the APBR. One of the initial sources of guidance and information for potential applicants is the SIF website. However, the survey found that only a small percentage of respondents found that the guidance, tools and support available on the SIF website was useful to a great or very great extent.

However, almost all survey respondents (87%) were either very satisfied or satisfied with the other guidance, tools, or support they received from the SIF program. Similarly, most interviews and case studies suggested that program guidance was both helpful and effective during the application submission process, as well as during the claims reporting process. SIF officials were noted as answering questions and providing guidance to companies prior to the submission of applications and/or claims – thereby ensuring that all paperwork was done correctly. A few suggestions were put forth, including providing better clarity upfront on what costs are eligible and ineligible, providing the main contact points for other federal programs, and having better coordination among the different groups within ISED that review SIF claims to improve the timeliness of this process.

Case Study Highlights:

Toyota Motor Manufacturing Canada: The recipient found that the provision of guidance and support was effectively coordinated between the Industry Sector and the SIF and both groups were helpful in the development of the proposal.

General Fusion: The recipient explained that the SIF staff helped work with the applicant to develop the project proposal, and to ensure that there was no overlap between the final phase of their SDTC project and the early stages of the SIF project.

Alignment of requirements and criteria with program objectives

Alignment of program criteria and objectives

There was general agreement among ISED officials that the eligibility requirements, project selection process and review criteria aligned with the program objectives – mainly because they are directly derived from the project objectives. It was noted that the eligibility criteria is broad and that the $10 million funding minimum allows funding to be directed towards large-scale projects – one of the objectives of the program. However, it was also remarked that SIF funding must balance trade-offs between different regions and sectors, such that some projects are stronger on economic benefits while being weaker on innovation and public benefits and vice versa. To help provide input into this balancing, it was indicated that attempts are being made to incorporate input from the Regional Development Agencies into the review committee. It was also noted that the SIF makes significant efforts to reach out to other government departments during the assessment process to elicit their expertise and views when a project is situated within their sector. However, there were a few areas of concern identified by stakeholders (noted below).

• While the SIF is effective in leveraging external expertise from the ISED Industry Sector and other federal departments to support the assessment of projects, a few case studies and interviews identified a few instances where the SIF program officials may not have had sufficient technical or subject matter expertise. This was sometimes attributed to the SIF's funding of a broader scope of industry sectors as compared to the legacy programming, thereby requiring the use of more technical experts external to the SIF program. Case studies and interviews indicated this sometimes occurs and can result in lengthier and more complex delivery processes.
• Case studies and interviews identified challenges with EDI targets in industries that are predominantly male, with some interviewees identifying challenges in meeting their EDI targets. To ensure alignment, the SIF consults with industry experts to help develop realistic targets. To further advance EDI objectives, alternative measures may include EDI targeted outreach, scholarships, training or apprenticeships.

Program flexibility and alignment with needs and capabilities of recipients

Flexibility and Alignment

Interviews and survey found that the project terms and conditions mostly align with the needs and capabilities of recipients, as there is some flexibility built in that takes into account both business requirements and bringing benefits to Canada. Interviews also identified flexibility in the types of investments supported, the industries covered and repayment terms. The survey found the funding terms and conditions to be flexible, with 75% of respondents indicating that they were greatly flexible or flexible to a moderate extent. Two-thirds of respondents indicated that there was a strong alignment between the program requirements of the SIF and the capacity of their firm. However, a few challenges were identified in the interviews and case studies:

• SIF job commitments: It can be difficult for some companies and sectors to meet the program expectations in terms of job commitments as the particular focus on jobs as a metric may, in some instances, not adequately account for the impact of business cycles and automation. For example, job creation requirements need to reflect business flexibility needed during a crisis as well as when the goal of the innovation is to automate a process (particularly for traditional sectors such as manufacturing and agri-food), which leads to a smaller number of, but more sustainable, high-skilled jobs. While there are instances where this flexibility was provided, some interviewees nonetheless indicated that there was a need for additional flexibilities to ensure recipients are not penalized for improving productivity.
• Definition of innovation: The SIF's definition of innovation emphasizes disruptive technology. It was suggested that the definition should be more flexible and be better targeted towards different sectors, as an innovation in one sector may not be considered an innovation in another sector and thereby exclude a potential SIF recipient. For example, interviews found that there was a need for improved alignment with the innovation capabilities in the agri-food sector, as this sector tends to involve incremental and process innovations rather than disruptive technological innovations. Similarly, for a project such as a manufacturing facility, the benefits do not always align well as compared to companies in some other technology sectors.

Intake and selection process and the Steel and Aluminum Stream

Intake and selection process

Business Innovation and Growth streams (streams 1-3) of the SIF use a continuous intake process, where applications are accepted on an on-going basis, using a two-step process (a statement of interest followed by a full application). Interviewees identified the two-step process as a strength of the program. The process allows for the filtering of serious projects from less serious projects. Although the process is getting timelier, many interviewees noted that it can still take months to receive feedback on proposals. Interviewees also identified some challenges with the continuous intake process. Because each funding stream has a limited budget, it is possible that a better project emerges but is unable to secure funding. This would be less likely with a closed process, as all potential applicants would likely apply before the submission deadline.

Steel and Aluminum (S&A) Sector – targeted intake

In 2018, the S&A funding envelope was established to address an urgent need that emerged when the U.S. imposed tariffs. The stream was a time limited, sector targeted funding vehicle, aimed at short-term projects that would be completed by March 31st, 2020. Interviewees indicated that this funding envelope was implemented very quickly and that key stakeholders were well informed of this stream. The survey found that S&A recipients perceived the SIF to be flexible in meeting their needs, comparable with recipients from the other SIF streams, however, the survey also indicated that the requirements of the SIF were not well aligned with the capacity of recipients to meet them. In particular, interviewees highlighted the following:

• The two year funding profile was not realistic. Numerous amendments were required for projects, because the initial funding profile did not align with the recipients' internal approval processes and their time requirements to implement large and complex projects. As a result, funding had to be re-profiled, which was a time consuming process.
• There was room for improvement in coordinating with the RDAs. RDA funding was announced after the SIF funding, causing confusion among some stakeholders, which led to some firms applying to both programs.
• Some interviewees suggested that the SIF was not the right tool for S&A sector funding, as they did not perceive this funding to be aligned with the innovation objectives of the SIF. It was noted that incorporating one-off streams slows down the rest of the program, as program resources are redirected towards these emerging priorities.

Length and complexity of application and approval processes

Application and approval processes

One of the reasons for the creation of the SIF was to reduce administrative complexity by simplifying and streamlining processes, requirements, and terms and conditions. The interviews, survey, and case studies found that the application and approval process was lengthy and complex. In particular, the survey found that 86% and 75% of recipients perceived the application process to be lengthy and complex, respectively.

Program data corroborates this, with an average turnaround for rejections/referrals of Statements of Interest (SOI) peaking at 34 weeks in Q2 of 2018-19. While the SIF has some service standards for the initial triaging of SOIs, it only recently established a service standard, on a pilot basis, for review turnaround (16 weeks). Interviews identified a lack of clarity in regards to the expected turnaround time for their applications. Interviews also found that for projects requiring more than $50 million, additional financial authorization is required, which further extends the approval process, contributes to a hesitancy to fund large projects, and increases the risk of a project not going forward when investment decisions are time-sensitive.

Interviewees noted that this lengthy process hampers business decisions and planning, which are typically made in a few months (particularly for SMEs), and that the SIF needs to move at a similar pace as industry to be effective. It was said that quicker turnaround could reduce uncertainty for applicants' business investment planning decisions. Interviews also identified that more clarity and communication was needed with the application status, expected timelines for decisions, and the reason for rejection. However, interviews and program data indicate turnaround and communication with applicants has improved, in part due to fewer applicants stemming from the change in SIF eligibility.

Length and complexity of project reporting, claims, and amendment processes

Project reporting, claims and amendment processes

The project claims and reporting processes were perceived to be relatively less lengthy, with half of survey respondents perceiving them as very lengthy or moderately lengthy, and a minority (26%) perceiving the funds disbursement to be lengthy. Respondents' perceptions of the complexity were similar, with projects claims and reporting less complex, and the disbursement of funds as the least complex. Interviews and case studies indicated that claims and reporting requirements were detailed and required a substantial amount of work to complete. However, it was noted the guidance and support provided by the SIF was very helpful in meeting the requirements. It was also found that the reporting and claims processes were well organized and clear, except for the Annual Performance Benefits Report which was often viewed as complex and difficult to complete; nonetheless, it is important for the program to maintain these data collection efforts into the future to assess performance of the program; improving data strategies was an area for improvement identified in previous ISED evaluations of the funding programs that were subsequently subsumed into SIF.

Interviews found that the project amendment process is lengthy. It was found that a lack of internal guidance for SIF staff on the amendment process contributes to delays, as multiple requests are required to gather information from the recipients. However, it was noted by interviews that guidance was in the process of being developed. It was also found that a more streamlined or automated process could be used for minor, non-substantive amendments. It was said that these amendments are sometimes delayed or postponed as they are considered to be lower risk and lower priority.

Findings: Efficiency

Efficiency

An operational efficiency analysis was conducted to measure administrative costs relative to program expenditures. The evaluation found that program operating costs as a percentage of program expenditures ranged from 1.4% to 1.7% for the 2017-18 to 2019-20 period. The SIF also uses resources from outside of the program, such as subject matter experts from the ISED Industry Sector and other departments. Although total grants and contributions and operating costs were lowest in 2017-18, the number of SOI applications submitted for SIF funding was high (762) compared to 2018-19 (197) and 2019-20 (124) – implying a disproportionate amount of work in assessing applications.

Notes:

¹ Section 15 ISED Public Accounts of Canada (2017-18 and 2018-19) and Internal Financial Management System (2019-20).

² Internal reporting – combining resources allocated to both SIF and Legacy.

³ Includes expenditures related to ASIP, SADI, AIF, TDP and C-Series before the launch of the SIF program on July 5, 2017.

In comparison to the legacy programs, the SIF executed more agreements in terms of quantity and value, using fewer resources than the legacy programs. In comparison to its direct predecessors (AIF, ASIP, TDP, & SADI), the SIF executed 50% more agreements per year, with a total value that was more than double that of the legacy programs. These outputs were accomplished by the SIF using only 25% more full-time equivalents (FTEs) than the legacy programs. The number of amendments has been comparable to the legacy programs, while the number of claims per year has increased under the SIF.

ISED officials interviewed noted that the SIF has achieved more agreements with fewer resources (FTEs), while also administering the legacy programs. It was also remarked that the SIF has more consistency due to a standard approach where applications and claims processes are centralized, as are client services. Further, lessons learned in program delivery have the ability to be transferred much more quickly within SIF streams than the legacy programming. One government partner also noted that the SIF is more adaptable, allowing it to pivot quickly to new priorities (e.g. steel and aluminum tariffs).

There were only a few recipients familiar with the legacy programs and views were mixed. The few survey responses noticed a moderate improvement in design and delivery. Interviews and case studies noted that the SIF allowed for an expanded scope and scale, but it was not as flexible for aerospace sector needs and it had a higher administrative burden, as more detailed information is required for claims.

Average Annual Statistics:

Legacy programs

• $262 million in average annual Contribution Agreement commitments
• 12 projects funded per year
• 80 ISED FTEs per year

SIF (2017-18 to 2019-20)

• $608 million in average annual Contribution Agreement commitments
• 18 projects funded per year
• 100 ISED FTEs per year

Conclusions

Relevance

There is a continued need to support innovation and growth among Canadian industry through the provision of direct funding of R&D, commercialization, and capital investments. The SIF contributed to addressing the needs of industry sectors and made a contribution towards addressing the needs of underrepresented groups. While the SIF generated substantial interest among targeted groups, the levels of support provided by the program were more heavily concentrated in some sectors and regions, but generally corresponded to business expenditures on R&D in the regions. However, the needs were not entirely met for some sectors and smaller scale projects.

Performance

The SIF influenced the scope, scale and timing of business investment decisions, effectively leveraged private investment, attracted foreign direct investment and helped support the growth of domestic SMEs. Increased investments in R&D and industrial facility expansion and improvement were supported, while the commercialization of most of those R&D investments are expected to occur at a later stage. The SIF increased the technological capacity of recipients and collaborators via increased capital investments and increased investments in highly skilled personnel, training, and employee skills development. The SIF also contributed to increased collaboration amongst recipients, particularly with universities, as some of these collaborations may not have otherwise occurred. While most projects are still ongoing, recipients reported that they advanced the development of their technologies, and in some case new or improved processes, products, and services were developed and intellectual property protection was sought.

Design and Delivery

Public announcements effectively elicited applications from industry, with targeted outreach and engagement used to solicit applications for sector targeted funding and complex project streams. However, some sectors and regions may benefit from additional outreach and engagement. Most recipients were satisfied with the guidance, tools and support they received from the SIF program. While the project terms and conditions generally align with the needs and capabilities of most recipients, some sectors experienced less flexibility due to the SIF's emphasis on job creation and disruptive innovation. The project intake and selection was effective in filtering projects using a two-step process, but some challenges in prioritizing the selection of projects were identified with the continuous intake process. As a targeted intake process, the steel and aluminum stream met industry needs but a compressed funding profile and timelines led to implementation challenges for recipients and program officials. The application and approval processes are perceived as lengthy and complex, although this has improved over time, in part due to changes in eligibility requirements. While project claims and reporting processes were viewed as relatively less burdensome, the project amendment process was seen as lengthy and complex, in part due to an absence of internal guidance documents.

Efficiency

The program is being efficiently from an administrative cost standpoint, as it has executed 50% more agreements per year than the legacy programs using only 25% more resources to do so.

The findings from the evaluation produced three recommendations:

1. ISED Innovation Canada should consult with key federal partners to help inform efforts to provide a full continuum of business innovation support for firms with viable smaller-scale projects.
2. ISED Innovation Canada should explore approaches to tailor project assessment criteria to better meet the needs of targeted sectors and regions, such as assessment criteria that take sector characteristics into consideration (e.g. innovation and job creation capabilities), conduct targeted outreach and engagement, and provide sector-specific guidance.
3. ISED Innovation Canada should examine opportunities to streamline program application, review and amendment processes, including improved triaging of the statement of interest applications and earlier communication with applicants, as well as the incorporation of additional service standards.

Annex A: SIF logic model

Annex A: PI Logic Model

Annex A depicts a logic model for Strategic Innovation Fund (SIF). A logic model shows how program activities are expected to produce outputs and in turn how these outputs are expected to lead to different levels of results or outcomes.

The top section of the logic model describes the SIF's activities and outputs. The activities of the program are listed in horizontal boxes and indicates that the SIF engages in three key activities:

• Program communication and outreach
• Administration of project selection process
• Program Monitoring

These three activities lead to four outputs that are listed horizontally and below of the list of activities:

• Communication and outreach products
• Approved business innovation and investment projects
• Approved technology development and demonstration projects
• Project amendments, progress and performance reports

An arrow leading to the below section of the logic model indicates that these activities and outputs lead to distinct immediate, intermediate, and long-term outcomes.

The immediate outcomes of the program are:

• Interest from target groups
• Increased investments in R&D, commercialization and industrial or technological facilities or projects in Canada
• Increased investments in industrial research and large-scale technology demonstration projects
• Increased collaboration between universities, colleges, research institutes, not-for-profit and the private sector

The intermediate outcomes of the program are:

• Development of innovative products, services or processes
• Retention or growth of Canada's highly-skilled workforce
• Retention or attraction of R&D or production mandates to Canada
• Expansion of pre-existing industrial or technological facilities in Canada
• Increase in technology demonstration activities in Canada

The long-term outcomes of the program are:

• Commercialization of innovative products, services or processes
• Business expansion and growth
• Development of high quality science and technology related jobs in Canada
• Increase in later stage technology development

These outcomes contribute to four distinct results for the SIF:

• Increase technological capacity for recipients
• Establishment of anchor firms in Canada
• Increase in spillover and/or supply chain development
• Longer term economic and social benefits to Canadians

Annex D: Legacy Programs

In Budget 2017, the Government committed to furthering the Inclusive Innovation Agenda introduced in Budget 2016 through the Innovation and Skills Plan, to propel Canada to the forefront of the economy of the future and achieve its vision for a more innovative country. This approach aimed to reduce program complexity and align policies and programs from across federal departments under a unified industrial strategy. In line with this approach, Budget 2017 announced the creation of the Strategic Innovation Fund (SIF). This new program consolidated four business innovation programs:

Strategic Aerospace and Defence Initiative (SADI)

Launched in 2007, SADI provided repayable contributions to support research and development (R&D) projects in the aerospace, space, defence and security (A&D) sectors. SADI was available to firms of all sizes to support product, service or process innovation.

Automotive Innovation Fund (AIF)

Announced in Budget 2008, the fund was established to provide repayable contributions to support strategic, large-scale research and development projects in the automotive sector in developing innovative, greener and more fuel-efficient vehicles.

Automotive Supplier Innovation Program (ASIP)

Announced in 2015, the ASIP was created to help Canadian automotive suppliers gain a competitive edge through new innovative products and processes. The program would help research and development projects to become commercially viable by supporting product development and technology demonstration on a cost-shared basis with participating firms.

Technology Demonstration Program (TDP)

Launched in 2013, the TDP provided non-repayable contributions to support one or more large-scale research and development (R&D) projects per year.

In Budget 2018, the federal government announced the consolidation of the Centres of Excellence for Commercialization and Research, and the Business-led Networks of Centres of Excellence programs into the SIF. These programs funded collaborative research networks and centres with the aim of bringing together the research and business communities to facilitate the commercialization of research.

Annex E: Summary Statistics for SIF Projects