COMMISSION STAFF WORKING DOCUMENT IN-DEPTH INTERIM EVALUATION of HORIZON 2020 - Hoofdinhoud

Council of the European Union

Brussels, 30 May 2017 (OR. en)

9791/17 ADD 3

RECH 212 COMPET 455 IND 143 MI 459 EDUC 273 TELECOM 149 ENER 256 ENV 552 REGIO 66 AGRI 290 TRANS 228 SAN 225

COVER NOTE

From: Secretary-General of the European Commission, signed by Mr Jordi AYET PUIGARNAU, Director

date of receipt: 29 May 2017

To: Mr Jeppe TRANHOLM-MIKKELSEN, Secretary-General of the Council of the European Union

No. Cion doc.: SWD(2017) 220 final - PART 4/4

Subject: COMMISSION STAFF WORKING DOCUMENT IN-DEPTH INTERIM

EVALUATION of HORIZON 2020

Delegations will find attached document SWD(2017) 220 final - PART 4/4.

Encl.: SWD(2017) 220 final - PART 4/4

EUROPEAN COMMISSION

Brussels, 29.5.2017 SWD(2017) 220 final

PART 4/4

COMMISSION STAFF WORKING DOCUMENT

IN-DEPTH INTERIM EVALUATION

of

HORIZON 2020

{SWD(2017) 221 final} {SWD(2017) 222 final} 9. HOW COHERENT HAS HORIZON 2020 BEEN SO FAR?

This question involves looking at the extent to which Horizon 2020 actions work together, internally and with other EU interventions/policies and to identify whether there are major complementarities, gaps or overlaps between the initiatives.

Expectations from Horizon 2020 as regards its internal and external coherence

Based on the Horizon 2020 impact assessment - and compared to FP7 - knowledge triangle and broader horizontal policy coordination are expected to be enhanced under Horizon 2020 through the integration of research, innovation, and researcher training and skills development into a single framework, and the explicit definition of links with other policies. In addition, the reduction of the number of programme pillars and funding schemes is expected to facilitate the gearing of all programme components towards the achievement of the objectives.

Summary box: Key findings on the internal and external coherence of Horizon 2020

 The integration of research and innovation, the three pillar structure, the challenge-based approach, and the use of focus areas contribute to the internal coherence of Horizon 2020 compared to FP7.

 Outside the 'Excellent science' pillar, Horizon 2020 is increasingly focused on research and innovation at higher Technology Readiness Levels. It should be ensured that this does not come at the expense of lower Technology Readiness Levels collaborative research, which is regarded as one key source of future breakthrough innovations in line with societal needs.

 The large number of European R&I funding instruments is difficult to understand for potential applicants and may lead to overlaps.

 Compared to FP7, efforts have already been made to increase the synergies between Horizon 2020 and other programmes, notably ESIF but these can be further strengthened.

 Given the different rules and implementation structures, promoting synergies at project level (in term of

combining different financing sources for the same project) is not always realistic. The difference in state aid rules further leads to legal uncertainty for potential beneficiaries.

 Horizon 2020 specifically aims to establish synergies with national programmes. Public-public partnerships

are creating long lasting collaborations between funding agencies and capacity building benefits but do not seem to really influence the alignment of national strategies and policies.

9.1. To what extent is Horizon 2020 coherent internally?

The sources of evidence mobilised for this interim evaluation point that the integration of research and innovation into a single programme, the structuring around three pillars and a set of challenges instead of thematic domains improved the overall coherence of the programme compared to FP7.

Figure 1 provides a quick overview of the different approaches pursued under each pillar and the main target groups.

Figure 1 Main approaches and target groups of Horizon 2020 pillars based on programming documentation

Horizon 2020 pillar Main approach Main target group

Excellent Science Bottom-up (ERC, Marie Skłodowska-Curie Actions, FET-Open) Scientific com

Excellence-driven 1 Top-down (FET Proactive, Research Infrastructures) munity

Either single beneficiary or collaborative projects

Industrial Leadership Primarily bottom-up (SME Instrument, Access to Risk Finance) Businesses and Technology-driven 2 Top-down (Leadership in Industrial and Enabling Technologies) industry

Either single beneficiary or collaborative projects

Societal Challenges Top-down (priority-based) Scientific com

Challenge-driven 3 Collaborative projects (+SME Instrument) munity, Businesses

• & 

  Society Source: European Commission

The internal coherence at programming level is also regarded as reinforced by the use of focus areas - even if their multiplication also results in some confusion - and the fact that

many projects are expected to have cross-cutting impacts. 4 Focus areas were introduced

where priorities identified from strategic programming cut across the parts of Horizon 2020 e.g. blue growth, circular economy, Internet of Things, Smart and Sustainable cities, Digital Security. They intend to concentrate resources and efforts on key areas of high policy and political relevance and societal concern, alongside increasing industrial competitiveness and providing better solutions and achieving higher impacts through stronger integration across different Horizon 2020 parts, in particular between the Societal Challenges and Leading Enabling and Industrial Technologies (LEITs) thus adding coherence to the programme and avoiding silos. These interdisciplinary solutions are expected to cut across multiple specific objectives, ensuring both coherence and increased cost-efficiency. So far, 21 focus areas were identified (12 in the 2014-2015 Work Programme and 9 in 2016-2017 Work Programme). The choice of focus areas was made following the EU's key priorities and setting these against the R&I activities which could meet these needs.

1 Based o, Horizon 2020 Specific Programme

2 Recital 9, Horizon 2020 Specific Programme

3 Annex I, Part III, Societal Challenges, Horizon 2020 Regulation

4 For example, the survey run among project coordinators in SC4 found strong evidence (85% agreement overall) of projects

simultaneously tackling several challenges and give rise to new competitive businesses and industries (e.g. contribution to decreased CO2 emissions, improved marine environment, while creating increased competitiveness of European paint industry). See more in-depth analysis in Section 8.3.1

Figure 2 Contribution of Horizon 2020 programme parts to focus areas in WP 2014-

2015 and 2016-2017

Source: European Commission, DG RTD, based on Horizon 2020 Work Programmes

Stakeholder position papers: The current pillar structure improves the clarity

of the programme but linkages across the pillars should be enhanced.

In their position papers, almost half of the stakeholders commented on the current programme structure. Half of those commenting have a positive view of the three pillar structure. They see it as a pragmatic and easy way to clarify the goals of different programme priorities.

However, others pinpointed that the coherence and linkages between activities and projects under the three pillars should be strengthened. In particular, they mentioned a need for better links between the excellent research

supported under Pillar 1 and topics in Pillars 2 and 3. To enhance such linkages one representative of academia, for instance, suggested to extend the principle of ERC proof of concept grants across the entire programme. As a further example, one business stakeholder noted that the current "hand off" between Pillar 1 and Pillar 2

hampers in particular the FET projects (more fundamental research) to be advanced to sufficient maturity on the TRL scale to entertain a go-to-market solution by industry which is currently incubated through the LEIT ICT calls. It was suggested that cross-pillar innovation should be enabled and ensured by the Work Programme by for example give a preferential score for proposals that build on previous project results.

A cross-analysis of the thematic assessments performed for this interim evaluation clearly

points out to the overall complexity of the EU research and innovation support landscape.

Looking at Horizon 2020 only the three pillars of Horizon 2020 consist of around 20

programme parts supporting different areas in different ways (e.g. grants, prizes, financial instruments)

in addition to the activities of the EIT (with the KICs), the JRC direct research actions

and Euratom. Cooperation networks have been established under 66 active ERA-

NET/ERA-NET Plus/ERA-NET Cofund, ten Joint Programming Initiatives (JPI), five Article 185 Initiatives, seven Joint Technology Initiatives (JTIs), ten Contractual Public Private Partnerships

(cPPPs). Outside Horizon 2020 but providing inputs to the strategic programming of

Horizon 2020 there are also five European Innovation Partnerships (EIP) 5 and 42 European Technology Platforms (ETPs) 6 . In addition, the COSME programme is supporting SMEs, the

COST actions support international research actions, and EU programmes such as LIFE+ or

the European Structural and Investments Funds also provide support for R&I although with

different objectives. This gives potential applicants an array of funding opportunities and/or

networking platforms to navigate through, when looking for R&I support at EU level. Stakeholder

consultation respondents also suggest that the funding architecture is too complex and

may hinder organisations from identifying the calls and instruments that would best fit their

needs and create risk of duplication.

To ensure coherence between the different specific objectives and programme parts and mitigate

the risk of overlaps, different internal coordination mechanisms across Horizon 2020

were put in place at the programming and implementation level, such as: regular meetings of

services interested in a given area; inter-service groups and consultations on the Work Programmes;

ex-post consultations on the list of projects retained for funding, informal contacts

at project officer level; joint events, joint publications, joint kick-off meetings, etc. Examples

of the search for coherence across Horizon 2020 parts are provided in the Box below. Some

thematic assessments however report that more systematic tools, channels and processes to

ensure access to internal (e.g on R&I activities/results supported by the EU outside of a given

area) and external information (e.g. on R&I spending and priorities at national and

regional level) are lacking.

Box: Examples of coherence between Horizon 2020 programme parts

In SEWP, since the funding does not cover the cost of R&I, a survey run among beneficiaries showed that 88%

of them have already received funding from other Horizon 2020 programme parts such as ERC, MSCA, Research Infrastructures to cover their R&I activities as such.

The newly introduced SME Instrument is seen by stakeholders as complementary to other interventions

from Horizon 2020, in particular the FTI pilot (also new) and collaborative projects, providing a welcomed addition

to the Framework Programme toolbox. It supports the efforts towards reaching larger market uptake of innovations from a different angle and in a different manner compared to the other Horizon 2020 instruments.

KIC InnoEnergy provides support to commercially mature concepts which have been developed under the EU

R&I Framework Programmes. However, activities facilitating the identification of promising concepts and the

bridging from Horizon 2020 support to KIC InnoEnergy support could be further developed.

The age profile of the MSCA fellows is complementary to ERC grantees as they tend to be younger and around 40% of MSCA fellows are doctoral candidates. Furthermore, there is evidence that former MSCA fellows tend

to be more successful when applying for ERC grants. An analysis of ERC applicants under Horizon 2020 who were MSCA fellows in FP7 estimates their average success rate at 16%, compared to 12% among all applicants

to the same calls. 7

5 European Innovation Partnerships (EIPs) act across the whole R&I chain, bringing together all relevant actors at EU, national

and regional levels. They were established in order to: (i) step up R&D efforts; (ii) coordinate investments in demonstration

and pilots; (iii) anticipate and fasttrack any necessary regulation and standards; and (iv) mobilise ‘demand’ in particular

through better coordinated public procurement to ensure that any breakthroughs are quickly brought to market.

6 European Technology Platforms (ETPs) are industry-led stakeholder fora recognised by the European Commission as key

actors in driving innovation, knowledge transfer and European competitiveness. ETPs develop R&I agendas and roadmaps

for action at EU and national level to be supported by both private and public funding. They mobilise stakeholders to deliver on agreed priorities and share information across the EU.

7 For this analysis the study team reviewed ERC applicant data from the following calls for proposals:in 2014: ADG, CoG,

PoC, STG; in 2015: AdG, CoG, PoC, STG; in 2016: ADG, COG, PoC, STG; in 2017: STG. Out of 22,784 eligible applicants Figure 3 Number of researchers supported under ERC and MSCA by age

Source: European Commission, DG RTD, based on CORDA data.

Horizon 2020 projects are implemented via different actions (policy-mix) that are specified in

the Work Programmes depending on the objectives pursued (see Section 5.1 for an overview).

Looking at the types of actions approximately 75% of the funding goes to instruments facilitating

collaborative research and innovation 8 bringing organisations across countries

together. This is slightly higher than under FP7 (72% (Cooperation + Capacities)). This represents

39% of the projects under Horizon 2020 whereas this was 40% of the projects under

FP7 (the small difference is related to the introduction of the SME instrument). A quarter of

the funding is allocated to single beneficiaries to support excellent science (ERC) or R&I projects

of SMEs (SME Instrument).

Figure 4 Type of actions per programme part by budget (left) and number of grants

(right)

Source: Corda, calls until end 2016, Signed Grants cut-off date by 1/1/2017. European Joint Programme Cofund

(COFUND-EJP), Coordination and Support Action (CSA), European Research Council (ERC), Innovation Action

(IA), Marie Skłodowska-Curie Actions (MSCA), Pre-Commercial Procurement (PCP), Public Procurement

of Innovative Solutions (PPI), Research and Innovation Actions (RIA), Specific Grant Agreement (SGA), SME

instrument phase 1 and 2 (SME-1 and SME-2).

overall, 1,591 (around 7%) were MSCA fellows in FP7. For a previous analysis with similar results, see Economisti Associati, Marie Curie researchers and their long-term career development: a comparative study, Final Report, 2014

8 Research and Innovation Actions, Innovation Actions, MSCA Innovative Training Networks (ITN) and RISE, and Coordination

and Support Actions.

Stakeholder position papers: The programme needs to ensure a coherent and simplified policy mix. Several instruments under the Horizon 2020 work particularly well such as the ERC and MSCA grants. Some new instruments could be further improved.

In their position papers, a few stakeholders expressed their concerns with the complexity of the Framework Programme. They believe that the policy mix of the overall programme should be simplified: the number of instruments should be limited, their intervention logic clearly defined and complementary/synergies with other instruments well stated.

Public authorities that commented on the instruments in majority noted that the collaborative projects and

grants were preferred over other types of projects and loans. Some of them have a positive view specifically

related to the instruments bringing together states and regions such as the P2Ps, cofound schemes and ERA-

NETs, others the SME instrument, INNOSUP and MSCA. Some stakeholders from academia and research

organisations also depict a very positive view on the current set of instruments fostering excellent science in

particular the ERC and MSCA grants. Furthermore some representatives of the business community specifically

commented on the Joint technology initiatives (JTIs), Joint Undertaking (Jus) and the (contractual) Private

Public Partnerships (c-PPPs). They noted the Horizon 2020 provides a ring fenced budget to PPPs, JTIs and

other Industry Initiatives which is in particular beneficial for the industries which are represented by or are

member of such initiatives. In addition, few SME and business representatives commented and welcomed the

inclusion of innovation activities in Horizon 2020. Finally, a small number of stakeholders discussed the Seal

of Excellence (SoE) instrument. A few stakeholders praise the initiative, whereas other pinpointed to the need

to review its effectiveness.

As regards the current balance of the support provided to more science-driven or innovation-driven

projects, 22% of Horizon 2020 budget goes to the Industrial Leadership pillar so

far, 36% to Societal Challenges (which is both research and innovation driven depending on

the projects) and 37% to Excellent Science (which also include Proof of Concept projects).

Looking at the types of actions 21% of the budget goes directly to innovation support through

the SME instrument and the Innovation Actions. Under FP7 no dedicated instrument focussed

specifically on innovation hence the comparison is not straightforward. However the Entrepreneurship

and Innovation Programme (EIP) had a budget of EUR 2166 million (approximately

4% of the FP7 budget but as a separate programme), whereas more than EUR 4.2 billion

have already been spent on the SME instrument and the Innovation Actions only under

Horizon 2020, after three years of implementation. This does not include the financial instruments

that gained in importance under Horizon 2020 compared to the Risk Sharing Finance

Facility in FP7.

Due to lack of centralised monitoring data, an overall picture of the current balance between

Horizon 2020 grants and financial instrument type of support (e.g loans, equity, guarantees)

proves difficult to construct. However, based on data provided in the thematic assessments on

Access to Risk Finance, SC1 and SC3 it is estimated that Horizon 2020 currently provides at

least EUR 1 in financial instruments for every EUR 12 in grants. 9

Overall, grants, loans and equity investments are complementary forms of finance for firms

and other entities undertaking innovation 10 . Looking at the costs of financing subsequent in

9 A discussion on the preferred types of support by stakeholders is provided under section 6.3.2 ‘Programme attractiveness

and take-up'.

10 Investment in innovation by a firm typically covers R&D, capital equipment, design and marketing, and training. The most

important item of expenditure in most sectors is R&D, accounting for over half the spend on innovation. R&D investments

have three key characteristics that make them different from other investments: most of the R&D expenditure goes towards paying researchers; returns on the R&D investment are highly uncertain; and the capital created from such investment is

largely intangible. From the academic literature grants are an essential form of funding when a researcher, research group,

public research organisation or very early start-up is at the earliest, most risky and most uncertain phase of the innovation

novation phases from loans or equity, although there is no formula for calculating the appropriate

debt / equity mix over time needed by an R&I-intensive firm or project, there are some

typical profiles:

 Equity: a company looking for equity investment is usually at the start-up early stage

or at a point where accelerated growth is in the offing. All available cash is needed for

developing and expanding the firm's means of production and working capital, rather

than servicing debt. Such firms have yet to establish the stable pattern of cash-flow required

by banks and other lenders, given that they are often breaking new ground.

 Debt is commonly used to fund an R&I project or initiative with a clear business plan

or plan of execution, and a clear timetable for implementation. Sufficient cash-flow is

needed to repay the loan, and collateral may be required.

Figure 5 Estimated balance between loans, guarantees, grants and other types of support

 between 2014 and 2017 in terms of budget implementation (left) and number of

beneficiaries (right)

Source: EC DG RTD analysis based on Access to Risk Finance thematic assessment state of play as of

31/12/2016 with regards to the EU contribution and as of 30/06/2016 with regard to operations, SC1- Health and SC3- Energy thematic assessment and CORDA, cut-off date by 1/1/2017.

Regarding financial instruments, feedback from the survey of financial intermediaries run in

the framework of the interim evaluation of Horizon 2020 financial instruments suggest a clear

majority consider that the main financial instruments complement each other and meet the

needs of businesses at different stages of the R&I funding cycle ‘to some extent’ (79.2% of respondents), and 20.8% to a ‘great extent’. No respondent said that they do not complement

each other at all. Also 45.8% of surveyed entities had no opinion. It should be noted that the

survey is still underway.

From an internal coherence perspective, the InnovFin programming architecture is seen

as generally consistent with the broader EU policy aim of ensuring that firms can access

either debt (guarantees or loan products) or equity through financial intermediaries, irrespective

of their stage in the development lifecycle. The four debt-based guarantee and

loans instruments appear to be internally consistent and coherent, since investment size / instrument

have all been defined in a distinctive way to avoid overlaps. Bringing in the EIB

loan scheme for large R&I projects has helped in this regard. Through the SMEG and Midcap

Guarantee schemes respectively, there is a funding continuum between EUR 25,000 and EUR

process: R&D. Typically, no lender or equity investor can tolerate the risk, nor can offer loans or investments on reasonable

terms.

50m which covers a large range of guarantee needs to firms of all sizes. Although microcredits of less than EUR 25,000 are not available through InnovFin, start-ups and microenterprises can take out a guarantee through COSME of less than EUR 25,000 which means that between InnovFin and COSME, all stages of the SME financing lifecycle are addressed.

Compared with the predecessor EIP programme in the 2007-13 period, the InnovFin thematic assessment concludes that there is arguably greater coherence between the design of the programming architecture and evolving EU policy in respect of access to finance. At least for the debt instruments, firms are supported along the “funding escalator” i.e. from SMEs through to mid-caps and large firms. The new ‘funding escalator’ concept is consistent

with the Communication for an Action Plan for the Capital Markets Union (2015) 11 ,

and describes a situation in which EU FI programmes (ideally mirrored in the financial system more widely) are designed to meet the financing needs of all businesses from the smallest micro-firm to the largest listed companies at different stages in their development.

Under InnovFin Debt, banks and other lenders are incentivised to provide loans on reasonable terms to SMEs by the provision of loan guarantees, while the EIB can provide attractive loans to midcaps, large firms, research infrastructures and other bodies through risk-sharing with the EU budget (via a portion of the Horizon 202o 'Access to Risk Finance' budget acting as a first-loss piece). Venture capital, business angel and technology transfer funds, together with funds-of-funds, are encouraged to invest in innovative firms, and to provide follow-on and scale-up funding, through cornerstone investments by the EIF under the InnovFin Equity scheme.

Figure 6 InnovFin debt-based Financial Instruments (guarantees/ counter-guarantees, direct loans)

InnovFin Financial Type of Financial Investment size per beneinstrument

schemes Instrument ficiary Investment duration

SME Guarantee Guarantee EUR 25,000-EUR 7.5m 1-10 years

MidCap Guarantee Guarantee EUR 7.5-50m (firms with Maturity from 2-10 years, with a 500-3000 employees) fixed repayment schedule

MidCap Growth

Finance' Loans EUR 7.5m to EUR25m Up to 10 years

Large Projects Loans EUR 25m-300m Up to 10 years

Source: European Commission

The two pie charts below show the Technological Readiness Levels (TRL) 12 of approved projects

under InnovFin– a key aspect of Horizon 2020 financing being to finance projects closer to the market. These illustrate a relatively equal share between TRL 1-3, TRL 4-6, and TRL 7-8.

11 The concept of Europe’s ‘funding escalator’ was mentioned in the Action Plan on Building a Capital Markets Union,

COM(2015)468 final i.

12 Technology Readiness Levels are indicators of the maturity level of particular technologies. This measurement system

provides a common understanding of technology status and addresses the entire innovation chain: TRL 1 – basic principles observed; TRL 2 – technology concept formulated; TRL 3 – experimental proof of concept; TRL 4 – technology validated in lab; TRL 5 – technology validated in relevant environment; TRL 6 – technology demonstrated in relevant environment; TRL

system proven in operational environment

Figure 7 – Technological readiness levels of InnovFin EU portfolio (amount and number)

Source: Annual Operational Report, 2017

An analysis of the TRL supported across thematic areas shows that whereas the Excellent

Science pillar focuses on more fundamental research and, with the exception of e

Infrastructures, does not move beyond the stage of an experimental proof of concept, the rest

of the programme is rather concentrated on higher TRLs, the majority of which are targeting

product demonstration in both the Industrial Leadership and the Societal Challenges pillar.

This is notably a result of Horizon 2020 being a combination of research and innovation: what

used to be FP7 plus the Competitiveness and Innovation Programme (CIP). FET plays a special

role here, building new communities and innovation eco-systems and pushing new technologies

up the first steps of the TRL scale towards innovation and impact.

Multiple types of stakeholders (interviewed or surveyed for the thematic assessments or having

replied to the Stakeholder Consultation) regret that Societal Challenges and LEIT do

not invest more in lower TRL collaborative research, which is regarded as one key

source of future breakthrough innovations, albeit longer-termed, in line with societal

needs. The European Economic and Social Committee (EESC) in its opinion on the interim

evaluation of Horizon 2020 also express concerns that collaborative research in the lower

TRL 1-5 lost ground to higher TRLs under Societal Challenges, driving many universities and

research organisations away from research on societal challenges with the effect that interaction

between industry and academia has been reduced rather than strengthened. 13

Stakeholder position papers on the balance between research and innovation

and TRLs: "the importance of basic, collaborative and frontier

research should not diminish".

In their position papers, almost half of the stakeholders commented on the balance between research and innovation.

The majority of those who commented stated that the programme needs to ensure a better balance. By stakeholder group, the majority of stakeholders from academia, research organisations and public authorities, pinpointed that currently Horizon 2020 seems to be moving away from funding basic, collaborative and frontier research. They believe there is a need to close the gap in funding lower TRL levels to create ground breaking

technological foundation for innovation. Only business representatives are positive about the shift towards innovation that took place under Horizon 2020. But still, a few pointed out the current lack of TRL 3-5 projects.

Figure 8 presents an overview of the use of TRL across different Horizon 2020 objectives

(alongside their budget) based on the assessments of each programme part performed for this

interim evaluation 14 . The information presented is partial and based on diverse methodologies

13 http://www.eesc.europa.eu/?i=portal.en.int-opinions.39284

14 See Annex 2 for the assessments of each Horizon 2020 programme part.

such as surveys among project coordinators, projects’ mapping by expert groups, external

studies, estimates by policy and projects’ officers, as well as internal project’s reviews 15 .

Within each programme part the focus is put more or less on specific TRL – for example, FET-Open is primarily TRL 2, FET-Proactive can be TRL 2-3 while the FET Flagships can be up to TRL 5. From this overview Horizon 2020 supports R&I on the full TRL-scale, in accordance with the specific intervention logic of each programme part, which is sometimes operationalised via technological roadmaps. This analysis, however, cannot be conclusive since the data is not comparable nor granular enough (e.g TRL-level at start or at end of the project) . This is an area where further monitoring is needed.

Figure 8 Overview of TRL supported across different Horizon 2020 programme parts based on non-comparable assessment methodologies

Source: European Commission, based on assessmentsof each Horizon 2020 programme part (Annex 2. The stronger the colour, the higher the concentration of projects on those TRLs.)

9.2. To what extent is Horizon 2020 coherent with other EU initiatives?

Expectations: Horizon 2020 shall be implemented in a way which is complementary to other Union funding programmes and policies, including the European Structural and Investment Funds (ESIF), the Common Agricultural Policy, the Programme for the Competitiveness of Enterprises and small

and medium enterprises (COSME) (2014-2020), the Erasmus+ Programme and the Life Programme. 16

9.2.1. Overview

Compared to FP7, Horizon 2020 attaches greater importance to the coherence with other

 EU instruments, which is illustrated by the legal base. 17 Linkages between Horizon

2020 and other EU initiatives are regarded as important in order to streamline resources, avoid duplications and simplify. It could also provide better and seamless support to the entire dis

15 A detailed overview of the approach adopted for this analysis for each programme part is presented in Annex 1.

16 Article 20 of the Horizon 2020 Regulation.

17 In the case of ESIF, both programmes include legal provisions to maximise synergies.

covery, research, development and innovation process and ensure better exploitation of projects/programmes

results.

It is however difficult to assess to what extent the political willingness to increase the external

coherence has been translated in practical implementation. The thematic assessments

show that the practical implementation of this approach seems to differ across specific

objectives of Horizon 2020.

For example, the assessment of Horizon 2020 financial instruments performed for this interim

evaluation suggest that they are seen as broadly coherent with other EU programmes,

although there are some areas of overlap with COSME in respect of the SME-targeting

financial instruments.

In the LIFE Programme Regulation 18 projects that foresee to take up the results of environmental

and climate-related research and innovation projects financed by Horizon 2020, mainly

under the SC5, or by preceding Framework Programmes are granted one extra point during

the evaluation proce ss. An analysis conducted by EASME for the “Nature and biodiversity”

theme showed that the number of successful projects linking their activities to the results of

EU-funded research projects has increased from a share of 5% in 2014 (corresponding to 2

projects out of 41 funded) to a share of almost 32% in 2015 (corresponding to 13 projects out

of 41 funded), without however identifying the quality of this integration so far.

Further examples on the external coherence of Horizon 2020 programme parts with other EU

initiatives are provided in the box below. An internal survey among Commission services on

the external coherence between EU instruments revealed that most complementarities/synergies

of Horizon 2020 with other EU policies or initiatives are perceived to be

with the European Structural and Investment Funds as well as with the EU Industry

competitiveness and SMEs policy. However, in many cases the coherence with other EU

initiatives is limited only to programming level, not followed by concrete actions. Indeed

the programmes might have different scales, scope or follow different implementation structures,

intervention logics, timeframes even if focussing on a similar thematic area. More detailed

analysis of complementarities, gaps and overlaps with specific EU initiatives are provided

in the thematic assessments (Annex 2).

Stakeholders were asked to comment on the coherence between Horizon 2020 and different

EU programmes. Illustrating the overall lack of knowledge of the coverage of other EU interventions,

most of the stakeholder consultation respondents felt they were unable to assess the

level of coherence of Horizon 2020 with other EU programmes because of their lack of familiarity

with other initiatives. When they could they indicated that the synergy with other EU

programmes is still very limited. More than 27% stakeholder consultation’s respondents

pointed out that Horizon 2020 and Erasmus + complement each other. 15.6% of respondents

said the same about ESIF and 12% judged that they work in synergy.

Box: Example of external coherence of Horizon 2020 with EU initiatives

In case of Research Infrastuctures for competitiveness and security reasons there is a major need to develop a more comprehensive approach to underpinning the success of the European Digital Open

Market, with a clearer focus on the key areas of the European Open Science Cloud and HPC.

18 Regulation (EU) No 1293/2013 i of the European Parliament and the Council of 11 December 2013 on the establishment of

a Programme for the Environment and Climate Action (LIFE) and repealing Regulation (EC) No 614/2007 i

Societal Challenge 2 shows a high degree of coherence, complementarity and synergies with several other EU policies, particularly the Common Agricultural Policy (in particular with the implementation of the EIP 'Agricultural productivity and sustainability'), the Common Fisheries Policy, maritime and climate policies, but also environment, energy, industry and competitiveness, public health and consumer protection.

Societal Challenge 6 priorities are fully linked to the main EU policies dealing with migration; jobs, growth and investment; the Digital single market; Justice and fundamental rights based on mutual trust; Making the EU a stronger global actor; and fostering a union of democratic change. Though, this coherence is not necessarily based on active coordination and exchanges, which can be further enhanced through more exchanges between relevant services.

9.2.2. Synergies with COSME, Connecting Europe Facility (CEF) and the European Fund for Strategic Investment (EFSI)

In the following table, a comparison is made between the characteristics of the financial instruments under Horizon 2020 (InnovFin) and a number of other relevant key programmes that are being implemented in the 2014-20 period in order to assess their coherence, check key differences and assess whether there is a sufficiently clear delineation between EU programmes. It should be noted that the European Fund for strategic Investment (EFSI) is a new financing scheme rather than a new financial instrument.

Figure 9 Comparative overview between InnovFin Financial instruments and COSME

Financial instru Type of inments

schemes Budget struments/ Eligibility cri Extent of differentiation with (title, description) budget teria InnovFin

Total budget of €  Funding available for start-ups

2.5 billion of under COSME, but not InnovFin which 60% (~ €  Intervention conditional on mar

1.4 billion) supket failure whereas under Innoports financial vFin, improving the conditions of instruments financing for innovative firms,

COSME - Europe’s Guarantees with a not only market failures are conprogramme for LGF - Guarantees particular focus COSME - all sidered sufficient justification for small and mediumand counteron financing of start-ups and intervention. sized enterprises. guarantees for SMEs <€150,000. SMEs provided  COSME focused on micro enter Financial instrufinancial interme Risk capital prefinancial intermeprises - expected that 90% of

ments strand: diaries (e.g. guardominantly into diary can demonbeneficiaries will have <10 emantee organisa SMEs at the strate market failployees with an average guaran

Loan Guarantee tions, banks, leasgrowth and exure teed loan of about €65,000

Facility (LGF) ing companies) to pansion stage InnovFin – 10  COSME focuses on SMEs operprovide more loan criteria that finanating across borders (but not ex

Equity Facility for and lease finance > € 150,000: for cial intermediary clusively).

Growth (EFG) to SMEs. SMEs not eligible must use to  Geographic scope - SMEs, estabin principle under demonstrate marlished and operating in one or

EFG - Investment Horizon 2020 ket failures apply. more EU Member States and in risk-capital (InnovFin) COSME Associated Countries. funds that provide  Applicable state aid rules are de VC and mezzaminimis under COSME whereas nine finance to the state aid rules that apply unexpansion and der InnovFin fall under Art. 21 of growth-stage the General Block Exemption SMEs. Regulation.

The Connecting CEF – Grants, CEF – transna The CEF funds trans-European

Europe Facility CEF programme contributions to tional requirenetworks in the fields of

(CEF) budget - €30.4 billion in total innovative finanment. transport, energy and digital services.

Supports targeted (€22.4 billion for

cial instruments,

developed togeth InnovFin Energy  Unlike the CEF, InnovFin does

infrastructure in Transport, €4.7 er with entrusted Demo: not fund trans-European energy vestment at EU levbillion for Energy, financial institu Only projects of networks, but instead finances el. Supports the and €0.3 billion tions such as the TRL 7-8 are eliinnovative first-of-a-kind energy development of infor Telecom). European Investgible under facilidemonstration projects in the

Financial instru Type of inments

schemes Budget struments/ Eligibility cri Extent of differentiation with (title, description) budget teria InnovFin

terconnected transment Bank, such ty fields of renewable energy, sus European networks as: the Marguerite tainable hydrogen and fuel cells. in the fields of Fund, the Loan Only pro InnovFin Energy Demo focuses transport, energy Guarantee for jects/companies on energy only, whereas the CEF and digital services. TEN Transport located in an EU focuses on three thematic areas.

(LGTT) and Pro Member State or  However, the CEF focuses on ject Bond Initia H2020 associated electricity and gas interconnective. countries eligible. tions between different European markets and is not based on InnovFin Energy demonstrating the market poten Demo: tial of renewable energies unlike Loans or loan InnovFin. guarantees be InnovFin doesn’t have transnatween EUR 7.5m tional requirements, whereas and 75m CEF projects are by definition

transnational. EFSI SME Window

EFSI is not formally a financial instrument

 itself. Rather, Equity and qua

InnovFin SMEG

qualifies for addi Non-duplicative since EFSI allows funding will be NA si-equity instruchannelled through ments (including

tional top-up InnovFin to simply increase volume

existing proguarantees) funding to expand effects and leverage. It is not an scale alternative source of funding.

grammes e.g. SME

Guarantee Facility

Source: InnovFin thematic assessment, See Annex 2

The table above shows that there are clear differences between InnovFin and other EU

funding instruments (especially the CEF 19 ), although there are some areas of possible

overlap. This includes a possible overlap between the InnovFin and COSME programmes in respect of SME finance, but the extent to which this is considered a problem varies among financial intermediaries.

9.2.2.1.Coherence with COSME

The InnovFin and COSME programmes were created as a result of a political decision to go ahead with two separate programmes. There was a subsequent need to differentiate the financial instruments supported through InnovFin and COSME respectively as part of the process for developing the detailed programming architecture. This has meant that that the two programmes, which risked being duplicative, have each developed their own intervention logic, programming and policy rationale, which underpins their differentiation at the implementation stage. In particular, coherence between InnovFin and COSME FIs has been achieved by:

 Setting different policy objectives:

o COSME - supporting start-ups and SMEs, promoting entrepreneurship and addressing clear market failures;

o InnovFin - promoting access to finance for innovators, improving the terms and conditions for access to innovation finance respectively.

 Defining different targeting strategies, although there is some overlap of targeting in

the SME segment.

19 https://ec.europa.eu/inea/en/connecting-europe-facility

 Drawing up different eligibility criteria – e.g. in the case of COSME, criteria relating

to market failure and being an SME, in the case of InnovFin, developing a list of 10 innovation-related criteria.

 Using different State Aid rules – de minimis (COSME) vs. Art. 21 of the GBER (InnovFin).

The fact that the two programmes have evolved in different ways to avoid duplication has led to a reasonably clear delineation emerging, even if there is some blurring of targeting strategies for the SME instruments (i.e. both the LGF and SMEG provide guarantees to SMEs). Whilst InnovFin puts a stronger emphasis on SMEs having to be innovative, the definition of innovative is quite broad and arguably investing in SMEs irrespective of their degree of innovativeness involves a higher risk given the two valleys of death than for other types of lending and equity investment. SME final beneficiaries may be eligible to participate in either programme through a financial intermediary, which has caused some confusion for financial intermediaries as to which FI to apply for until the schemes and their differences became better known.

However, some overlapping may not necessarily a negative in practice since in some Member States, financial intermediaries have only applied to COSME (or only applied to InnovFin). Therefore, SMEs may not be able to participate in both financing schemes through an intermediary in all 28 EU Member States. Moreover, the geographic coverage of InnovFin (EU28 plus 14 associated countries) is wider than for COSME which mainly focuses on the EU although there are a very limited number of COSME- associated countries.

From the InnvFin thematic assessment it appears that start-ups are not explicitly targeted through InnovFin. Given the central importance of start-ups from a jobs and growth perspective, and in light of the Europe 2020 strategy, this could be seen as a gap from an internal programming architecture perspective. However, it could be argued that start-ups and very early stage SMEs can already be funded through the COSME Loan Guarantee Fund (LGF) given the focus on market failures and the decision to focus on loans below the €150,000 threshold. It could also be argued that there is adequate provision for access to finance for start-ups in at least some Member States through national funding schemes, such as

BPIfrance’s start-up scheme, BPI Prêt d'amorçage 20 , although micro-loans made are not

backed by a guarantee.

There is a good level of coherence between the SME Instrument and the COSME programme, with synergies created by making use of the Enterprise Europe Network (EEN) to ensure access to the SME Instrument business innovation coaching and mentoring services. Also the thematic assessment of the Innovation in SMEs programme part points that concerning the INNOSUP call under "Innovation in SMEs", there is room for a stronger coordination with the COSME financial instruments.

9.2.2.2.Coherence with the European Fund for Strategic Investment (EFSI)

According to an ad-hoc audit, EFSI contributes mainly to the R&D (45%), energy (21%) and

ICT (17%) sectors. 21 The European Investment Bank (EIB)'s evaluation finds that there are

20 http://www.bpifrance.fr/Toutes-nos-solutions/Prets/Prets-sans-garantie/Pret-d-amorcage

21 Ad-hoc audit of the application of Regulation 2015/1017 (EFSI Regulation) https://ec.europa.eu/priorities/sites/beta

href="https://ec.europa.eu/priorities/sites/beta-political/files/ey-report-on-efsi_en.pdf">political/files/ey-report-on-efsi_en.pdf The EU-15 received 91% and EU-13 received 9% of EFSI support. Reasons mentioned for the lower support in EU-13 are the competition from ESIF, less capacity to develop large projects, less experience with PPPs, less developed Venture Capital market and small size of projects.

both risks and opportunities in the relationship between the EIB and Horizon 2020 and the Connecting Europe Facility (CEF). The risk is that EIB privileges EFSI operations over Horizon 2020/CEF operations. Opportunities could reside in the fact that EC could use Horizon 2020/CEF funds to finance the First Loss Piece (FLP) of operations, while the EIB would finance mezzanine tranches under EFSI. Stakeholders indicated that there is competition with other EU funds such as certain financial instruments under CEF and Horizon 2020 or financial instruments and grants under ESFI.

The ad-hoc audit recommends to further structure and enable complementarity with and avoid overlap with Horizon 2020, ESFI, and other funds. This is more urgent in certain countries (EU-13) and sectors.

From the InnovFin thematic assessment (Annex 2 Part H) the availability of supplementary financing through the European Fund for Strategic Investment (EFSI) has not caused any problems relating to coherence, because the funding will be used to top up the InnovFin SMEG rather than to create new rival, alternative financial instruments.

However, efforts are currently underway to refocus some of the InnovFin instruments partly because since the set-up of EFSI in 2015, it has proved challenging to reach InnovFin's objectives, as a significant part of the products deployed overlap with EFSI in terms of both risk spectrum and eligibility. Indeed, the introduction of EFSI has arguably slowed down the deployment of InnovFin. In 2016, the Bank signed only EUR l.5 billion of InnovFin transactions, reaching just 56% of the annual objective, while EIB signed EUR 2.4 billion of EFSI financing under the RDI EFSI objective. This modest 2015 InnovFin activity confirmed a declining trend since the launch of the initiative: EUR 2.5 billion signed in 2014, followed by EUR 2.0 billion in 2015, the year EFSI was launched. In cumulative terms, as at 31 December 2016, the EIB had deployed EUR 5.9 billion of financing under the InnovFin programme across 96 operations spread between the EU and EIB windows. The budgetary contribution from the EU to date to support the existing portfolio amounts to about EUR 0.8 billion. In total, over the first three years to December 2016, only 73% of the target of EUR 8.1 billion for the period was achieved, representing a cumulative shortfall of EUR 2.2 bn. InnovFin was expected to make up for this shortfall in the period 2018-2020 in the period following the deployment of EFSI as originally conceived; but in the context of discussions on the extension of EFSI to 2020 ("EFSI 2.0'), this assumption is no longer valid.

Given the above, the EIB and DG RTD of the Commission have concluded that the current approach is no longer sustainable and that changes are necessary to re-focus InnovFin's deployment in the post-EFSI context. The aim is to improve cooperation with EFSI through better complementarity and to combine InnovFin and EFSI financing where needed, building on the success achieved under EFSI's SME window (implemented with the European Investment Fund (EIF)) for both equity and debt joint InnovFin/EFSI products.

Work is underway to transform InnovFin into two portfolios — one for debt, one for quasi-equity —covering a wider range of risk profiles and underpinning a suite of products that more closely target, in marketing terms, a variety of constituencies. This approach will, in turn, make it possible to build on the experience of the current thematic pilots - 'InnovFin Energy Demo Projects', 'InnovFin Infectious Diseases' - and open up possibilities for crafting risk- finance products for other sectors.

Currently, the EIB InnovFin product portfolio is composed of two high-risk thematic products (see above) and three non-thematic products, which represent the main overlap with EFSI:

InnovFin Large Projects has a very similar eligibility to the debt financing under EFSI Infrastructure

and Innovation Window (IIW); InnovFin MidCap Guarantee has an equivalent

product offering as EFSI's Risk Sharing; and equity-type operations under InnovFin Midcap

Growth Finance have already been fully transferred under EFSI with the European Growth

Finance Facility.

Two new facilities are envisaged with minimal overlap with EFSI: InnovFin Research Institutes,

Universities, Research Organisations Facility (RIURO), and InnovFin Moderate &

Modest Innovator Countries and Associated Countries Facility (MMI). RIURO will strengthen

the InnovFin focus on research organisations, including public entities. MMI will target

regions which are currently underserved by InnovFin operations, in particular in Associated

Countries, but also in EU countries indicated as less innovative in the 2016 Innovation Scoreboard.

The possibility of combining InnovFin and EFSI finance is also being explored, with InnovFin

used as a junior tranche to credit-enhance EFSI equity-type deployments. This would

potentially unlock new financing options in the fields of risk-sharing for corporate R&I and

corporate venture. Besides this refocusing, the equity side of operations, implemented by EIF,

has been remodelled to improve its relevance to a wider range of constituencies.

9.2.3. Synergies with European Structural and Investment Funds

Horizon 2020 shall also contribute to the closing of the research and innovation divide within the Un

22

ion by promoting synergies with ESIF. Where possible cumulative funding may be used.

Synergies between Horizon 2020 and the European Structural and Investments Funds (ESIF)

aim at maximising the quantity and quality of R&I investment and their impact. In doing so,

synergies seek to bring together research and business communities as well as relevant national

and regional policy designers and implementing bodies, thus ensuring a higher impact

of the funds for a knowledge-based economic transformation.

The Commission has taken specific measures to facilitate the coherence between Horizon

2020 and ESIF through the publication of guidance for policy-makers and implementing bodies

and a specific brochure for interested parties with examples of synergies 23 . Even though it

is possible for the first time to combine Horizon 2020 and ESIF support in a single project in

this programming period and a strong focus and political importance are put on synergies,

Horizon 2020 is implemented under central management by the Commission whereas ESIF is

under shared management, with joint responsibility of the Member State authorities and the

Commission. This leads to differences in implementation rules and procedures, as the commitment

of actors as well as the application and project

execution requirements differ. [Horizon 20202 should]…focus on areas that national funders don´t support and

synergies with ESIF- good practices- are

Under ESIF the selection of specific type of R&I interneeded; Foster clustering and sharing

vention can vary among regions (research infrastructure, experience in Widening projects (Teaming, ERA chair).

industry academia cooperation, etc.) and Member States, Czech Republic,CEITEC

22 Article 21 of the Horizon 2020 Regulation.

23 Guidance: "Enabling synergies between European Structural and Investment Funds, Horizon 2020 and other research, innovation

and competitiveness-related Union programmes" . Brochure: "EU Funds working together for jobs and growth" with

examples of synergies between Horizon 2020 and ESIF was published in 2016.

https://ec.europa.eu/research/regions/index.cfm?pg=synergies

according to the strategic choices made in each territorial context. ESIF supports the same type of beneficiaries for several Horizon 2020 areas: Research Infrastructures, LEIT NMBP, Innovation in SMEs, SC1, SEWP and SWAFS and has similar objectives compared to Horizon 2020's specific objectives (LEIT NMPB, SC1, SC3 and SWAFS) in terms of TRL covered. However, the overlapping between the two programme is seen as not possible due to the different geographical coverage of the two instruments, i.e. the ESIF fund national or regional initiatives, while Horizon 2020 is based on trans-national partnerships and networking and hence helps strengthen scientific collaboration at the EU and international scale.

According to an external study, 24 the development of synergies between both programmes is

still at the early stage and their development is limited. Although a clear legal basis for synergies is in place and overall implementation guidance have been compiled by the Commission, the study concluded that communication, coordination and support to the synergies between all the institutional actors involved is not optimal. The enhancement of the coherence between Horizon 2020 and ESIF is rather incidental which is confirmed by the thematic assessments. During the project phase, there are notably discrepancies that hinder a combined use of funding means. These include different funding rates and eligibility rules, which are not always coherent with each other. The difference in State Aid rules under ESIF and Horizon 2020 further leads to legal uncertainty for potential beneficiaries.

Over the last years Smart Specialisation Strategies (S3) have taken their place as a key programming tool for R&I in Member States and regions. S3 are the ex-ante conditionality underpinning R&I funding through ESIF. The now over 120 existing S3 at national or regional levels provide a framework for synergy action with Horizon 2020 as they identify priority areas and activities. Smart specialisation can provide a framework to develop complementarities through "upstream actions" to prepare regional R&I stakeholders to participate in Horizon 2020, and "downstream actions" to exploit and diffuse R&I results, developed under Horizon 2020 and previous programmes, into the market. A pilot project Stairway to Excellence (S2E)

was launched to support EU-13 Member States and their regions in this regard. 25

In order to build up on the evaluations of high quality proposals under Horizon 2020 SME Instrument and MSCA actions, the Seal of Excellence (SoE) initiative has been launched by the Commission to support synergies with national/regional initiatives by highlighting high quality projects for further public or private funding. The 'SoE' certificate is an official recognition of the value of proposals that have succeeded the evaluation by independent experts run by Horizon 2020. The regional and national authorities, who decide to fund the 'SoE' proposals (with national or ESIF resources), select them on the basis of eligibilities and ways to achieve the objectives if their regional or national their smart specialisation strategies. This is expected to result is an operation with a clear added value in terms of maximising the impact of investments in R&D, with the national and regional authorities also benefitting from a technical evaluation already performed. While comprehensive data is not yet available on the exact number of proposals for which these quality labels allowed applicants to secure other sources of public or private funds, there is currently evidence of an increasing number of national and regional funding schemes that offer support to SME Instrument pro

24 "Synergies between Framework Programmes for Research and Innovation and the European Structural and Investment

Funds" [forthcoming]

25 Stairway to excellence (S2E) is a European Parliament Pilot Project executed by DG-JRC together with DG-REGIO. It

assists EU-13 Member States and their regions in closing the innovation gap, in order to promote excellence and to stimulate the early and effective implementation of national and regional Smart Specialisation Strategies http://s3platform.jrc.ec.europa.eu/stairway-to-excellence

ject proposals awarded with a SoE 26 . It is however recognised that the SoE initiative has not

yet tapped into its full potential, which would be possible with alignment of rules, for

example in the case of State Aid rules 27 . According to the Innovation in SMEs assessment,

opinions are divided when it comes to the usefulness of the SoE to effectively influence

funding decisions. While agencies indicate a limited influence of the SoE so far, there are

signs that this may change in the nearby future. SoE holders display a strong confidence that

the SoE makes and will make a difference in funding decisions, be it implicitly or explicitly.

The following Box provides examples of the coherence of Horizon programme parts with the

European Structural and Investment Funds.

Box: Examples of coherence of Horizon 2020 programme parts with ESIF

In the LEIT ICT area a number of cases have been identified where research activities under national programmes

act as stepping stones to Horizon 2020 projects and, conversely, where FP/Horizon 2020 projects have led to research being funded by national or regional sources. However, survey results suggest that respondents had limited knowledge or experience regarding the synergies that could be developed by combining Horizon

2020 and other sources of funding. Survey findings also suggest that participation in Horizon 2020 does not seem to offer any competitive advantage for securing funding from other sources.

An analysis of FP7 in the LEIT NMBP areas 28 showed that most regions have participated as much as one

would expect from their level of activity: regions with more R&D resources tend to participate more. The main factors for high performing regions are the track record and level of specialisation, but also the level of regional

expertise. In this context, the creation of regional research centres, some of which were established in the 80’s and 90’s to diversify incentives to innovation, appears to pay off.

The SC5 Work Programmes 2014-15 and 2016-17 underline the possibility of complementing Horizon 2020

support with private or public funding, “including for relevant national/regional schemes under the European

Structural and Investment Funds (ESIF), in particular under the European Regional Development Fund

(ERDF) ”. The procedures to access to this complementary ESIF financing further to the Horizon 2020 funding

are explained in the Work Programme. Until end 2016, looking at a sample of 75 proposals, 29 mentioned synergies.

Amongst the 75 proposals, 19 have been granted, of which 15 referring to complementarity with ESIF. Interestingly, proposals arguing their potential synergies have a higher success rate. There is however no strong commitment to deliver on synergies as a project outcome and the actual complementarity between Horizon 2020 and ESI Funds may not be reflected in the annual ESIF reporting.

In case of SEWP the design of the programme entails synergies with cohesion policy in particular for Teaming

where applicants are obliged to ensure appropriate co-financing for the infrastructure and equipment component of the centres of excellence from the ESIF or other sources. Beyond the mere financial dimension, the programme is well aligned with the overall objectives of cohesion policy notably to help less developed European countries and regions in order to catch up and to reduce the economic, social and territorial disparities that still

exist in the EU.

In MSCA, ESIF investments can be in support of COFUND, for instance in the form of investment in infrastructures, large equipment (European Regional Development Fund) or training and networking (mainly European Social Funds).

26 SoE friendly funding schemes for SME Instrument Phase 1 and/or Phase 2 proposals are operational at a national or regional

level in Spain, Italy (8 Southern regions, plus Lombardy, Piedmont, Friuli-Venezia Giulia), France (Ile-de-France),

Sweden, Norway, Czech Republic (Brno), Cyprus, Slovenia, Hungary and Finland. Moreover it is expected that more seal schemes will be soon launched in other EU counties and/regions. In the case of MSCA, Cyprus and the Czech Republic have

already introduced funding schemes using ESIF to support recipients of the Seal of excellence while Croatia, Greece, Lithuania, Poland, Slovenia, Sweden have initiatives in the pipeline.

27 For further information about the state aid implications of the Seal of Excellence, see: European Commission (2017),

Commission Staff Working Document - Explanatory note of the Commission services on the application of State Aid Rules

to national and regional funding schemes that offer alternative supp support to SME Instrument project proposals with a Horizon 2020 'Seal of Excellence', SWD(2017) 11 final

28 Study “Mapping the regional embeddedness of the NMP programme”, INNOVA et al., 2016. No significant differences

between NMBP areas were detected, therefore these results are considered relevant also for Horizon 2020.

9.3. To what extent is Horizon 2020 coherent with other initiatives at national, regional and international level?

With the aim of achieving the greatest possible impact of Union funding, Horizon 2020 should develop closer synergies, which may also take the form of public-public partnerships, with international, national

and regional programmes that support research and innovation. In that context, Horizon 2020

should encourage the optimal use of resources and avoid unnecessary duplication. 29

9.3.1. Overview

At the international level, the establishment of Sustainable Development Goals (SDGs), the successor of the United Nations' Millennium Development Goals, and the COP21/22 paved the way for developing a more coherent approach within Horizon 2020 to address these objectives. Research and innovation policy and related implementation measures are

seen as engines of a transformative agenda built around universally applicable SDGs. 30 Examples

 of coherence between Horizon 2020 and international initiatives can be found in

LEIT-Space and the European Space Agency (ESA) 31 or in case of SC5's strong alignment at

international level e.g. through the Belmont Forum, GEO, IPCC, Transatlantic Research Alliance. In addition, in SC3 the Commission’s active role in Mission Innovation is expected to improve coherence with regard to similar initiatives of the main global actors outside the EU.

With regard to the alignment of national research strategies and programmes, Horizon 2020 further strengthens instruments already developed under FP7 – for instance, ERA-NET Co Fund and Article 185 initiatives – to pool resources across Member States, define common strategic research agendas, avoid duplication, implement joint calls, etc. Furthermore, the Policy Support Facility under Horizon 2020 aims to assist Member States to implement effective reforms, in line with the priorities of the European Research Area.

It follows from the thematic assessments that the scope and scale of coherence between Horizon 2020 and other non-EU initiatives depends on the policy domain. Complementarity between Horizon 2020 and national activities is often seen by Member States as crucial. Duplication, on the other hand, is not always perceived by default as a negative aspect, especially taking into account the global character of Horizon 2020 Societal Challenges. However, in some countries it is difficult to ensure national coordination with Horizon 2020, notably in R&I systems where the bottom-up approach dominates. In general, in the context of coordination with R&I activities carried out at national level, the Horizon 2020 Programme Committees plays a special role, and the involvement of Member States in its activities is of special interest and often underlined by Member States.

Horizon 2020 aims to improve the coherence with national programmes through support to Public-to-Public Partnerships (P2Ps) including Joint Programming Initiatives (JPIs), the ERA-

NET Cofund instrument and the Art.185 initiatives 32 . Efforts from Member States towards

coordination in the field of P2Ps are core components of the ERA Roadmap and national ERA

29 See recital 39 of the Horizon 2020 Regulation.

30 Report “Follow-up to Rio+20, notably the Sustainable Development Goals (SDGs)” of an independent expert group.

https://ec.europa.eu/programmes/horizon2020/en/news/role-science-technology-and-innovation-policies-fosterimplementation-sustainable-development

31 The Delegation Agreement for Galileo R&D has represented a major achievement in this regard.

32 Article 185 of the Treaty on the Functioning of the European Union (TFEU) [ex Article 169 of the Treaty establishing the

European Community (TEC)] enables the EU to participate in research programmes undertaken jointly by several Member States, including participation in the structures created for the execution of national programmes.

action plans, notably for priority 2a of the ERA Roadmap ( ‘Jointly addressing societal challenges ”).

EU support for P2Ps was introduced under FP6 (mainly with the notion of networking of national

programmes) and financed with EUR 380 million (2.1% of the FP6 budget). Support

under FP7 reached EUR 802 million (1.4% of the budget) and will reach approximately EUR

2.500 million in Horizon 2020 (around 3.1% of the budget). In parallel, the EU contribution

mobilised around EUR 1.250 million of national funding under FP6 and around EUR 2.900

million under FP7, whereas the EU contribution in Horizon 2020 is expected to mobilise EUR

6.000-8.000 million of national funding. Since 2004 more than 5,500 projects with a cumulative

budget of about EUR 5.000 million from national sources have been implemented

through P2Ps.

Box: Examples of coherence of Horizon 2020 with the regional and national

levels

In the LEIT NMBP area, links between activities at European level and national or regional strategies and programmes

were developed through previous framework programmes and there are now examples of follow-up investments made by regions to take the results of successful projects further. Regions with more R&D resources tend to invest more; the track record of regions and their degrees of specialisation, as well as the level of regional

expertise, are key factors. This support is indispensable to the NMBP strategy of developing pilot lines as a means of supporting innovative SMEs in validation and scale-up activities. Out of the projects financed by the InnovFin Large Projects scheme of the EIB, at least 40% are KETs-related.

SC1 has set encouraging precedents of associating EU regional partners to the definition and implementation of

roadmaps and strategic agendas. However expanding this would require some additional work.

Several actions of SC2 have supported the links with the national or regional plans and smart specialisation strategies using ESIF, for instance those targeting development of new bio-based industries. The ERA-NET cofund actions included in the calls are potentially highly effective strategic investments in ERA, with structuring effect and ensuring coherence between EU and national research programmes in the bioeconomy.

In SC3 the situation has improved in Horizon 2020 thanks to the progress achieved in the SET-Plan which rallies national programme owners and managers from 32 European countries around common priorities

9.3.2. ERA-NET Cofund under Horizon 2020

ERA-NET Cofund is a policy instrument across all priorities of Horizon 2020 to catalyse joint

calls and other joint activities of national R&D programmes through additional Horizon 2020

top-up funding. The joint activities of Member States and Associated States contribute to increasing

the quality of national research, to increasing the level of national funding and to

avoiding the duplication of research. Mobility is promoted through the transnational research

projects resulting from the joint calls and additional mobility and staff exchange schemes. As

the selection of topics for ERA-NET Cofund actions is part of the Horizon 2020 Work Programmes,

a high coherence between national and EU programming can be ensured via the

responsible Programme Committees.

The Horizon 2020 Work Programme 2014-2015 included calls for proposals for ERA-NET

Cofund actions, resulting in a total of 27 proposals selected for funding by the European

Commission. The direct leverage effect of the 27 actions is 2.31, i.e. for each euro invested by

the EU, the participating states invest an additional amount of 2.31 €. The average budget per

co-funded call is around EUR 21.6 million while the average number of countries participating

in each co-funded call is 16.

In addition, over 30 topics are included in the 2016-2017 Horizon 2020 Work Programmes. Funding from Horizon 2020 for these is expected to reach about EUR 280 million and leveraging budgets from national sources of about EUR 700 million. The overall leverage effect for the period 2014-2017, adding the national funding for the co-funded calls and the unfunded calls (only financed by national sources) is expected to exceed the leverage effect measured for the period 2014-2015. Based on the planning of the current ERA-NET Cofunds and past experience, an overall leverage effect of 3-5 can be expected.

Figure 10 Calls for ERA-NET Cofund actions published in the WPs 2014-2017

2014 2015 2016 2017 €M no €M no €M no €M no Excellent Science

Future and Emerging Technologies 18,0 2 5,0 1 Industrial Leadership

ICT 6,0 1 Nano, Materials, Biotech and Manufacturing 12,5 1 30,0 3

Societal challenges

Health, demographic change and wellbeing 27,4 4 15,0 3 5,0 1

Food security, agriculture, marine, 5,0 1 15,0 3 35,0 5 15,1 3 bioeconomy

Secure, clean and efficient energy 36,8 3 36,3 4 45,8 5 33,5 4 Smart, green and integrated transport 10,0 1 Climate action, environment, resource 18,2 2 51,0 3 13,0 3 30,0 3 efficiency and raw materials

Europe in a changing world – inclusive, 5,0 1 5,0 1 5,0 1 5,0 1 innovative and reflective Societies

Science with and for society 5,0 1

Total 92,4 11 140,8 16 161,8 21 93,6 13

Source: Horizon 2020 Work Programmes

The evaluation of the ERA-NET Cofund scheme 33 highlights that the main added value

of the ERA-NET scheme is the lasting collaboration between and learning among funding agencies as well as capacity building. ERA-NET Cofund actions are relatively less perceived as strategic instruments that can influence national strategies and lead to alignment of national policies among participating states and/or EU R&D policies. With respect to efficiency, the evaluation concluded that a number of simplification measures have been introduced under Horizon 2020 that are appreciated by the ERA-NET community. The lack of clear understanding of the financial aspects of the ERA-NET Cofund instrument was however highlighted as an area for improvement. While the relevance of the ERA-NET Cofund instrument has been confirmed, coherence among ERA-NETs but also between the ERA-NETs and other joint initiatives is clearly underdeveloped. ERA-NET Cofund actions contribute effectively to strengthening transnational cooperation and creating a critical mass of resources to tackle EU societal challenges. The instrument has facilitated widening participation of lower performing countries. ERA-NET Cofund actions are also gradually increasing the participation of third countries.

9.3.3. Article 185 initiatives

Public-public partnerships may be supported though Union participation in Article 185 initiatives where the participation is justified by the scope of the objective pursued and the scale of the resources

required. 34

33 https://bookshop.europa.eu/en/analysis-of-era-net-cofund-actions-under-horizon-2020-pbKI0116995/

34 See Article 26 (b) of the Horizon 2020 Regulation.

Article 185 initiatives are jointly implemented Member States programmes. 35 They include

Associated Countries and, in some cases, extend the collaboration towards third countries. They are implemented on the basis of Annual Work plans of research activities that receive

EU funds from Horizon 2020 36 . Horizon 2020 currently implements four Article 185 initiatives. 37 The following table summarises the financial contribution of the EU and Participating

States to the four Article 185 initiatives.

Figure 11 Financial contribution of the Union and the Participating States to the Article 185 initiatives under Horizon 2020

Article 185 initiatives adopted under Horizon 2020 EU (max)) Participating States [EUR million] (min) [EUR million]

European & Developing Countries Clinical Trials Partnership 2 683 683 (EDCTP2) EDCTP (FP6) 200 200 European Metrology Research Programme (EMPIR) 300 300 EMRP, FP7 200 200 Eurostars2 (for SMEs) 287 861 Eurostars1 (FP7) 100 300 Active and Assisted Living R&D Programme (AAL2) 175 175 AAL (FP7) 150 200

Source: European Commission, (in bold: contributions for the predecessor programmes FP6/FP7)

On 18 October 2016 the Commission adopted a proposal to establish a new public-public Partnership for Research and Innovation in the Mediterranean Area (PRIMA) under Article 185, currently under negotiation in the Council and European Parliament, and expected to start implementation in 2018. PRIMA would focus on two key socioeconomic issues that are important for the region: food systems and water resources.

As foreseen in their respective basic acts, the Article 185 initiatives are currently being evaluated with the assistance of independent experts, with results to be reported to the Council and the European Parliament by end of 2017.

9.3.4. Joint Programming Initiatives (JPIs)

In July 2008, the Commission presented joint programming as a Member State-led process, designed to coordinate research in Europe and to address major societal challenges. Member State participation in the Joint Programming Initiatives (JPIs) follows the principle of variable geometry and open access. Currently there are 10 JPIs (see Figure below), which all have Strategic Research Agendas (SRAs) as one Key Performance Indicator (KPI) of Horizon 2020.

All JPIs have received support to their initial preparation phase (Coordination and Support Action (CSA) under FP7) that has been or will be extended by a second CSA under Horizon

35 Art.185 initiatives can only be proposed in cases 'where there is a need for a dedicated implementation structure and where

there is a high level of commitment of the participating countries to integration at scientific, management and financial levels'. See Art 26 of Horizon 2020 Regulation.

36 The origin of the name stems from the legal basis, Art.185 of the Treaty on the Functioning of the European Union

37 European and Developing Countries Clinical Trials Partnership 2 (EDCTP2): new or improved treatments for povertyrelated

diseases in sub-Saharan Africa; European Metrology Programme for Research and Innovation (EMPIR): new measurement solutions for industrial competitiveness and societal challenges; Eurostars 2: support to transnational collaboration of R&D performing SMEs; Active and Assisted Living Research and Development Programme (AAL): innovative ICT-based solutions for active and healthy ageing.

2020. In addition all but one JPI received additional EU funding from both FP7 and Horizon 2020, for the implementation of joint calls. From the currently planned 61 ERA-NET Cofund actions, 16 are in support of the JPIs.

Figure 12 Support from the Framework Programme for the JPIs via Coordination and support actions (CSA), ERA-NET-Plus (EN+) and ERA-NET Cofund (EN-CF)

Joint Programming Initiatives FP7 Horizon 2020

Until 2013 2014 2015 2016 2017

Antimicrobial Resistance CSA EN-CF CSA

Climate CSA EN-CF EN-CF

Cultural Heritage CSA, EN+ CSA

Agriculture, Food Security & Climate Change CSA, EN+ CSA, EN-CF EN-CF EN-CF

A Healthy Diet for a Healthy Life CSA CSA, EN-CF EN-CF

Neurodegenerative Diseases CSA EN-CF CSA

More years better lives CSA CSA

Oceans CSA CSA EN-CF

Urban Europe CSA EN-CF EN-CF CSA, EN-CF EN-CF

Water CSA EN-CF EN-CF CSA EN-CF

Source: European Commission

The Joint Programming evaluation 38 highlights in particular that the societal challenges of the

JPIs were selected by Member States but the overall level of ambition does not meet the initial expectations; the level of co-investment so far in joint calls and actions is no greater than for some ERA-NETs and there is no indication that Member States will increase their contributions significantly; most countries are not adapting their national research activities towards the SRA/SRIAs; inter-ministerial structures to support the joint programming process is rather mixed. Some demonstrate high level commitment but too many have not really made any progress; financial support through CSAs and the ERA-NET instruments has been vital to the development of the JPIs. The Commission also plays an important role in helping the JPIs to position themselves within both the European and international societal challenge landscape. There is a risk that the joint programming process is not sustainable, without a stronger role for the Commission; too many resources seem to be devoted to securing financial support from the Commission, while not sufficient resources are invested in ensuring the overall socio-economic impact of JPIs.

9.4. Key conclusions on the coherence of Horizon 2020

The integration of research and innovation into a single programme, the three pillars structure of Horizon 2020 and its focus on finding solutions to challenges (notably through the use of focus areas) rather than being domain-oriented improved its internal coherence compared to FP7. In such an integrated programme, there is however a need to ensure an appropriate balance between funding basic, collaborative and frontier research and higher Technology Readiness Levels, in order to maintain a link between industry and academia and to create groundbreaking technological foundation for innovations. An analysis of the Technology Readiness Levels supported across thematic areas shows that whereas the Excellent Science pillar focus

38 http://bookshop.europa.eu/en/evaluation-of-joint-programming-to-address-grand-societal-challenges

href="http://bookshop.europa.eu/en/evaluation-of-joint-programming-to-address-grand-societal-challenges-pbKI0416204/?CatalogCategoryID=7QwKABstDHwAAAEjK5EY4e5L">pbKI0416204/?CatalogCategoryID=7QwKABstDHwAAAEjK5EY4e5L

es on more fundamental research and, with the exception of e-Infrastructures, does not move beyond the stage of an experimental proof of concept, the rest of the programme is rather concentrated on higher TRLs, the majority of which are targeting product demonstration in both the Industrial Leadership and the Societal Challenges pillar. Multiple types of stakeholders regret that Societal Challenges and LEIT do not invest more in lower TRL collaborative research, which is regarded as one key source of future breakthrough innovations, albeit longertermed, in line with societal needs.

The different types of action of Horizon 2020 appear coherent to address the different objectives of the programme, but the large number of instruments at EU level and complex funding rules are difficult to understand for potential applicants and may lead to overlaps.

In terms of external coherence there is a clear strategic willingness to ensure complementarity and synergies of Horizon 2020 with other EU programmes, in particular the European Structural and Investment Funfs (ESIF) and the European Fund for Strategic Investments (EFSI). Compared to FP7, efforts have already been made to increase the synergies between Horizon 2020 and other programmes, notably ESIF. However, strong evidence is lacking on how far this has materialised in practice yet. Given the different rules and implementation structures, promoting synergies at project level (in term of combining different financing sources for the same project) still appears difficult. The difference in state aid rules further leads to legal uncertainty for potential beneficiaries.

Member States' support to public-public partnerships has significantly increased over the past years. Although generating lasting collaborations between entities and improved capacities, the public-public partnerships are not seen as influencing the alignment of national strategies and policies. The Member States-led joint programming process is regarded as unsustainable, without Union intervention, especially during times of economic austerity in many countries.

• 10. 

  WHAT IS THE EU ADDED VALUE OF HORIZON 2020 SO FAR?

This question aims to assess the value resulting from Horizon 2020 that is additional to the value that could result from interventions which would be achieved by Member States at national and/or regional levels.

Expectations on the European added-value of Horizon 2020

Based on the Horizon 2020 impact assessment, compared to a renationalisation of R&I policies, Horizon 2020 is expected to allow for the orientation of European research and innovation programmes to commonly agreed objectives, as well as for the fostering of initiatives that fundamentally restructure the European R&D landscape. The programme is also expected to allow for research that only takes place through EU-funded collaborative research projects and to produce more scientific, technological and innovation impacts, which should translate into higher economic and competitiveness, social, environmental and EU policy impacts.

Summary box: Key findings on the EU added value of Horizon 2020

 Horizon 2020 produces demonstrable benefits compared to national and regional-level support to R&I in terms of scale, speed and scope, notably through the creation of excellence through competition, the creation of international, trans-national, multidisciplinary networks; pooling of resources; creating a big leverage effect and creating critical mass to tackle global challenges.

 Horizon 2020 increases the EU's attractiveness as a place to carry out R&I.

 Horizon 2020 is seen as improving the competitive advantage of participants for example through international multi-disciplinary networks, the sharing of knowledge and technology transfer and access to new markets.

 The additionality of Horizon 2020 is very strong – support is given to fund distinctive projects, which are unlike those funded at national or regional level.

 The impacts of discontinuation are difficult to quantify, but are likely very large.

10.1. Additional value compared to national and/or regional levels

In the field of R&I the application of the concept of EU Added Value (EAV) has expanded along with successive Framework Programmes (FPs). The basic principle underlying the FPs has been from the start the undisputed justification for public intervention in R&I, which is

linked to well-studied and important market and systemic failures 39 .

Figure 13 The evolving character of European Added value through successive FPs

Dimensions of European Added Value FP1 FP2 FP3 FP4 FP5 FP6 FP7

1984- 1987- 1990- 1994- 1998- 2002- 2007- 1988 1991 1994 1998 2002 2006 2013

Scale too big for Member States (MS) to handle alone X X X X X X X

Financial benefits: a joint approach would be advantageous X X X X X X X

Combines complementary Member States efforts to tackle X X X X X X X European problems

Cohesion X X X X X X X

Unification of European science & technology across borders X X X X X X X

Promotes uniform laws and standards X X X X X X X

Mobilising EU potential at European and global level by co X X X X ordinating national and EU programmes

Contributes to implementing EU policy X X X

Contributes to societal objectives (later ‘grand challenges’) X X X

Exploits opportunities for the development of European sci X X X ence, technology and industry

Structures the EU R&D community and ‘fabric’ X X

Improves quality through exposure to EU-wide competition X

Source: Technopolis, Science Metrix, Understanding the Long Term Impact of the Framework Programme, 2012

The design of earlier FPs had taken up considerations such as scale, complementarity of efforts, trans-national interaction, standardisation, implementation of EU policy, achievement of societal objectives and structuring effects on the European research and innovation ecosystem. FP7 introduced the concept of EAV derived from EU-wide competition for excellence

39 See the Horizon 2020 Impact assessment for an extensive list of studies, evaluations and publications

http://ec.europa.eu/research/horizon2020/pdf/proposals/horizon_2020_impact_assessment_report.pdf

(notably used when introducing the ERC). Horizon 2020 was designed to address all these

considerations and also added a focus on coordination with respect to ’internationalisation’. 40

Much of the EU support to R&I is unique compared with national funding. In the case of

MSCA, which is a mobility programme, evidence shows for example that the research impact

of internationally mobile researchers is up to 20% higher than the impact of those who opt to

stay in their home country 41 . Furthermore, the full value and impact of opportunities is often

revealed after many years, illustrated by the number of Nobel Prize winners who had previously

benefited from the MSCA (see Section 8.1.1.4).

A key aspect of EAV for EU support to R&I is the synergy it creates across Europe (and beyond)

through trans-national collaborations of systemic importance. This collaboration brings

the R&I effort closer to the critical mass required to tackle challenges of a societal scale. This

is most evident in challenges of such a scale and complexity that no single Member State can

provide the necessary resources to tackle them. Several examples of such areas are provided

below (on Antimicrobial resistance) and in Annex 1 (EAV case studies). These examples

highlight the Framework Programme response to European (and global) policy challenges

(e.g. Antimicrobial Resistance, Climate Change) whose resolution is increasingly dependent

on the establishment of a common scientific base leading to harmonised laws and standards

that can support innovation.

Box: European Added Value Case Study – The fight against antimicrobial

resistance

Antimicrobial Resistance (AMR) is the ability of microorganisms to resist antimicrobial drugs.

Various pathogens, including bacteria, viruses, fungi and parasites can evolve to be resistant to antimicrobial drugs due to gene mutations over time. Excessive and inappropriate use of antimicrobial medicines on humans and animals, and poor infection control practices, are both speeding up the evolution of resistant strains of microbes

and transforming AMR into a worldwide public health threat. A subset of multidrug-resistant bacteria in

Europe are responsible for about 25 000 of human deaths annually. 42

In addition to the avoidable deaths, this also translates into extra healthcare costs and productivity losses of at least EUR 1.5 billion each year. In 2007, infections caused by antibiotic-resistant bacteria resulted in approximately 2.5 million extra hospital days, which translated into EUR 900 million hospital costs. According to a report

commissioned by the UK Government in collaboration with the Wellcome Trust, 700 000 people die of resistant

infections every year. 43

In order to tackle Antimicrobial Resistance, the EU employed a "One Health" approach and also initiated coordination efforts between countries and international organisations. In 2011 the Commission adopted an action

plan against the rising threats of Antimicrobial Resistance. 44 Through its research framework programmes (e.g.

FP7, Horizon 2020) the Commission contributed to several of these areas by funding research activities in the fields related to antimicrobial resistance.

Research projects directly or indirectly related to Antimicrobial Resistance were conducted under different

themes, including Health, Nanosciences, Nanotechnologies, Materials & New Production Technologies (NMP), Knowledge Based Bioeconomy (KBBE), Information and communication technologies (ICT) and others.

40 Technopolis Group, Empirica, European Added Value of EU Science, Technology and Innovation actions and EU-

Member State Partnership in international cooperation, Report for the European Commission, 2014

41 http://www.oecd.org/sti/Science-briefscoreboard.pdf “Outflows tend to be associated with higher rated publications than their staying or returning counterparts. Assuming one could raise the performance of “stayers” to the level of their internationally mobile researchers […] this would help countries catch up with leading research nations.”

42 EMEA and ECDC Joint Technical Report. The bacterial challenge: time to react. 2009.

43 Tackling drug-resistant infections globally: final report and recommendations. The review on Antimicrobial resistance

chaired by Jim O‘Neill. (2016).

44 Communication from the Commission to the European Parliament and the Council - Action plan against the rising threats

from Antimicrobial Resistance. COM (2011) 748 final i.

In order to promote adequate use of antimicrobial drugs, the Commission launched in 2015 a EUR 1 million

challenge prize to develop a rapid diagnostic test for upper respiratory tract infections that can be safely treated without antibiotics. The prize was awarded to MINICARE HNL for a finger prick test that can diagnose in less than ten minutes a bacterial infection and identify if a patient can be treated safely without antibiotics.

To foster the engagement of industry in antibiotic research, several Antimicrobial Resistance related projects

were launched under the Innovative Medicines Initiative. The Innovative Medicines Initiative was launched in 2008 and is currently one of the largest public-private partnership between the EU and the European Federation of Pharmaceutical Industries and Associations. Overall, the EU has contributed more than EUR 1 billion towards combating Antimicrobial Resistance over the years.

A direct outcome of the R&I trans-national networks built as a result of participating in the

FPs is the trans-national co-publication of research articles. A study carried out by Elsevier 45

observed similar patterns when comparing FP7 and Horizon 2020 co-publication networks,

despite the lower number of publications at this early stage of Horizon 2020. The following

figures displays the intra-European and international co-publications networks under FP7 and

Horizon 2020.

Figure 14 Co-publication networks in FP7 and Horizon 2020 – inside the EU and internationally

FP7 Horizon 2020

l na

a ti

o

rn te In

U -2

8

a -E

Intr

Node colour is determined algorithmically to designate clusters. Nodes that have similar collaborations and

volume of collaborations have the same colour. Node size is number of publications. Edge thickness is number of collaboration publications between entities. Source: Scopus.

45 Elsevier, Study of FP7 and Horizon 2020 publications (forthcoming), see details in Annex 1.

From this analysis, in terms of intra-EU 28 collaboration as reflected by co-publications, the most frequent collaborations occurred between the larger and more R&D intensive countries. Collaboration frequencies are highest between these countries, but the countries with smaller R&I domestic ecosystems collaborate often with each other and with at least one of the R&D intensive nations. Germany, the Netherlands and the UK continue to collaborate largely with each other, as was observed in FP7, however in Horizon 2020 Belgium and France also joined this trend. Spain and Italy remain part of their own group but are now collaborating more with smaller Member States (compared to FP7), including Cyprus, Romania, Croatia and Greece. While the Nordics and Ireland formed their own group under FP7, they now collaborate more with the eastern European countries.

Another key aspect of EAV concerns the concept of project additionality - i.e. the capacity of the project beneficiaries to carry out the same or very similar projects without EU funding.

The underlying finding of a recent external study 46 is that EU FPs fund distinctive projects

which are unlike the projects funded at national or regional level. More than 4 in 5 Horizon 2020 projects (83%) would not have gone ahead without Horizon 2020 funding, and particularly in Research Infrastructures (100%) Space (95%) and FET (95%). On average, only around 14% of Horizon 2020 projects would have gone ahead without EU funding; the actual

size of this potential crowding-out is likely to be even lower 47 . A disaggregation by participant

type of the survey data is not possible, but indicatively, in the LEIT and FTI parts (where private sector participation is high) additionality would be even higher (92% of projects would have gone ahead only with significant changes or not at all). Overall, this points to a high additionality of the EU FPs, which results from the distinctive characteristics of EU-funded research projects.

Figure 15 Continuity of Horizon 2020 projects had they not received EU funding.

The project would The project would The project have gone ahead have gone ahead would not with none or miwith significant have gone

nor modifications. modifications. ahead.

Excellent Science

Future and emerging technologies 4,9% 29,0% 66,1% Research Infrastructures 0,0% 29,7% 70,4%

Industrial leadership

NMPB 12,9% 35,8% 51,3% Subtotal within NMPB: PPP projects 19,0% 81,0% 100,0% Information and Communication Technologies 19,5% 30,0% 50,5% Space 5,6% 27,6% 66,8% Innovation in SMEs 13,4% 16,2% 70,4%

46 PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon

2020)”, forthcoming. Several novel quantifications of EAV are drawn from this study, which performed a counterfactual analysis (based on a regression discontinuity with propensity matching) of FP7 top-scoring applicants who happened to be just above (intervention group) and below (control group) the funding threshold. This FP7 evidence is corroborated with a survey of Horizon 2020 beneficiaries (for which it was not possible yet to carry out the same design analysis) and in-depth case studies of EAV.

47 Idea Consult (2009), Assessing the behavioural additionality of the Sixth Framework Programme, European Commission,

Brussels; or PPMI (2013), Interim evaluation of FP7 Marie Curie Actions, European Commission, Brussels

The project would The project would The project have gone ahead have gone ahead would not with none or miwith significant have gone

nor modifications. modifications. ahead.

Societal Challenges

Societal Challenge 1 12,3% 39,6% 48,0% Societal Challenge 2 25,6% 32,9% 41,5% Societal Challenge 3 14,9% 30,6% 54,5% Societal Challenge 4 7,3% 43,7% 49,0% Societal Challenge 5 17,7% 39,5% 42,8% Societal Challenge 6 6,7% 29,2% 64,1% Societal Challenge 7 11,2% 33,4% 55,3%

Spreading Excellence and Widening participation + Science with and for Society + other programmes

Spreading Excellence and Widening Participation 8,7% 25,0% 66,4% Science with and for Society 8,4% 29,7% 61,8% Fast Track to Innovation Pilot 20,0% 60,0% 20,0% Euratom 0,0% 23,0% 77,0%

Total 13,7% 33,2% 53,2%

Source: PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon 2020)”, forthcoming

Both the Horizon 2020 and FP7 surveys provide consistent evidence that the FPs did not duplicate national R&D efforts and supported distinctive research activities. The lack of alternative funding for the type of activities funded in Horizon 2020 projects (92%) was mentioned as a key reason for not going ahead with their projects had Horizon 2020 beneficiaries not received EU funding.

Very similar data were obtained in the survey of unsuccessful FP7 applicants, where the lack of similar national or regional funds meant that 4 in 5 FP7 applicants who did not receive EU funding had to cancel their projects.

Even though in some analysed EAV areas (e.g. anti-microbial resistance, Fuel Cell research) some Member States funded similar research activities, the national projects were in most cases less ambitious in size and scope. Importantly, EU funding opened avenues for crosscountry research and data collection, leading to faster and better quality research results and impacts. The area of large-scale data gathering, omics research and biobanks is an exemplary area where the research performed greatly benefitted from the collection and analysis of cross-country patient cohort data coupled with a large quantity of omics, clinical, lifestyle and imaging information. The multi-centre and inter-disciplinary approach practised in EU-funded research projects strongly contributed to the development of personalised medicine approaches.

Following the counterfactual analysis, the PPMI study established that the EU FPs teams had, on average, 13.3 collaborations versus six collaborations in the control group. The beneficiary teams also built almost two times more collaborations with partners from outside the EU (on average, 3.6 partners from third countries versus 2.1 partners in the control group). Overall, these data point to the substantial structuring effect of the EU FPs and provide a quantification for the additionally built collaborations both across the EU and outside of it. Consistent results were found among Horizon 2020 beneficiaries.

Figure 16 Number of partners from other EU28 countries and third countries with which the analysed teams collaborated in 2015

Source: PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon 2020)”, forthcoming

The underlying trend is that there would be a significant decrease in the research capacities, particularly with respect to the transfer of knowledge (63%) and collaboration with industry and business (70%).

On average, a researcher produced 1.9 publications in 2015 in both the FP and control

groups 48 . However, the publications produced in FP7 projects were published in higher impact journals (average SJR 49 of 2.4) than non-FP publications (average SJR of 1.9) published

by the same authors who participated in EU-funded projects during 2007-2015. Based on the PPMI estimates, the SSH (estimated difference of 115%), Energy (56%) and Health (52%) programmes produced the largest positive difference in SJR values. The substantial difference in SJR values shows the benefits of the networking opportunities created in FP- funded projects to gain access and exposure to higher scientific impact.

Figure 17 Share of the project and consortium partners for whom their research capacity outputs would have decreased had they received national/regional instead of Horizon 2020 funding

Source: PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon 2020)”, forthcoming

48 This finding (i.e. no concrete evidence of the research teams becoming more productive or economical because of their

participation in the FPs) is similar to assessments done elsewhere large-scale international research programmes, notably for the NIH (National Institutes of Health in the USA): Brian A. Jacob, Lars Lefgren, Corrigendum to “The impact of NIH postdoctoral training grants on scientific productivity” [Res. Policy 40 (2011) 864–874], Research Policy, Volume 41, Issue 2, March 2012, Page 497

49 SCImago Journal Rank (SJR indicator) is a measure of scientific influence of scholarly journals that accounts for both the

number of citations received by a journal and the importance or prestige of the journals where such citations come from

Moreover, as regards wider availability and dissemination of knowledge between sectors, Horizon 2020 seems to be more effective than similar national or regional research support schemes, as shown by two thirds (66%) of Horizon 2020 coordinators who answered that transfer of technology and knowledge as an output of their project would have decreased if their projects had been funded by national/regional programmes.

The analysis showed that the distinctive research activities and the better results subsequently lead to better addressed pan-European/societal challenges. Around 3 in 4 Horizon 2020 project coordinators thought that the capacity to address the needs of EU citizens (74%) and tackle global challenges (73%) would have decreased if the project had been funded with national or regional funds instead of Horizon 2020. A similar share (71%) thought that their Horizon 2020 projects addressed pan-European issues that could not be addressed solely at national level. The beneficiaries of large projects exceeding EUR 5 million in budget size were particularly likely to report this finding, which suggests that sufficient scale of the research activities was a key factor in addressing pan-European challenges.

Figure 18 Share of research units that produced outputs in open access

Source: PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon 2020)”, forthcoming

The study also found that the EU FPs helped achieve results faster in almost half of the projects (i.e. 45%). In some analysed EAV areas such as food waste, the Horizon 2020 and FP7 allowed research results to reach the market 3-5+ years faster than national projects. This suggests that time reduction was a universally perceived impact of Horizon 2020 across different types of beneficiaries and projects. Regarding the wider impact of European projects, an especially large proportion of Horizon 2020 projects are expected to have an effect on ‘Climate action, environment, resource efficiency and raw materials’ (51%) and ‘Health, demographic change and wellbeing’ (47%), followed closely by ‘Secure, Clean and Efficient Energy‘ (42%).

Horizon 2020 primarily brought about benefits by improving the beneficiaries’ competitive position internationally (78% expected a decrease in this area and access to new markets (71%). Revenue would have decreased for 58% of survey respondents had their projects been implemented at national level. Overall, this evidence points to the international/intra-national dimension of Horizon 2020 and the commercial advantages this programme aspect brings as opposed to national or regional research activities. The Horizon 2020 and FP7 survey findings consistently show that the EU FPs were substantially more effective in producing economic and innovation outputs, and particularly large-scale demonstration initiatives, prototypes/testing activities, new/improved commercial products, business models and IPR.

Lastly, the improvement of the quality of R&I through exposure to EU-wide competition is another important element of EU added value. This is evident in the individual thematic assessments (Annex 2), notably in those where mono-beneficiaries are possible, like the SME Instrument and the ERC. The EU added value of the ERC from its exclusive focus on excellence-has been proven beyond doubt; it has become a global beacon of excellence; the number of ERC-grantees hosted by academic institutions is now a badge of honour comparable to Nobel prizes or Fields medals. An in-depth evaluation study of the SME Instrument carried out by Technopolis positively assessed it’s EU Added Value; it is unique compared to similar support schemes at national/regional level (which are only focusing on certain priority domains; do not have rolling submissions; have significantly smaller project volumes; require project collaboration with other SMEs or universities). Moreover, the EU Added Value at individual project level is assessed in the evaluation process.

Figure 19 Share of the project and consortium partners to whom their commercial advantage would have decreased had they received national/regional instead of Horizon 2020 funding

Source: PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon 2020)”, forthcoming

The issue of the EAV of R&I is complex, and different views are taken by stakeholders, EU institutions, EU Member States. For example, the European Economic and Social Committee “believes that trans-national cooperation between academia, industry, SMEs and research organisations is the main added value of Horizon 2020. The EESC believes that this transnational collaboration and networking is more important than the absolute amount of funding”

50 . Similarly, the report of the High Level Group chaired by M. Monti 51 found research

and innovation to be one of the two areas consensually identified as having a high potential added value (together with internal and external security). The report also noted “that EU research and development accounts for a much more modest share of the EU budget than agriculture and cohesion policies. In a global context where EU research is compared to American, Indian or Chinese research, this should be one of the essential policy priorities in the future.”

In the public stakeholder consultation, 62.5% (2,176) of the respondents rate higher the added value of Horizon 2020 compared to national and/or regional programmes for research and in

50 EESC information report INT/807

51 High Level Group Own Resources report, http://ec.europa.eu/budget/mff/hlgor/library/reports-communication/hlgor

href="http://ec.europa.eu/budget/mff/hlgor/library/reports-communication/hlgor-report_20170104.pdf">report_20170104.pdf

novation (see below). Research organisations and business respondents are the group with the highest percentage of respondents agreeing that the programme to be of higher value (respectively 66 and 65%), while public authorities are the group with the lowest percentage (56%).

Furthermore, out of the 835 respondents who did not participate in Horizon 2020, a rather low number prefer participating in other regional/ national programme (63 respondents) or in other European or international programmes (30 respondents). The consultation results also show that cooperation with partners from other countries is the main added value for respondents that participated or are expecting to participate.

Figure 20 How do you rate the overall added value of Horizon 2020 compared to national and/or regional level research and innovation programmes in EU Member States?

Source: Replies to stakeholder consultation questionnaire launched in the framework of the Interim Evaluation of Horizon 2020, October 2016-January 2017, N=3483

In terms of effectiveness, the respondents strongly agree with statements suggesting that Horizon 2020 strengthened the quality and visibility of research in the EU. For 1908 respondents, it contributes to improve international visibility and 1,357 are confident it improves excellence in research and innovation. In their open comments, respondents also outline the visibility and reputation they gain from being selected. Horizon 2020 is qualified as a “prestigious”

programme that set high standards for R&I in Europe and could lead to career

development or help organisations attract top researchers.

In terms of efficiency, for 1,076 (31%) stakeholder con I see in Horizon 2020 an added value in potentialsultation

 respondents, the programme strengthens critily increasing the relevance of social sciences. The

cal mass to address pan-European challenges. In their effort to embedding social science research into specific challenges forces us - social scientists - to

open comments, respondents go as far as saying that put in place a dialogue with other disciplines -

Horizon 2020 promotes trust between partners and a both inside and outside social sciences - and to

more coherent and integrated Europe through shared critically reflect on the impact the social sciences might have, and should have, in making this

goals and joint work. 1,574 respondents highlight that it world a better place to stay.

finances projects which otherwise could not be support Italy, University Bologna, E. Mollana

ed at national or regional level. European funding is all

the more important that the reimbursement of costs is higher than national/regional programmes

for 788 respondents. In open responses, some respondents also outline that 100%

cost funding for SMEs is a main incentive to participate (although it should also be noted that

a few comments are against full reimbursement).

In terms of synergy, Horizon 2020 is said to have contributed to strengthen interdisciplinary

cooperation (by 1,147 respondents, 33%) as well as cooperation between academia and the

private sector (873 respondents, 25%). Additional comments provided by respondents suggest

that the programme offers opportunities (qualified by some respondents as “unique opportunities”) to access new partners and new expertise, to work with the best and internationalise

their activities. It promotes a more integrated vision of the research and innovation system,

one that links together business, academy, industry and SMEs.

Working with different types of organisations and across dif Horizon 2020 is a big sandpit. If

ferent countries fosters cross-cultural experiences (to the beneyou are lucky you find toys and somebody to play and spend a

fits of young researchers more particularly), thus encouraging great time with you. If you fail you

the confrontation of different points of views, stimulating ideas […] only watch all others play. and fostering creativity and the emergence of disruptive ideas. Germany, Emschergenossenschaft

To provide a further analysis of the programme added value and additionality, stakeholder

consultation’s respondents were asked what would be the impact if the EU support to research

and innovation (Horizon 2020 and its possible successor) were to be discontinued. Very few

of the respondents judge that a discontinuation of the framework programme would only have

a limited impact on their organisation and most of the ones who do are NGOs and public authorities

(a few businesses, very few academics). Overall, the discontinuation of the programme

would be judged as “catastrophic”, “devastating” “a nightmare”, or a significant “drawback”.

Potential negative impacts are numerous and vary based on the dependence of the organisation

to Horizon 2020 funding. The impacts are worst for businesses whose activities are very

much dependent on EU funding – the programme’s discontinuation would result in a reduction

in scope or even in a stop to research and innovation activities, slower product development

and reduced business activities).

For academia and research organisations, it would mean: less funding for fundamental, interdisciplinary,

risky and disruptive research; less drive to cooperate; less international contacts;

less exposure to new knowledge; and more limited capacity to anticipate new trends – in

short, losing the ability to create critical mass at the European level. It will lead to the disappearances

of existing network since a stable framework would no longer be available to support

joint work.

Ultimately, since without an EU Framework Programme for R&I most of the strategically important research and innovation actions would simply not take place or be far less ambitious, discontinuation of Horizon 2020 will be a drawback for research and innovation in the EU, affecting the ability of Europeans to carry out top research and to address global challenges, thus resulting in a loss of competitiveness, loss of social, environmental and economic and EU policy impacts and loss of international visibility of the EU on the international research and innovation stage.

The economic costs of discontinuation can be considered to be the foregone growth and employment benefits of Horizon 2020. These are detailed in section Error! Reference source not found.: over EUR 27 billion per year until 2030 (or a lost cumulative GDP of over EUR 400 billion).

10.2. Key conclusions on the EU added value of Horizon 2020

Horizon 2020 produces demonstrable benefits compared to national and regional-level R&I support in terms of scale, speed and scope, notably through the creation of trans-national, multidisciplinary networks; pooling resources and creating critical mass to tackle global challenges. It thus increases the EU's attractiveness as a place to carry out research. Stakeholders find that Horizon 2020 has higher added value than other programmes.

The programme's additionality (i.e. not displacing or replacing national funding) is very strong (83% of projects would not have gone ahead without Horizon 2020 funding). The strong and direct pan-European competition guarantees the EU added value of single beneficiary programme parts, like the SME Instrument and the European Research Council. The latter is now a beacon of scientific excellence across the world.

Stakeholders feel that a possible discontinuation of the programme would have strong negative impacts, which would extend far beyond a simple reduction of R&I funding for their organisations. Costs of discontinuation (foregone economic benefits) are estimated to be over EUR 400 billion until 2030.

• 11. 

  IMPACT OF PREVIOUS FRAMEWORK PROGRAMMES

  11.1. Results from FP7

The Ex-Post Evaluation of FP7 52 was published in 2016 - approximately two years after the

end of the programme and after the start of Horizon 2020.

Figure 21 presents updated data on the state-of-play of FP7.

Excellence was one of the overarching goals of FP7. Figure 22 shows the average number of citations per publication. Publications funded in FP7 are more often cited than Member States publications. On average, EU funded FP7 publications are cited 21.4 times per publication, 7 times more than the Netherlands, 12 times more than the EU average number of citations per publications, and also higher than the world, United States and Japan's averages. Further findings on FP7 publications are:

52 https://ec.europa.eu/research/evaluations/index_en.cfm

 53 A higher impact (Field-Weighted Citation Impact) compared to that for all Member

States, the USA and Japan.

 Strong support for international collaboration defined as international co-authorship in

publications, which resulted in significantly more publications co-authored compared to those at international level (54.5%) as well as EU and world averages (34.4% and

17.3% respectively).

 A high score in terms of share of academic-private sector publications, which indicates

publications with both academic and corporate affiliations. FP7 funded publications have a 3.9% share of publications that are co-authored, which is higher than the EU

(2.2%), US (3.2%) and world averages (1.7%). 54

Figure 21 State of play on outputs from FP7 projects

FP7 output

Signed Grants in FP7 25 289

Finalised projects 55 15 612 Publications 56 207 501 Share of open access publications 57 61.8% Patent applications 58 2 669 Commercial use of R&D results 59 10 260

Source: eCORDA, 21/12/2016

According to an external study on EAV, FP7 funded research units tended to grow 11.8 percentage points (p.p.) more than similar non-funded units. This implies an increase of at least 40000-94,000 researchers due to the implementation of FP7. Similarly, FP7 had a positive effect on the R&D budget of high-quality European research units. The R&D budget of FP7 beneficiaries tended to grow around 24.6 p.p. more than similar non-funded units (i.e. a high leverage effect). An estimated 8-17.5 billion euros of R&D funding were attracted to European high quality research units because of FP7. FP funding also helped to attract more private R&D funding into EU-13 research organisations.

FP7 funding increased research collaborations of funded organisations with research teams in other EU countries by 120%. FPs also increased collaborations of beneficiaries with non-EU based research units by of 60% (at least 100,000 collaborations).

At least 7000-15,500 European researchers would have moved from one EU country to another as a direct consequence of FP7.

53 It divides the number of citations received by a publication by the average number of citations received by publications in

the same field, of the same type, and published in the same year, thus adjusting it for field and year.

54 To see all details please see Horizon 2020 Annual Monitoring Report 2015, pp 66-68

55 13341 from Sesam-Respir extraction date 22/2/2017 and 992 from ERC and 1279 from DG CONNECT.

56 The number of publications and open access rate is calculated using OpenAire on FP7 publications:

https://www.openaire.eu/fp7-stats extraction date 21/12/2016

57 The number of publications and open access rate is calculated using OpenAire on FP7 publications:

https://www.openaire.eu/fp7-stats extraction date 21/12/2016

58 Excluding ERC, 2374 from Sesam-Respir 22/2/2017 and 295 from DG CONNECT.

59 Excluding ERC, 9006 from Sesam-Respir extracted on 22/2/2017 and 1 254 from DG CONNECT.

Figure 22 Citations per FP7 publication, average (2007-2016)

Source: SciVal based on Corda-Sesam-Respir data, 9/8/2016

11.2. Longer term impact of previous Framework Programmes

As for Horizon 2020, the longer-term impact of FP7 60 was simulated using the Nemesis macro-economic

model in order to estimate economic impacts, in particular in terms of GDP and job growth, compared to a reference scenario in which FP7 was not implemented. Similarly, the economic impacts of FP7 are non-linear and follow three main phases (Figure below):

 The first phase (maturation) from 2007 to 2016, where GDP gains (compared with the

reference scenario) are mainly due to the R&D investment flows and the private consumption favoured by FP7 investment whereas the external balance, penalised by the inflationary pressure and by the raise of the internal demand, is contributing negatively to EU GDP.

 During the “Innovation” phase (i.e. 2017-2023), the GDP gains are resulting from the

acceleration of the arrival of process and product innovations, with a peak in 2023

where the GDP increases by 0.25% compared to the reference scenario.

 Finally in the Obsolescence phase (i.e. 2024-2040), where, under the progressive obsolescence

 of the new innovations, the GDP gain declines progressively to reach an

increase of 0.09% in 2040 compared to the reference scenario.

On average, the GDP gain is estimated to amount to EUR 22.4 billion (in 2014 prices) per year during 2007-2023. Over the same period of 17 years, the total GDP gain is EUR 380 billion: each EUR of FP7 direct budget (EUR 42.6 billion in 2014 prices) brought an estimated

GDP increase of about EUR 9. 61

60 The analysis (PPMI, “Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes

(FP7, Horizon 2020)”, forthcoming) quantified also the effect of FP7 on the EU economy and employment, simulating FP7 socio-economic impacts up to 2040 compared to a situation in which the Framework Programme would have ceased in 2007, after the end of FP6. The estimations are based on similar assumptions as those used for Horizon 2020, except for the amount of financing which is based on historical contributions from 2007 to 2014 and the related direct leverage effect.

61 These estimates are based on a crowding-in factor of 0.64 (each EUR of Commission contribution leading to an additional

R&D expenditure of EUR 0.64 from other public and private actors) as observed in the real data extracted from CORDA (January 2017).

Figure 23 The impact of the FP7 funding on EU GDP (in % deviation from reference

scenario)

Source: NEMESIS model

In terms of employment, as in the Horizon 2020 simulations, two phases can be distinguished:

 In the first phase, up to 2016, the total employment at EU level was positively impacted

by the FP7 (job creation peaks at 217 000 units in 2013 compared to the reference

scenario in the same year) with a strong contribution at the beginning of the period

coming from the employment in R&D activities (up to 121 000). With the decline of

the FP7 funds after 2013, the total employment started falling, due to the inflationary

pressures of the first period.

 In the following phase, the total employment gains increase again as a result of new

innovations entering into the market. Job creation peaks in 2025 (with 249 000 more

jobs compared to the reference scenario) and declines progressively. On average, during

the period 2007-2023, the EU contribution through FP7 has increase the level of

employment by 123 000 units, including 42 000 in research.

Figure 24 Impact of the FP7 on total employment (difference in thousand from reference

scenario)

Source: NEMESIS model

• 12. 

  KEY FINDINGS AND ISSUES FOR FUTURE CONSIDERATION

The results of this interim evaluation will help improve the implementation of Horizon 2020

in its last Work Programme 2018 – 2020, to provide input to the report of the High Level Expert

Group on maximizing the impact of EU Research and Innovation programmes and to inform

the design of future Framework Programmes. This section summarizes the key findings

and outlines issues for future consideration.

12.1. Limitations of the interim evaluation exercise

Key findings

 Few projects are finalised to date, limiting the data availability on the programme's performance.

 Projects already started have not had the time yet to produce the full set of

outputs, results and impacts, in line with the usual and widely acknowledged long time

lags in research and innovation.

 Research and innovation programmes are notoriously difficult to evaluate because the

pathways to impact are not linear.

 It is difficult to capture all direct and indirect results and impacts of a comprehensive

programme like Horizon 2020, which operates in a multi-faceted policy context, raising

the challenge of the attribution of the changes observed.

 The Horizon 2020 interim evaluation has been hampered by data availability, measurability

and reliability challenges and by the lack of a clear pre-defined intervention logic.

 Most monitoring indicators are focussing on input and output, and not on results and (societal) impacts.

 There are monitoring gaps, including lack of data beyond the life-time of a project.

Areas for improvement

Short term:

 Make the calls more specific by clarifying how projects are expected to contribute to the objectives through their expected outputs, results, and impacts.

 Ensure the availability/reliability of data to monitor progress.

 Solve monitoring problems with the flagging of cross-cutting issues and understanding of certain notions, e.g. with the notion of “gender dimension in research content”.

 Track longer term impacts beyond the project’s life time.

Longer-term:

 Develop a clear intervention logic at the beginning of a new Programme, starting with the expected impacts (longer term) and link them to results (medium term) and output (short term).

 Set clear indicators that are understandable by the wider public and suitable to monitor in real time short-term output, results and longer-term impact on the economy and society.

 Ensure the availability of data on results and impacts beyond the project’s life time.

12.2. Relevance

Key findings

 Horizon 2020's original rationale for intervention and objectives remain valid also in light of the Juncker priorities and the implementation of the Sustainable Development Goals.

 Horizon 2020 has proven fairly flexible to respond to new emerging needs.

 Horizon 2020 is broadly in line with stakeholders’ needs and is attractive for newcomers.

 The strategic programming process improved the intelligence-base underpinning programming choices and helped better define the focus in line with stakeholder needs.

 Emerging priorities and new developments need to be scouted continuously and the right balance has to be found between being too prescriptive or not prescriptive enough.

 The 2-year programming is at times seen as too rigid to swiftly respond to emerging needs dictated by disruptive and counter-intuitive technologies and business models.

 The translation of high level challenges and objectives into specific calls and topics is not always clear.

 The involvement of civil society organisation remains low and there is a gap in society in understanding the benefits of publicly-funded research and overall room for improvement in bringing research closer to the general public.

Areas for improvement

Short term:

 Improve the transparency and understandability of the Work Programme through improved ‘impact’ statements.

 Better communicate on the projects’ results and their scientific, technological, economic and societal impacts to the citizen and the wider scientific community.

Longer-term:

 Find ways to further increase the flexibility of the Framework Programme through an appropriate balance between top-down and bottom-up approaches.

 Establish an impact-focused mission-oriented approach to deliver on the implementation of the Sustainable Development Goals.

 Reconsider the length of the programming cycle and stakeholder involvement processes (e.g. more inclusive and transparent).

 Involve end-users and citizens in co-designing the R&I agenda and co-create solutions, which should also stimulate user-driven innovation.

12.3. Efficiency

Key findings

 Based on macro-economic projections, Horizon 2020 is as cost-effective as FP7 and comparable to the expected cost-effectiveness of public spending in research.

 Compared to FP7, Horizon 2020's efficiency is positively influenced by the extensive externalisation of programme implementation to new management modes including Executive Agencies.

 Simplification reduced administrative burden for participants and led to large decreases in the time to grant.

 Current administrative expenditure is below the target and is particularly low for the executive agencies.

 The new funding model is attractive for stakeholders and did not led to a significant change in funding rates compared to FP7.

 Horizon 2020 suffers from underfunding resulting in large-scale oversubscription, much larger than under FP7, which constitutes a waste of resources for applicants and a loss of high quality research for Europe.

 The proposal evaluation process is generally highly regarded but some aspects such as the feedback to applicants could be improved.

 Despite the low success rates, and cost of proposal writing, the costs on stakeholders seem to be proportionate given the (expected) benefits of participation, which go beyond the financial contribution received.

 The balance in project size did not change significantly compared to FP7 and does not seem to have a negative impact on newcomers in the programme.

 Horizon 2020 funding reaches a wide range of stakeholders, including SMEs and a high share of newcomers, but is also rather concentrated.

 Horizon 2020 is open to world and has a broad international outreach but funding of participants from third countries has decreased compared to FP7.

 Horizon 2020 promotes intensive collaboration between different types of organisations, scientific disciplines and sectors.

Areas for improvement

Short term:

 Address the issue of oversubscription; e.g. by expanding the use of two-stage procedures and improving proposal evaluations (for example the quality of feedback provided to applicants) and expand the use of the Seal of Excellence.

 Reinforce international cooperation activities for the remainder of the Horizon 2020 Programme.

 Continue with the externalisation of the implementation of the Framework Programme.

 Aim for further simplification and reduction of administrative burden for participants (e.g. via piloting output-based funding).

 Maintain the balance in project size.

Longer-term:

 Pursue further simplification and efficiency gains, for instance by assessing certain aspects of the proposal evaluation process could be further improved.

 Seek alternative ways to increase participation of international partners.

12.4. Effectiveness

Key findings

 Horizon 2020 is on track towards achieving its general objective of building a society and economy based on knowledge and innovation - based on its early progress towards achieving scientific, economic and societal impact.

 Horizon 2020 is projected to produce large-scale economic impacts.

 Horizon 2020 makes an important contribution to the Commission’s policy on "Budget

for Results" 62 because investing R&I in one area is expected to generate multiple impacts

in various domains. All Horizon 2020 pillars are also expected to produce scientific, economic and societal impacts.

 Horizon 2020 is attracting the best universities, research organisations, researchers and many of the top “established” innovative companies but has not been able to reach out young and quickly growing innovative companies worldwide.

 Horizon 2020 builds cross-sectoral, inter-disciplinary, intra- and extra-European research and innovation networks.

 Horizon 2020 projects already produce numerous outputs like publications, patents, prototypes, new or improved products, processes and methods, including in domains of societal relevance with the potential to generate scientific breakthroughs.

 Technological, regulatory, standards, technical and access to finance, as well as lack of customer acceptance of new solutions may impede Horizon 2020's full effectiveness in terms of market uptake.

 Horizon 2020 is making progress, albeit slowly, on spreading of excellence and widening participation, with noticeable performance differences and heterogeneity among the EU-13 countries and across Horizon 2020 programme parts.

 Progress is made with respect to promoting gender equality under Horizon 2020 but data quality concerns remain.

 The expenditure targets for sustainable development and climate change are not achieved yet.

 Results are encouraging in terms of the integration of social sciences and humanities (SSH) in Horizon 2020, even if highly uneven across the programme.

62 http://ec.europa.eu/budget/budget4results/index_en.cfm

Areas for improvement

Short term:

 Identify and support in particular SMEs that are developing breakthrough technologies at the intersection of different sectors, and support companies to scale up rapidly at EU level in order to stimulate market-creating disruptive innovation.

 Further strengthen feedback from R&I projects to policy-making.

 Ensure gender balance in terms of representation on Horizon 2020 advisory groups and project evaluation panels.

 Step up efforts to reach the sustainable development and climate expenditure target by the end of Horizon 2020.

 Deepen and broaden the embedding of SSH across the Horizon 2020 work programmes (contributions from certain SSH disciplines are relatively well-represented, while others are hardly present at all).

 Continue progressing with making scientific publications and data it generates openly accessible to the wider scientific community and the public.

Longer-term:

 Better support market-creating disruptive innovation, e.g. by identifying and supporting companies, in particular SMEs, that are developing breakthrough innovations at the intersection of different sectors and technologies and supporting their scale-up at EU level.

 Pursue further reinforcement of the R&I systems of low-performing R&I countries through a better policy coordination at EU, national and regional level stimulating national reforms e.g. through the European Semester, the Policy Support Facility and smart specialisation strategies.

 Ensure a complementarity/a better connection between all types of funding instruments across the EU, in particular between grants and non-grants, to facilitate scaling up of young innovative firm.

 Engage future users in the agenda-setting and development of market-creating innovations.

 Focus investments in areas of strategic interest for the EU and that are relevant to society, where multiple impacts are expected, for example through focus areas.

 Better address the barriers to innovation (regulations, standards, access to finance, customer acceptance) and support the creation of the right framework conditions for full market-uptake, including by developing approaches to identify the dual-use potential of projects’ results.

12.5. Coherence

Key findings

 The integration of research and innovation, the three pillar structure, the challengebased approach, and the use of focus areas contribute to the internal coherence of Horizon 2020 compared to FP7.

 Outside the 'Excellent science' pillar, Horizon 2020 is increasingly focused on research and innovation at higher Technology Readiness Levels. This has to be ensured that this does not come at the expense of lower Technology Readiness Levels collaborative research, which is regarded as one key source of future breakthrough innovations in line with societal needs.

 The large number of European R&I funding instruments is difficult to understand and may lead to overlaps.

 Compared to FP7, efforts have already been made to increase the synergies between Horizon 2020 and other programmes, notably ESIF but these can be further strengthened.

 Further coherence with other EU funding programmes is hampered by the different intervention logics and complexity of the different funding and other rules such as State Aid rules.

 Horizon 2020 specifically aims to establish synergies with national programmes. Public-public partnerships are creating long lasting collaborations between funding agencies and capacity building benefits however do not seem to really influence the alignment of national strategies and policies.

Areas for improvement

Short term:

 Improve internal coherence further, for example through the use of a limited number of focus areas.

 Ensure an appropriate balance between fundamental research, applied research and innovation support across all pillars in line with societal needs.

Longer-term:

 Rationalise the R&I funding landscape.

 Strengthen coherence, by integrating different EU funding schemes/programmes with the same intervention logic and further harmonisation of rules for participation in EU funding programmes.

 The alignment of the programme with policy priorities and the challenge-based approach need to be strengthened further and the work programme fragmentation needs to be reduced in order to maximise the impact of the supported activities.

 Focus on enhancing synergies between the EU Framework Programme for research and innovation and other EU funding programmes by ensuring complementary intervention logics at the design stage.

 Ensure a coherent approach at EU level for policies supporting research, education and innovation.

12.6. EU added value

Key findings

 Horizon 2020 produces demonstrable benefits compared to national and regional-level support to R&I in terms of scale, speed and scope, notably through the creation of excellence through competition, the creation of international, trans-national, multidisciplinary networks; pooling of resources; creating a big leverage effect and creating critical mass to tackle global challenges.

 Horizon 2020 increases the EU's attractiveness as a place to carry out R&I.

 Horizon 2020 is seen as improving the competitive advantage of participants for example through international multi-disciplinary networks, the sharing of knowledge and technology transfer and access to new markets.

 The additionality of Horizon 2020 is very strong – support is given to fund distinctive projects, which are unlike those funded at national or regional level.

 The impacts of discontinuation are difficult to quantify, but are likely very large.

Areas for improvement

Longer-term:

 Consider an impact-focussed mission-oriented approach to continue to deliver on global challenges at a scale, speed and scope that adds value compared to what can be done at national or regional level.

Met de EU Monitor volgt u alle Europese dossiers die voor u van belang zijn en bent u op de hoogte van alles wat er speelt in die dossiers. Helaas kunnen wij geen nieuwe gebruikers aansluiten, deze dienst zal over enige tijd de werkzaamheden staken.

De EU Monitor is ook beschikbaar in het Engels.