COMMISSION STAFF WORKING DOCUMENT INTERIM EVALUATION of HORIZON 2020 ANNEX 2 - Hoofdinhoud

Council of the European Union

Brussels, 31 May 2017 (OR. en)

9787/17 ADD 15

RECH 211 COMPET 454 IND 142 MI 458 EDUC 272 TELECOM 148 ENER 255 ENV 551 REGIO 65 AGRI 289 TRANS 227 SAN 224

COVER NOTE

From: Secretary-General of the European Commission, signed by Mr Jordi AYET PUIGARNAU, Director

date of receipt: 29 May 2017

To: Mr Jeppe TRANHOLM-MIKKELSEN, Secretary-General of the Council of the European Union

No. Cion doc.: SWD(2017) 221 final - PART 16/16

Subject: COMMISSION STAFF WORKING DOCUMENT INTERIM EVALUATION of HORIZON 2020 ANNEX 2

Delegations will find attached document SWD(2017) 221 final - PART 16/16.

Encl.: SWD(2017) 221 final - PART 16/16

EUROPEAN COMMISSION

Brussels, 29.5.2017 SWD(2017) 221 final

PART 16/16

COMMISSION STAFF WORKING DOCUMENT

INTERIM EVALUATION

of

HORIZON 2020

ANNEX 2

{SWD(2017) 220 final} {SWD(2017) 222 final} N. C LIMATE ACTION ENVIRONMENT RESOURCE EFFICIENCY AND RAW MATERIALS

N.1. I NTRODUCTION

N.1.1. Context

This first section presents the intervention logic of Horizon 2020-Societal Challenge 5: “Climate action, environment, resource efficiency and raw materials” (Horizon 2020-SC5), based on the Ex Ante Impact Assessment and the legal texts, and compares it with FP7- Cooperation Theme: “Environment, including Climate Change” (FP7-Environment), its predecessor. It also explains the strategy that the Commission services are carrying-out to reach the objectives stated in the legal base.

N.1.2. Objectives and intervention logic

The specific objective of Horizon 2020-SC5 is defined in Annex I of the Horizon 2020 Regulation:

The specific objective is to achieve a resource- and water-efficient and climate change resilient economy and society, the protection and sustainable management of natural resources and ecosystems, and a sustainable supply and use of raw materials, in order to meet the needs of a growing global population within the sustainable limits of the planet's natural resources and ecosystems. Activities will contribute to increasing European competitiveness and raw materials security and to improving well-being, whilst assuring environmental integrity, resilience and sustainability with the aim of keeping average global warming below 2°C and enabling ecosystems and society to adapt to climate change and other environmental

changes.

The lines of activity of Horizon 2020-SC5 according to the Horizon 2020 Regulation are:

(a) Fighting and adapting to climate change; (b) Protecting the environment, sustainably managing natural resources, water, biodiversity and ecosystems; (c) Ensuring the sustainable supply of non-energy and non-agricultural raw materials; (d) Enabling the transition towards a green economy and society through eco-innovation; (e) Developing comprehensive and sustained global environmental observation and information systems; and (f) Cultural heritage.

Points (a), (b) and (c) imply primarily research activities for increasing scientific understanding and evidence-based knowledge on climate change, biodiversity and the availability of raw materials. They have to be complemented with innovation actions focused on addressing societal challenges in the medium to long term.

Point (f) focuses on methodologies and tools for preserving, in interaction with society, a cultural heritage which risks to be lost as a consequence of emerging environmental threats linked to climate change. These areas will require a combination of both research and innovation activities.

Points (d) and (e) concentrate primarily on innovation actions, building on the former Eco Innovation Programme and developing technologies and basic scientific infrastructure necessary for earth observation.

These activities are further detailed in Annex I of the Specific Programme. Table 191 presents them with more detail.

We can observe that the co-legislator puts a strong emphasis on the economic and financial components of environment, climate and raw materials. For example, when the Specific Programme defines the problems to address, it provides estimates on the costs of climate change (between 5% and 20% of the GDP depending on the scenarios), on the business opportunities related to natural resources (EUR 2 trillion by 2050), on the added value of

industries that depend on raw materials supply (EUR 1,000 billion and 30 million jobs 1 ) or on

the contribution of cultural heritage to growth and job creation. The narrative focuses on economy rather than on the intrinsic value of nature.

By comparison, the objective of the FP7-Environment was 2 :

Sustainable management of the environment and its resources through the advancement of knowledge on the interaction between the climate, biosphere, ecosystems and human activities, and the development of new technologies, tools and services, in order to address global environmental issues in an integrated way. Emphasis will be placed on prediction of climate, ecological, earth and ocean systems changes, on tools and technologies for monitoring, prevention, mitigation and adaptation of environmental pressures and risks, including risks to health, and on tools and technologies for the sustainability of the natural

and man-made environment.

The terms used for FP7 suggest a less marked challenge-driven and solution-oriented than in Horizon 2020. FP7-Environment focused on the “advancement of knowledge” or “development of new technologies, tools and services”, while Horizon 2020 stresses immediate action to “achieve a resource- and water-efficient and climate change resilient economy and society (…)”. Horizon 2020 explicitly sets the overarching objective that “activities will contribute to increasing European competitiveness and raw materials security and to improving well-being”, which is in line with the general objective of the programme and priorities of Europe 2020.

The activities that FP7-Environment aimed to address were: (a) climate change, pollution and risks; (b) sustainable management of resources; (c) environmental technologies; and (d) earth observation and assessment tools. These domains are congruous with the areas of intervention of Horizon 2020-SC5.

“Raw materials” is the main new area included in Horizon 2020–SC5. Horizon 2020-SC5 reserves a dedicated part for R&I on the challenges related to the sustainable supply of nonenergy, non-agricultural raw materials, with a total expected funding of around EUR 600 million, i.e. around 20% of the budget of SC5. This represents a significant increase in both importance and funding compared to FP7 (with approximately EUR 180 million for raw materials mainly supported by the NMP Theme). The European Innovation Partnership (EIP) on Raw Materials is instrumental in securing and multiplying funding for the field of raw materials under Horizon 2020.

1 These are the data provided by the Specific Programme, which look over-estimates. More updated figures: The EU raw

materials industries in 2012 provided €280 billion of added value and more than four million jobs, while more than 11 million jobs are affected in the EU’s entire manufacturing sector. See: European Innovation Partnership on Raw Materials (2016) Raw Materials Scoreboard. Luxembourg: OPOCE. Indicator 7 'Value added and jobs.

2 FP7 Decision, Annex 1.

The following areas covered under FP7-Environment are not included under Horizon 2020– SC5:

 environment and health (which used to be under “Climate change, pollution and risks”) 3 , and

 management of marine environments (under “Sustainable management of resources”).

These activities are now addressed respectively within SC1 and SC2 in Horizon 2020.

Moreover, understanding and addressing natural hazards (e.g. earthquakes, floods) had a stronger emphasis under FP7-Environment than in Horizon 2020-SC5. “Technology assessment, verification and testing”, formerly under “Environmental technologies”, are not explicitly mentioned any more as areas of intervention, but could be part of actions aiming at increasing the understanding of environment-related phenomena.

Table 191 – From problems to actions (Horizon 2020’s Societal Challenge 5)

Line of activity Problem Objective Actions

Fighting and Emissions are leading Develop and assess Understanding of climate change and adapting to climate to climate change and cost-effective and reliable projection

change their economic cost sustainable

measures and Assessing impacts and developing coststrategies, effective adaptation measures

technological and Supporting mitigation measures (new

non-technological, climate-energy-economy models)

through generation of evidence for early and effective action

Protecting the Increasing pressures Provide knowledge Understanding of biodiversity and environment, lead to degradation of and tools for the ecosystem services and their interaction sustainably ecosystems, which management and with society

managing natural leads to losing business protection of

resources, water, opportunities related to natural resources Integrated approaches to address water

biodiversity and natural resources challenges and the transition to sustainable water management: tools,

ecosystems technologies and solutions.

Knowledge and tools for effective decision-making and public engagement: biodiversity and ecosystem services valuation; forecasting, early warning and resilience to risks

Ensuring the Very relevant Improve knowledge Improving knowledge base: assessment sustainable supply economic sectors rely base on raw of availability and more efficient use, reof non-energy and on access to raw materials and use of recycling. Develop global rules non-agricultural materials. Europe is develop innovative and standards.

raw materials dependent on imports solutions for costand

supply is not effective and Promoting sustainable supply and use of

guaranteed forever. sustainable raw materials: exploration, extraction, processing, re-use, recycling and

exploration,

extraction, recovery

Finding alternatives for critical raw

3 Nevertheless, the "impacts and growing risks for human health stemming from climate change, climate-induced hazards

and increased greenhouse gas concentrations in the atmosphere" as well as "the impact […] of environmental changes on human well-being" are covered by the Horizon 2020 Specific Programme for SC5.

Line of activity Problem Objective Actions

processing, re-use materials

and recovery Improving societal awareness and skills

on raw materials

Enabling the Decoupling growth and Foster all forms of Strengthening eco-innovative transition towards resources consumption eco-innovation to technologies, processes, services and a green economy requires structural enable transition to products to reduce quantity of raw

and society changes to safeguard a green economy materials in production and consumption. through ecoenvironment and Emphasis on transition to market

innovation increase European

competitiveness Supporting innovative policies and societal changes (new sustainable

lifestyles and consumption patterns)

Measuring and assessing progress towards a green economy: robust indicators and measurement methods beyond GDP, technology assessment methodologies and behavioural research.

Fostering resource efficiency through digital systems.

Developing Earth observation systems are necessary to ensure the delivery of long-term data, in order comprehensive to assess and predict the status and trends of nature and develop new global markets and sustained (climate services).

global Encourage free, open and unrestricted access to data.

environmental observation systems

Cultural heritage Cultural heritage is Provide knowledge Identify resilience levels via subject to deterioration an innovative observations, monitoring and modelling

and damage, due to solutions, through Better understanding on how

human activities and adaptation and

extreme weather mitigation, for communities perceive and respond to

events. This challenges preservation and climate change and seismic and volcanic

social cohesion, management of hazards

identity, well-being and cultural heritage at growth and jobs risk from climate creation change

Specific The EU action on environment and climate must be coordinated with international and implementation multilateral processes and initiatives, such as the International Panel on Climate Change

aspects (IPCC), the Intergovernmental Platform on Biodiversity and Ecosystem Services (IPBES), the Group on Earth Observation (GEO) or the Rio + 20 process.

Coordination also with European Innovation Partnerships (EIP), European Technology Platforms (ETPs), Copernicus or programmes like LIFE, ESI Funds and external

cooperation programmes.

Actions will provide continuous analysis of scientific and technological progress in the Union and partner countries, and an early investigation of market opportunities.

Source: European Commission services

Raw materials within Horizon 2020 - SC5

The raw materials part under Societal Challenge 5 aims at securing the supply of minerals and metals through sustainable innovative production technologies for primary and secondary raw materials.

The EU raw materials industries in 2012 provided EUR 280 billion of added value and more than four million jobs, and more than 11 million jobs are raw materials-dependent in the EU’s

entire manufacturing sector. 4 However, increasing demand at global level puts the raw

materials markets under pressure. Projections of future trends indicate that global resource use

could double between 2010 and 2030. 5 The EU is highly dependent on raw materials. They

are crucial for a strong European industrial base, an essential building block of growth and competitiveness. Transition from a linear economy to a circular one, or from an economy dependent on fossil-fuels to one based on renewable energies will require primary and secondary raw materials which are necessary to build the new infrastructures. Particularly, demand for many raw materials used in low-carbon energy technologies will double or even

triple by 2030. 6 The digitalisation of our economy, sine qua non condition to keep the

competitiveness of our industry and the potential of our internal market, may not happen without securing access to the necessary raw materials.

Nevertheless, the EU is confronted with a number of challenges along the entire raw materials value chain (i.e. from sustainable exploration, extraction, processing, recycling and after mining activities) to secure a sustainable access to non-energy non-agricultural raw materials used for industrial purposes. The EU addressed these challenges in the EU Raw Materials

policy and strategy called "Raw materials initiative" 7 , launched in 2008. Based on this

initiative, the European Innovation Partnership (EIP) on Raw Materials was launched in 2012. The EIP gathered many different players to develop its Strategic Implementation Plan (SIP), which is composed of a comprehensive set of actions under three pillars: technology, nontechnology and international cooperation. Additionally, the European Commission has created a list of Critical Raw Materials (CRMs). CRMs are those raw materials combining a high economic importance to the EU with a high risk associated with their supply. It should be noted that the second edition of the list of CRM published in 2014 increased to twenty

materials – which shows how raw materials supply make the EU’s economy vulnerable. 8

Horizon 2020 is expected to contribute to the implementation of both the Raw Materials policy and the Strategic Implementation Plan (SIP) of the European Innovation Partnership (EIP) on Raw Materials. These actions aim to maximise the positive impacts of the EIP on Raw Materials and achieve its targets, including innovative pilot actions, finding substitutes for critical raw materials, creating an innovation-friendly regulatory framework, and developing a proactive international cooperation strategy. The management of the raw materials part of Societal Challenge 5 is therefore very specific, since it is closely connected

with the EIP on Raw Materials and should contribute to its objectives. 9

The operational management of research projects and activities under Horizon 2020 must be complemented with actions from Member States, industry, academics, researchers and civil society in general. In this regard, the Commission has launched two Calls for Commitments

of the EIP on Raw Materials in 2013 and 2015. 10 A commitment is a joint undertaking by

4 EIP on Raw Materials (2016) op.cit., Indicator 7 'Value added and jobs

5 VVAA (2011) Decoupling natural resource use and environmental impacts from economic growth. A Report of the Working

Group on Decoupling to the International Resource Panel. UNEP.

6 EIP on Raw Materials (2016) op.cit., Introduction.

7 Communication from the European Commission (2008) The raw materials initiative — meeting our critical needs for growth and jobs in Europe, COM(2008)699 final i

8 The European Commission publishes a list of CRM, based on a methodology introduced in 2010. In 2011, 14 CRM were identified (see COM(2011) 25 final i, at: http://eur-lex.europa.eu/legalcontent/EN/TXT/PDF/?uri=CELEX:52011DC0025&from=EN). In 2014, the list increased to 20 (see COM(2014) 297 final i, at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52014DC0297&from=EN ).

9 For more information on the EIP on Raw Materials, see: https://ec.europa.eu/growth/tools-databases/eip-raw

href="https://ec.europa.eu/growth/tools-databases/eip-raw-materials/en/content/european-innovation-partnership-eip-raw-materials">materials/en/content/european-innovation-partnership-eip-raw-materials

10 https://ec.europa.eu/growth/tools-databases/eip-raw-materials/en/call-commitments

several partners, who engage to carry out activities that will contribute to achieving by 2020 the objectives set out in the SIP. At this moment, approximately 980 unique partners from very different sectors and from more than 50 different countries collaborate in 123 Raw Materials commitments, which cover the three-pillar structure of the EIP and have an indicative budget of close to EUR 2 billion. As will be seen later, the stakeholder communities of raw material commitments and Horizon 2020 projects on raw materials are closely linked.

Evolution of Horizon 2020’s context and objectives

On 15 July 2014, the then candidate for President of the European Commission, Jean-Claude Juncker, presented to the European Parliament a set of 10 political priorities: A New Start for

Europe: My Agenda for Jobs, Growth, Fairness and Democratic Change. 11 These priorities

significantly affect the implementation of Horizon 2020. Starting from the Work Programme 2016-2017, Horizon 2020 actions are designed in line with these objectives.

The Horizon 2020-SC5 mainly contributes to priority 1 (Growth, jobs and investments), 2 (Digital Single Market), 3 (Energy Union and climate change policy) and 9 (A Stronger Global Actor).

Other relevant events have been the publication of the Circular Economy package, the approval of the Sustainable Development Goals and the COP 21 Paris Agreement on climate change.

In December 2015, the Commission published its Communication Closing the loop – An EU

Action for the Circular Economy 12 . This document defends “the transition to a more circular

economy, where the value of products, materials and resources is maintained in the economy for as long as possible, and the generation of waste minimised, is an essential contribution to the EU's efforts to develop a sustainable, low carbon, resource efficient and competitive economy”. The Communication considers that Europe is losing opportunities due to low secondary materials use (with some exceptions like steel or paper). In particular, it alerts about the social and environmental damages that an unsustainable extraction of raw materials can produce. The role of Horizon 2020 is underlined to ensure such transition.

The year 2015 was a landmark for global environment and climate policy. It was marked by two extremely important international agreements: the UN's 2030 Agenda for Sustainable Development, with the related Sustainable Development Goals (SDGs), and the COP21 Paris Agreement on climate change. They will orientate future policy agendas all over the world, especially with regards to R&I, environment and climate.

N.1.3. Implementation logic

The European Commission services implement Horizon 2020 Societal Challenges mainly through calls for proposals outlined in multiannual Work Programmes (2014-2015, 2016- 2017, and 2018-2020 in preparation), which are prepared by following a lengthy consultation process (including an Expert Advisory Group, Member States and Associated Countries represented in Programme Committees, stakeholders, other Commission services, etc.).

11 https://ec.europa.eu/priorities/sites/beta-political/files/juncker-political-guidelines_en.pdf

12 COM(2015)614 final i, at: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52015DC0614

An analysis of the Work Programmes shows how Horizon 2020-SC5 has moved from traditional approaches, focusing on environmental sectors like waste and water, towards a systemic approach, understood as “innovation that aims at responding to a societal challenge by obtaining a system-wide transformation through affecting the system’s economic, social and environmental dimensions as well as their interconnections. This implies a transdisciplinary perspective that integrates technology, business models and economic organisation, finance, governance and regulation, as well as skills and social innovation. Systemic innovation therefore calls for the adoption of a challenge-driven, solutions-oriented research and innovation strategy that crosses disciplinary boundaries and involves co-creation of knowledge and co-delivery of outcomes with economic, industrial and research actors,

public authorities and/or civil society”. 13

Systemic innovation is presented as a pre-requisite of a transformative agenda focused on developing the solutions of the future which should in turn attract further public and private investments (i.e. the financial leverage stated in the general objective of Horizon 2020). From the Work Programme 2016-2017, "traditional" sectors like waste and water are mainstreamed under the various calls and topics, for instance in the cross-cutting focus area call on “Industry 2020 in the Circular Economy”.

The Work Programme 2016-2017 focuses on the following “future solutions”:

 Climate services, already introduced in the previous WP, are further developed with

demonstration, proof-of-concept and market research-related actions.

 Nature-based solutions, defined as “inspired and supported by nature (…) such as

well-connected green infrastructure, green and unsealed surfaces in cities, green roofs, natural water retention measures, and salt marshes and dunes of coastal protection [which] use the properties and functions of ecosystems to provide water regulation,

flood risk protection, climate change adaptation, etc.” 14 Nature-based solutions are

supported through large-scale demonstrations in cities and in rural and natural areas, and through a topic on the operationalization of the insurance value of ecosystems.

 Cultural heritage for sustainable growth, supported through large-scale demonstrations

as well as through research on innovative financing, business and governance models for adaptive re-use of cultural heritage.

 Systemic, eco-innovative approaches for the circular economy, supported through

large-scale demonstrators on value and supply chains, systemic services and water innovation, together with actions to develop new business models and economic

incentives for the circular economy.

 Implementation of both the Raw Materials policy and the EIP-SIP. Raw materials

contribute to a systemic change; they are enablers of many technologies such as advanced ICT or low carbon energy technologies.

Horizon 2020-SC5 contributes to several focus areas, such as “Blue Growth”; “Disasterresilience”; “Energy Efficiency”; “Sustainable Food Security”, “Competitive Low Carbon Energy”, “Industry 2020 in the Circular Economy” and “Smart and Sustainable Cities”. This is consistent with the cross-cutting nature of environmental and climate concerns.

13 Ibid, Part 12, p.5.

14 Ibid, Part 12, p.12.

Figure 242 illustrates the logic of the Work Programme 2016-2017 and Figure 243 shows the overall intervention logic of Horizon 2020-SC5 so far: rationale, as well as expected outputs, outcomes and impacts.

Figure 242 – Approach of the Work Programme 2016-2017

Source: European Commission services

Figure 243 – Intervention logic of Horizon 2020, Societal Challenge 5

Source: European Commission services

1306

N.2. I MPLEMENTATION STATE OF PLAY

This section shows how Horizon 2020-SC5 has been implemented until the 1 st January

2017, i.e. the Work Programmes 2014-2015 and 2016.

N.2.1. Overview of programme inputs and activities

The Work Programme 2014-2015 (as reviewed in October 2015) 15 foresaw a total budget

of EUR 725.83 million, on which EUR 348.26 million were allocated to 2014 and EUR

377.57 million to 2015. The Work Programme 2016-2017 16 foresee a total budget of

EUR 760 million, breakdowned as follows: EUR 358.25 million in 2016 and EUR 401.75 million in 2017. Of these amounts, around 3% correspond to European Free Trade Association (EFTA) contributions. The budget for the whole Horizon 2020-SC5 is

EUR 2,965.7 million for the period 2014-2020.

On 1 st January 2017, EUR 1,117.3 million have been allocated to selected projects,

corresponding to all 2014-2015 and 2016 calls (EUR 741.4 million and EUR 375.9

million respectively). These data include:

 Projects funded under the topics BG-14-2014 and LCE-2016-ERA, co-funded by

Horizon 2020-SC5 but managed by other Commission services (EC contribution:

13.4 million).

 The SME Instrument Phase 1, with EUR 1.7 million and EUR 1.9 million

foreseen in the Work Programmes 2014-2015 and 2016-2017 respectively.

The data do not include :

 “Ad-hoc” financing in the Work Programme 2014-2015 (e.g. support to the

IPCC, support to the implementation of the Cultural Heritage Strategic Research

Agenda, organisation of high-level Conferences, GEO subscriptions).

Table 192 presents how projects and budgets have been allocated to lines of activities. 17

Table 192 - Number of projects and budget breakdown by Horizon 2020-SC5 lines

of activities as defined in the legal base

Line of Description No. of Project Share Project Share of activity prorequested EC of total total total

jects contribution (%) costs (%)

5.1 Fighting and adapting to climate change

5.1.1 Improving the understanding of climate 6 50.0 4.5 52.4 3.7 change and the provision of reliable

climate projections

5.1.2 Assessing impacts and vulnerabilities 19 117.2 10.6 177.3 12.4 and developing innovative cost-effective

adaptation, risk prevention and managing measures

5.1.3 Supporting mitigation policies, including 9 33.5 3.0 35.0 2.4 studies that focus on impact from other

sectoral policies

15 European Commission Decision C (2015)7154 of 23 October 2015. 16 European Commission Decision C (2016)4614 of 25 July 2016. 17 The classification of topics by lines of activities is just an illustrative estimate. Topics use to respond to several lines

of activity, while they are allocated here to only one. The breakdown includes also a high degree of subjectivity.

Line of Description No. of Project Share Project Share of activity prorequested EC of total total total

jects contribution (%) costs (%)

5.2 Protecting the environment, sustainably managing natural resources, water, biodiversity and ecosystems

5.2.1 Furthering our understanding of 14 129.6 11.7 176.9 12.4 biodiversity and the functioning of

ecosystems, their interactions with social systems and their role in sustaining the economy and human well-being

5.2.2 Developing integrated approaches to 21 105.0 9.5 155.0 10.8 address water-related challenges and the

transition to sustainable management and use of water resources and services

5.2.3 Providing knowledge and tools for 12 43,1 3,9 65,7 4,6 effective decision making and public

engagement

5.3 Ensuring the sustainable supply of non-energy and non-agricultural raw materials

5.3.1 Improving the knowledge base on the 7 15.2 1.4 16.1 1.1 availability of raw materials

5.3.2 Promoting the sustainable supply and use 23 166.9 15.0 186.0 13.0 of raw materials, including mineral

resources, from land and sea, covering exploitation, extraction, processing, reuse, recycling and

5.3.3 Finding alternatives to raw materials 4 18.7 1.7 19.9 1.4

5.3.4 Improving societal awareness and skills 6 10.0 0.9 10.4 0.7 on raw materials

5.4 Enabling the transition towards a green economy and society through eco-innovation

5.4.1 Strengthening eco-innovative 66 252.1 22.7 312.7 21.8 technologies, processes, services and

products, including exploring ways to reduce the quantities of raw materials in production and consumption, overcoming barriers in this context and boosting their market uptake

5.4.2 Supporting innovative policies and 6 20.1 1.8 20.1 1.4 societal changes

5.4.3 Measuring and assessing progress 2 3.5 0.3 3.5 0.2 towards a green economy

5.4.4 Fostering resource efficiency through 6 7.1 0.6 8.2 0.6 digital systems

5.5. Developing comprehensive and sustained 13 102.8 9.3 149.2 10.4 global environmental observation and

information systems

5.6. Cultural heritage

5.6.1 Identifying resilience levels via 3 18.8 1.7 19.2 1.3 observations, monitoring and modelling

5.6.2 Providing for a better understanding on 1 9.9 0.9 10.6 0.7 how communities perceive and respond

to climate change and seismic and volcanic hazards

5.7. Specific implementation aspects 3 5.9 0.5 13.5 0.9

Total 221 1109.3 100 1431.5 100

Source: CORDA, extraction: 1/01/2017. SME Instrument Phase 1 excluded.

In January 2017, 221 projects have been selected; one of them was finalised (a SME

Instrument Phase 2), 195 were ongoing and 22 under contract preparation. In addition,

160 SME Instrument Phase 1 actions were launched, with only one finalised. The programme has so far been implemented through Research and Innovation Actions (RIA: 73 projects, 44.3% of the total funding), Innovation Actions (IA: 57 projects, 34.3%), Coordination and Support Actions (CSA: 45 projects, 7.9%), ERA-Net Cofund (9 actions, 7.8%), SME Instrument Phase 2 projects (37 projects, 5%) and SME Instrument

Phase 1 (160 actions, 0.7%). 18

The budget is being allocated through 94 topics 19 included in the 11 calls for proposals.

The following table summarises the current results of the evaluation of the proposals received for these topics.

Table 193 - Number of projects, EC funding, oversubscription rate and success rate per type of action (SC5)

Instrum No. of No. of Share Total EC Actual Share Over Success Averag ent eligible projects of contribut EC of total subscri rate e EC

proposal total ion contrib EC ption (%) contrib s projec requeste ution (€ contrib rate ution ts (%) d (€ million) ution per million) (%) project (€

million) CSA 179 45 20.4 359.2 88.6 7.9 24.7 25.1 1.97

ERA- 9 9 4.1 87.8 87.2 7.8 99.3 100 9.69 NET- Cofun d

IA 671 57 25.8 4531.2 383.1 34.3 8.5 8.5 6.72

RIA 749 73 33.0 4357.7 495.0 44.3 11.4 9.7 6.78

SME-2 875 37 16.7 1309.8 55.3 5.0 4.2 4.2 1.5

Total 1755 221 100 10645.7 1109.3 99.3 10.4 12.6 5.02

SME-1 2168 160 - 108.35 8 0.7 7.4 7.4 0.05

Source: CORDA, extraction: 1/01/2017.

One important conclusion from Table 195 is the extremely low success rate, especially for Innovation Actions, Research and Innovation Actions and the SME Instrument Phase

• 2. 

  Overall, for the FP7 Cooperation Theme “Environment (including Climate Change)”,

the success rate was 18.9%. 20

The implementation of Horizon 2020-SC5 by funding instrument is in line with the Work

Programmes, presented in Table 5 21 . The main difference so far concerns IAs, which are

expected to receive a higher share of EC contributions by 2017.

Table 5 also illustrates how the Work Programme 2016-2017 has evolved towards more

Innovative Actions, mainly large scale demonstrations in fields like Nature-Based

18 The scope of each type of action can be found in http://ec.europa.eu/research/participants/data/ref/Horizon

2020/legal_basis/rules_participation/Horizon 2020-rules-participation_en.pdf . For the SME instrument, please check https://ec.europa.eu/programmes/horizon2020/en/Horizon 2020-section/sme-instrument .

19 This figure includes sub-topics.

20 E-Corda database, extraction 8/08/2016.

21 The table does not include the contribution of SC5 to Fast-Track to Innovation (FTI) and other “ad-hoc” actions

like grants to named beneficiaries, administrative arrangements, public procurement or subscriptions (e.g. GEO).

CSAs include Coordinated and Support actions to prepare other financial instruments like pre-commercial procurement, prizes, etc.

Solutions or the Circular Economy. They represented around half of the SC5 budget for 2016-2017, compared with around one-quarter in 2014-2015. On 1st January 2017, 46 projects underway or under preparation were tagged as demonstrations. Altogether they comprise a total of EUR 348.2 million in financial contributions from the European

Commission, for a total project budget of EUR 413.3 million.

Table 194 - EC funding per type of action (SC5), according to the Work Programmes

Instrument EC contribution (million €) Share of total (%)

Work Programme 2014-2015

CSA 66.5 9.4

ERA-NET-Cofund 73 10.3

IA 188 26.6

RIA 344.5 48.7

SME Instrument Phases 1 and 2 36 5.1

Sub-total 708 100

Work Programme 2016-2017

CSA 34.0 4.6

ERA-NET-Cofund 49.0 6.6

IA 391.0 52.7

RIA 202.0 27.2

SME Instrument Phases 1 and 2 61.0 8.2

Pre-Commercial Procurement 5.0 0.7

Sub-total 742.0 100

Total: Work Programmes 2014-2017

CSA 100.5 6.9

ERA-NET-Cofund 122.0 8.4

IA 579.0 39.9

RIA 546.5 37.7

SME Instrument Phases 1 and 2 97.0 6.7

Pre-Commercial Procurement 5.0 0.3

Total 1,450.0 100.00

Source:European Commission services

The SME Instrument requires a different analysis. It is a novelty of Horizon 2020, defined in article 22 of the Horizon 2020 Regulation and further developed in the

Specific Programme (Annex I, Part II, Section 3). It includes three possible phases: (1) concept and feasibility assessment; (2) R&D, demonstration and market replication; and

• (3) 

  Commercialisation. The first phase provides a support of EUR 50,000 to assess “the scientific or technical feasibility and the commercial potential of a new idea (proof of concept) in order to develop and innovation project”. The SME Instrument Phase 1 supports very preliminary analysis of an idea. These projects are not R&I actions properly speaking, since they do not develop research or an innovation. Indeed the amount allocated to SME Instrument Phase 1 actions is very small compared with more traditional R&I projects. Table 4 presents SME Instrument Phase 1 actions separately to avoid bias.

Table 6 shows how the actual funding and projects supported through the SME

Instrument 22 . It shows that, in 2014 and to a lesser extent in 2015, the actual funding of

the SME Instrument exceeded the budget foreseen in the Work Programme. In 2014, the deviation was 13.5% (mainly coming from the Phase 1: 76.4%), while in 2015 it was

3.9% (23.7% for the SME Instrument Phase 1) 23 . Section 6.1 analyses these deviations.

Table 195 - EC funding and beneficiaries for the SME Instrument

Budget No. Of No. Of Success Actual EC Average foreseen eligible funded rate (%) contribution TTG

in WP proposals projects (€) (days) (€M)

2014

SME Instrument Phase 1 1.7 687 60 8.7 3,000,000 134.4

SME Instrument Phase 2 15.3 98 13 13.3 16,294,262 196.7

Sub-total 17 785 73 9.3 19,294,262

2015

SME Instrument Phase 1 1.9 787 47 6 2,350,000 85

SME Instrument Phase 2 17.1 315 11 3.5 17,393,394 165.1

Sub-total 19 1102 58 5.3 19,743,394

2016

SME Instrument Phase 1 2.5 693 53 7.6 2,650,000 84.4

SME Instrument Phase 2 22.5 462 13 2.8 21,648,718 138.1

Sub-total 25 1155 66 5.7 24,298,718

Total

SME Instrument Phase 1 6.1 2167 160 7.4 8,000,000 103.3

SME Instrument Phase 2 54.9 875 37 4.2 55,336,374 168.3

Grand total 61 3042 197 6.5 63,336,374

Source: CORDA, extraction: 1/1/2017.

The Error! Reference source not found. shows that, in 2014 and to a lesser extent in 2015, the actual funding of the SME Instrument exceeded the budget foreseen in the Work Programme. In 2014, the deviation was 13.5% (mainly coming from the Phase 1: 76.4%), while in 2015 it was 3.9% (23.7% for the SME Instrument Phase 1). Section N.6.1 analyses these deviations. Table 201 completes the data presented with total cost of projects and the time-to-grant (TTG).

Table 196 - Key data on signed grants per type of action: time-to-grant (TTG) and total cost

Instrument No. of Actual Total Ratio EC Average No. of % of projects EC projects contributi TTG grants contract contribu cost (€ on/Total (days) signed s signed tion (€ million) cost (%) within within

million) target target

CSA 45 88.6 93.1 95.2 238.7 40 88.9

ERA-NET- 9 87.2 294.7 29.6 242.2 8 88.9

Cofund

22 There are no deadlines to submit SME Instrument proposals. However they are evaluated during three or four

annual cut-off dates.

23 Annex 1 shows the details of the SME Instrument evaluations for SC5.

Instrument No. of Actual Total Ratio EC Average No. of % of projects EC projects contributi TTG grants contract contribu cost (€ on/Total (days) signed s signed tion (€ million) cost (%) within within

million) target target

IA 57 383.1 455.7 84.1 241.2 52 91.2

RIA 73 495 508.9 97.3 237.5 62 84.9

Total 184 1053.9 1352.4 77.9 238.9 162 88.0

SME-1 160 8 11.4 70.2 103.3 75 46.9

SME-2 37 55.3 79.1 69.9 168.3 21 56.8

Source: CORDA, extraction: 1/1/2017. The percentage of grants signed within targets does not include data not available. The targets are: 245 days for CSA, ERA-NET, IA and RIA; 90 for SME-1 and 180 for

SME-2.

The ratio EC contribution/Total project cost is so far similar than under the FP7 Cooperation Theme “Environment (including Climate Change)”, 77.4%. The Horizon 2020-SC5 figure would nevertheless be higher without the ERA-NET Cofund

Instrument, which presents a very low share of EC contribution (29.6%). 24

The time to grant indicator (TTG) shows a positive figure for CSA, ERA-NET, IA and RIA, with 88% of the grants signed within the objective of 245 days. However, a more detailed reading of Table 7 indicates that the average TTGs are very close to the target. Most contracts have been signed with a very short margin of manoeuvre. The objective of signing the grant agreements before 245 days is almost accomplished, but with almost no margin for further improvements… and with some perceived quality issues in section N.6. The situation is much more difficult for the SME Instrument, because TTG targets are tighter.

Only the raw materials part of Horizon 2020-SC5 mentions the expected Technology Readiness Levels (TRL), in all RIAs and IAs.

N.2.2. Participation patterns

N.2.2.1. Participation per type of organisation

Private for profit entities (33.9%) hold the highest percentage of total participations in Horizon 2020-SC5, followed by Research Organisations (25.5%) and Higher or secondary education institutions (20.3%). Public bodies receive 12.5% of the total EU funding, while “Others” (which in principle include Non-Governmental Organisations, NGOs) make up 7.8% out of the total.

Industry participation has significantly increased from FP7-Environment (19.4%). The category “Others” represented only 3% in the Horizon 2020-SC5 predecessor. The authors of the Ex Post Evaluation of FP7-Environment considered that those low figures

were challenging for a programme that aimed to move towards innovation. 25

These data suggest that Horizon 2020 is more open to new and more diversified categories of participants. Overall, 29.8% of individual beneficiaries are newcomers. This figure is higher amongst private for profit bodies: 58.9% (55% if the SME Phase 1 instrument is not counted).

24 A detailed compilation of the Horizon2020-SC5 projects can be found in:

http://ec.europa.eu/research/environment/pdf/research_and_innovation_sc5_projects_2014-2016.pdf

25 op.cit., p. 25

Interestingly, 742 (62.9%) private for profit participants are SMEs. 26 The figure is higher

than in FP7-Environment, even if the overall participation of enterprises was much lower.

Table 197 - Participation per type of organisation

No. of % Of No. of % of No. of Average EC % of partici which: newco newco individ particip contribut total

pants coordi mers mers ual ation per ion (€ fundi nators partici beneficia million) ng pants ry

HES: Higher 706 20.3 49 26 3.7 347 2.0 258.9 23.2 or Secondary

Education

Establishments

OTH: Other 270 7.8 7 132 48.9 196 1.4 57.0 5.1

PRC: Private 1178 33.9 228 694 58.9 1039 1.1 349.5 31.3 for Profit

Entities

(excluding

HES)

Of which: In 180 5.2 160 144 80.0 179 1.0 8.0 0.7 SME-1

PUB: Public 434 12.5 16 130 30.0 291 1.5 119.8 10.7 bodies

(excluding

REC and

HES)

REC: 886 25.5 81 52 5.9 366 2.4 332.2 29.7 Research organisation

Total 3474 100 381 1034 29.8 2239 1.6 1117.3 100

Source: CORDA, extraction: 1/1/2017.

Enterprises participate 1.1 times on average in the programme, i.e. less than academic institutions or even public bodies. The number of participations of single beneficiaries can be considered an indicator of the attractiveness of the programme: renewed participation may suggest that previous or ongoing experiences are positive, and

subsequently organisations submit further proposals. 27 In the case of private for profit

entities, the figure cannot be considered alarming yet, taking into account the very high rate of newcomers.

The main beneficiaries of Horizon 2020-SC5 (top-30) come from academia (research organisation or universities), the exceptions being public entities like the French Agence Nationale de la Recherche (ANR), the British Met Office and the Swedish Meteorologiska och Hydrologiska Institute and Forskningsrådet för Miljö, Areella Näringar och Samhällsbyggande. We observe major research organisations (e.g. the German Deutsches Zentrum Fuer Luft –und Raumfahrt EV, which is the main recipient of EU support so far, Fraunhofer or two Helmholtz Centers; the French CNRS, the Italian CNR; the Spanish Tecnalia; the Finnish VTT; the Flemish VTO or the British Natural Environment Research Council), as well as universities like the Technische Universiteit Delft, Exeter, Bergen, Lulea, the KU Leuven or the Universidad Autónoma de Barcelona.

26 There are 228 private for profit participants without any information about their size (SME/large).

27 Ibid., p.24.

In order to identify who are the enterprises supported by Horizon 2020-SC5 the data in CORDA have been merged with the ORBIS database in order to determine which are the dominant economic sectors of Horizon 2020-SC5 private for profit beneficiaries. The merging covers only EU enterprises and, overall, 37.3% of private for profit organisations do not have any NACE code. Excluding those “missing” companies, the main activity sector is “Professional, scientific and technical activities” (42.8% in terms of participations, 40.7% in terms of EC contribution), and more precisely “Engineering activities and related technical consultancy” (almost one-third of the sector’s participation) followed by “Other research and experimental development on natural sciences and engineering” and by “Business and other management consultancy activities”.

The second most frequent sector is “Manufacturing” (21.5% in terms of participations and 27% in terms of support received), led by “Manufacture of instruments and appliances for measuring, testing and navigation” and by “Manufacture of other generalpurpose machinery”. Participants cover a wide range of manufacturing sub-sectors (67).

The ICT sector represents around 10% of participations and of funding share. It is dominated by “Computer programming activities”, followed by consultancy and other ICT services.

In any case, there are no enterprises amongst the top 30 beneficiaries of Horizon 2020- SC5. Most of the top private for profit organisations are equipment manufacturers, but there are also mining, as well as engineering and consulting firms. Major players in environmental sectors, such as Acciona (which construction filial is ranked third), Veolia or Suez; from infrastructures sectors like Alstom or from aero-spatial like Airbus are also supported by Horizon 2020-SC5. It is relevant to underline the presence of industrial companies from both upstream sectors (such as mining companies, like Cobre las Cruces or Eramet) and downstream industrial sectors (e.g. equipment providers such as Soil Machine Dynamics or Atlas Copco), as requested in several raw material topics.

Table 198 shows that public organisations are those that perform better in terms of success rate (calculated as participations/applications), followed by “others”. Enterprises and higher and secondary education institutions present the lowest success rates. This figure does not include the SME Instrument.

Table 198 - Success rates per type of organisation for Horizon 2020-SC5

Success Rate Success Rate

Type of Applicant (No. of participations/ No. of (actual EC contribution/ Funding

Applications, %) requested by applicants, %) HES 15.8 9.8

OTH 28.3 19.1

PRC 13.6 9.2

PUB 33.5 24.8

REC 23.6 14.9

Total 18.5 11.9

Source: CORDA, extraction: 1/1/2017. SME Instrument Phase 1 excluded.

Private for profit organisations present the highest number of applications to Horizon 2020-SC5 calls (41.2%, far beyond universities with 25%). Their low success rate could be considered an issue: enterprises are the main actors of innovation but applying to a R&I programme cannot be considered as part of their core business. Low success rates may reduce the attractiveness of the programme for this kind of players. However private companies are the organisation category with more newcomers, which probably lack the experience and network that facilitates success in Horizon 2020 proposals.

N.2.2.2. Attraction of new participants / newcomers

There are 1,034 newcomers (29.8%), including the SME instrument Phase 1. Most newcomers are enterprises (58.9%), while the vast majority of universities and research centres involved in Horizon 2020-SC5 were already experienced with FP7. Their newcomer rates are equal to 3.7% and 5.9% respectively. The higher proportion of organisations that did not participate in FP7 appears amongst EU-13 and Third Countries (35%). EU-15 countries and Associated ones present similar figures (29.1% and 24.4%). It has to be noted that Switzerland was considered a Third Country until 2017.

N.2.2.3. Geographical participation

Table 191 presents the key data on geographical participation for SC5. It does not include the SME Instrument Phase 1.

Geographical distribution of EC contributions

The bulk of the EU funding goes to EU Member States (93.2%) and more concretely to EU-15 countries (87.4% of the total EC contributions). EU-13 and Associated countries receive similar shares (5.7% and 5.1% respectively) of the EC budget, while the EC contribution going to Third countries is merely symbolic (1.7%).

Table 199 - Participation by country

Country Acrony Participati % Total % Total requested % m ons costs (€ EC contribution

million) (€million)

Austria AT 90 2.7 32.5 2.3 26.4 2.4

Belgium BE 193 5.9 67.6 4.7 58.2 5.2

Bulgaria BG 13 0.4 2.4 0.2 1.5 0.1

Croatia HR 15 0.5 2.5 0.2 2.0 0.2

Cyprus CY 21 0.6 6.6 0.5 4.9 0.4

Czech Republic CZ 29 0.9 9.9 0.7 6.6 0.6

Denmark DK 74 2.2 34.2 2.4 26.2 2.4

Estonia EE 11 0.3 4.3 0.3 2.5 0.2

Finland FI 87 2.6 47.4 3.3 35.7 3.2

France FR 232 7.0 105.8 7.4 79.2 7.1

Germany DE 335 10.2 195.1 13.6 141.3 12.7

Greece EL 91 2.8 34.2 2.4 29.7 2.7

Hungary HU 31 0.9 8.0 0.6 6.7 0.6

Ireland IE 38 1.2 18.2 1.3 13.4 1.2

Italy IT 300 9.1 128.1 9.0 101.4 9.1

Latvia LV 6 0.2 1.3 0.1 0.7 0.1

Lithuania LT 8 0.2 2.2 0.2 1.8 0.2

Luxembourg LU 6 0.2 1.8 0.1 1.3 0.1

Malta MT 7 0.2 1.3 0.1 1.3 0.1

Netherlands NL 223 6.8 106.0 7.4 91.3 8.2

Poland PL 60 1.8 19.3 1.4 14.8 1.3

Portugal PT 117 3.6 36.8 2.6 31.5 2.8 Country Acrony Participati % Total % Total requested %

m ons costs (€ EC contribution million) (€million)

Romania RO 50 1.5 11.7 0.8 7.4 0.7

Slovakia SK 17 0.5 2.6 0.2 1.9 0.2

Slovenia SI 43 1.3 16.3 1.1 11.5 1.0

Spain ES 389 11.8 181.9 12.7 145.0 13.1

Sweden SE 111 3.4 72.8 5.1 52.3 4.7

United Kingdom UK 315 9.6 154.8 10.8 137.0 12.3

Sub-total 2912 88.4 1305.8 91.2 1033.5 93.2

Of which:

EU-15 2601 79.0 1217.1 85.0 969.9 87.4

EU-13 311 9.4 88.7 6.2 63.6 5.7

Associated countries

Albania AL 3 0.1 0.3 0.02 0.3 0.03

Armenia AR 1 0.0 0.2 0.01 0.2 0.02

Bosnia and BA 7 0.2 1.2 0.1 1.1 0.1 Herzegovina

Faroe Islands FO 2 0.1 0.4 0.03 0.4 0.04

Former Yugoslav MK 4 0.1 1.1 0.1 1.1 0.1 Republic of

Macedonia

Georgia GE 1 0.0 0.3 0.02 0.2 0.02

Iceland IS 5 0.2 1.3 0.1 1.0 0.1

Israel IL 16 0.5 4.9 0.3 3.7 0.3

Moldova (Republic MD 5 0.2 0.6 0.04 0.2 0.02 of)

Montenegro ME 1 0.0 0.03 0.00 0.0 0.00

Norway NO 66 2.0 42.1 2.9 32.6 2.9

Serbia RS 8 0.2 1.8 0.1 1.7 0.2

Switzerland CH 53 1.6 24.3 1.7 1.0 0.1

Tunisia TN 6 0.2 1.3 0.1 0.9 0.1

Turkey TR 38 1.2 14.6 1.0 11.3 1.0

Ukraine UA 5 0.2 3.4 0.2 1.3 0.1

Sub-total 221 6.7 97.9 6.8 57.0 5.1

Third countries

Argentina AR 4 0.1 0.8 0.1 0.5 0.04

Australia AU 4 0.1 0.5 0.04 0.3 0.03

Azerbaijan AZ 1 0.0 0.0 0.00 0.0 0.00

Botswana BW 1 0.0 0.2 0.01 0.2 0.02

Brazil BR 5 0.2 1.3 0.1 0.3 0.03

British Virgin VG 1 0.0 0.2 0.01 0.2 0.01 Islands

Burkina Faso BF 1 0.0 0.1 0.01 0.1 0.01

Canada CA 6 0.2 0.9 0.1 0.1 0.01

Chile CL 2 0.1 0.8 0.1 0.4 0.04

China (People's CN 10 0.3 1.5 0.1 0.4 0.04 Republic of)

Colombia CO 3 0.1 0.2 0.01 0.2 0.02

Cote d'Ivoire CI 3 0.1 0.2 0.02 0.2 0.02

Ecuador EC 2 0.1 0.4 0.03 0.4 0.03 Country Acrony Participati % Total % Total requested %

m ons costs (€ EC contribution million) (€million)

Ethiopia ET 3 0.1 0.3 0.02 0.3 0.03

Ghana GH 1 0.0 0.1 0.01 0.1 0.01

Greenland GL 2 0.1 0.1 0.01 0.1 0.01

Hong Kong HK 2 0.1 0.3 0.02 0.0 0.00

India IN 4 0.1 0.4 0.03 0.3 0.02

Indonesia ID 1 0.0 0.1 0.01 0.1 0.01

Japan JP 8 0.2 2.0 0.1 1.7 0.16

Jordan JO 1 0.0 0.1 0.0 0.1 0.01

Kenya KE 16 0.5 2.8 0.2 2.7 0.25

Korea (Republic KR 2 0.1 0.1 0.01 0.1 0.01 of)

Lebanon LB 1 0.0 0.1 0.01 0.1 0.01

Malawi MW 1 0.0 0.1 0.01 0.1 0.01

Mexico MX 1 0.0 0.2 0.02 0.0 0.00

Morocco MA 4 0.1 0.6 0.04 0.5 0.04

Mozambique MZ 3 0.1 0.4 0.03 0.4 0.03

Namibia NA 2 0.1 0.3 0.02 0.3 0.03

Peru PE 1 0.0 0.1 0.01 0.1 0.01

Russian RU 7 0.2 1.0 0.1 0.2 0.02 Federation

South Africa ZA 28 0.9 4.9 0.3 4.2 0.37

Sri Lanka LK 1 0.0 0.3 0.02 0.3 0.02

Taiwan TW 1 0.0 1.2 0.1 0.1 0.01

Tanzania (United TZ 6 0.2 2.0 0.1 2.0 0.18 Republic of)

Uganda UG 2 0.1 0.2 0.02 0.2 0.02

United States US 11 0.3 0.7 0.1 0.1 0.01

Viet Nam VN 2 0.1 0.3 0.02 0.3 0.03

Zambia ZM 3 0.1 0.4 0.03 0.4 0.03

Sub-total 161 4.9 27.8 1.9 18.8 1.70

Grand total 3294 100 1431.5 100 1109.3 100

Source: CORDA, extraction: 1/1/2017. SME Instrument Phase 1 excluded. The Associated Countries in the table have either entered into force on 1/01/2017 or are provisionally applicable. 28

Compared with the Cooperation Theme “Environment (including Climate Change)” of

FP7 (FP7-ENV) 29 , the share of funding received by Member States has increased by

7.6% (i.e. from 86.6% to 93.2%). EU-15 countries are increasing their FP7-ENV share by 7.5% (i.e. from 81.3% to 87.4%) and EU-13 by 29.5% (i.e. from 4.4% to 5.7%). Instead, Associated Countries see their share reduced from 9.3% to 5.1% and Third countries from 4.4% to 1.95%. Part of the reduction of the Associated Countries share can be attributed to Switzerland, not considered anymore Associated until January 2017. It used to represent 3.1% of the total FP7-ENV’s contributions.

28 See: http://ec.europa.eu/research/participants/data/ref/Horizon 2020/grants_manual/hi/3cpart/Horizon 2020-hi-list

href="http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/3cpart/h2020-hi-list-ac_en.pdf">ac_en.pdf

29 Data extracted from e-Corda, on 28/08/2016.

The situation of EU-13 Member States merits a further analysis. The share of EC contributions received so far (5.7%) looks a priori low, despite the increase from FP7- ENV. However, this figure is higher than their part of the European Union’s Gross Domestic Expenditure in R&D (GERD). EU-13 countries together sum-up around 4% of

the EU’s GERD 30 , while they receive 6.2% of the Horizon 2020-SC5 funding going to

Member States.

The Ex Post Evaluation of FP7-Environment 31 showed a strong correlation between the

FP7-funding awarded to Member States and the scale of their GDP (R-square= 81.8%) and of their R&I investments (R-square= 72.7%). This means that, overall, the more a country invests in R&D (which depends also of the economic size of the country), the more likely it received FP7 support. It looks logical that, as the Framework Programmes are based on excellence, Member States that have attained a critical mass in terms of national R&D investment and capacities are in a better position to benefit from EU funding.

Those correlations are still statistically significant for Horizon 2020-SC5, but the R-

square indicator is less strong 32 than under FP7-ENV.

The bivariate regression between national R&D expenditure and the SC5 funding secured is also useful to determine which countries over-perform and which ones under-perform, using as reference the expected EC contribution. This is illustrated in Figure 258.

The regression indicates that all EU-13 Member States under-perform. Their regression residuals are always negative, Slovenia being the EU-13 country closer to the expected EC contribution (with a standardised residual of -0.25), followed by Poland (-0.33) and Romania (-0.37). A critical issue for EU-13 countries is that they present lower success rates than EU-15 and Associated countries. This means that proportionally more national R&I resources are spend in applying unsuccessfully to Horizon 2020- SC5 (see Table 202). On the other hand, Spain, the Netherlands, the United Kingdom, Italy and to a lesser extent Belgium, are the countries that secured the most funding compared with the national investment in R&D. Germany and France, despite being amongst the main SC5 beneficiaries, received a smaller amount than expected. The picture is rather similar than in the Ex Post Evaluation of FP7-Environment.

30 4.3% in 2014, according to Eurostat’s database on 1/01/2017.

31 op.cit., pp.26-27.

32 For GERD (year of reference, 2014 – Eurostat data extracted on 1/01/2017): R-square= 59.7%. The R-square for

the regression between the GDP (year of reference: 2014) and the EC contribution is 75.4%, lower than under FP7- ENV.

Figure 244 - Correlation between national R&D investment (X-axis, EUR million) and SC5 contribution (Y-axis, EUR million) – EU Member States only

160,0

140,0 ES UK DE

120,0

100,0 IT

NL

80,0 FR

60,0 BE

SE

40,0 FI PT

EL AT

20,0 PL DK SI IE

CZ 0,0

0,0 10.000,0 20.000,0 30.000,0 40.000,0 50.000,0 60.000,0 70.000,0 80.000,0 90.000,0

Sources: CORDA and Eurostat. Extraction: 1/1/2017. SME Instrument Phase 1 excluded.

Table 200 - Success rates per group of countries for Horizon 2020-SC

Success Rate Success Rate

(No. of participations/ No. (actual EC contribution/ Funding of Applications, %) requested by applicants, %)

EU28 20.7 13.0

EU-15 20.9 13.3

EU-13 19.2 9.4 Associated countries 21.4 11.4

Third countries 18.3 13.6 Total 20.6 12.9

Source: CORDA. Extraction: 1/1/2017. SME Instrument Phase 1 excluded. Applicants which country is not identified (1,840) are not included in the calculations.

Geographical distribution, by participations

The picture changes when the number of participations is taken as reference instead of the EU funding received. This indicator appears more suited to analyse the openness of Horizon 2020-SC5, because it is not biased by economic factors such as the different living standards. Table 201 shows that beneficiaries from EU-13 countries represent 9.4% of the total. This almost doubles their share of EC contributions (5.7%). Associated countries reach 6.7% (instead of 5.1%) and the share of organisations from Third countries is 4.9% (instead of 1.7%).

N.2.2.4. International cooperation

Horizon 2020-SC5 has received so far 878 applications from Third countries. After evaluation, there are 135 distinct beneficiaries from Third countries in 161 total participations. These participants are coming from 40 different countries. South Africa (28), Kenya (16), United States (11) and China (10) are the Third Countries with more participations in projects. In terms of funding, South Africa, Kenya and Japan received the highest shares, between 0.37% and 0.16% of the total EC contributions.

As expected, the topics with more participation from Third countries are those with a strong international dimension, either described in the Work Programme or due to a strong connection with international agendas (e.g. IPCC, Arctic, Rio+20/SDGs). The concentration of funding in Member States, explained in section N.2.2.3, can be considered as an issue, taking into account the principle of openness to the world of the European Research Area – “a unified area open to the world, (…) in which scientific

knowledge, technology and researchers circulate freely” 33 – and further defended by Commissioner Moedas. 34

However the picture is rather different when the number of participations is taken as reference. The breakdown of EU funding includes biases due to with different living standards or with the legal possibility of funding organisations from some countries. It has to be taken into account that that some relevant Third countries, such as Brazil, China or India, are in principle not supposed to be funded anymore by Horizon 2020, while they were under FP7. This is likely to (i) decrease the EC contribution to Third countries, and (ii) reduce their participation. These factors have an impact on the overall figures. In FP7-Environment, 9.3% of the participants came from Third countries, compared with 4.9% under the ongoing Horizon 2020-SC5..

N.2.3. Cross-cutting issues

N.2.3.1. Overview

As of 1 January 2017, 48.6% of the Horizon 2020-SC5 projects’ budget is devoted to climate change and 95.9% to sustainable development. Unsurprisingly, Horizon 2020- SC5 is one of the main contributors to these targets (35% and 60% respectively).

N.2.3.2. Gender

The CORDA database does not provide complete data on the number of researchers involved in the projects and their gender, as it used to be the case under FP7. 28% of the

coordinators are women and women represent 39.2% of the project’s participants 35 .

Between 2014 and 2016, 57 Horizon 2020-SC5 projects (EC contribution: EUR 348.3 million) have been tagged as having a gender dimension. 28.2% of the projects include a gender dimension in the research content.

33 Communication from the Commission (2012) A Reinforced European Research Area Partnership for Excellence and

Growth, COM(2012)392 final i.

34 European Commission (2015) Open innovation, open science, open to the world. A vision for Europe. Luxembourg:

OPOCE.

35 Only projects where the gender of the coordinator is known (approximately two thirds of he projects.)

N.2.3.3. Social Sciences and Humanities (SSH)

In terms of the promotion of social sciences and humanities (SSH), in the period 2014- 2016 there were 80 SSH flagged projects, for a total EC contribution of EUR 463.9 million. The Horizon 2020 legal base establishes no targets and there were no similar methodologies in previous Framework Programmes. It is therefore not possible to compare with previous data.

N.2.3.4. Digital agenda

The contribution of Horizon 2020 to the Digital Agenda 36 is calculated with a similar

methodology than the expenditure in sustainable development, climate change and

biodiversity. On 1 st of January 2017, 14.3% (EUR 160.1 million) of the EC contributions

to Horizon 2020-SC5-funded projects is dedicated to projects related to the Digital Agenda. This figure looks very low and raises questions either about the methodology, or about its implementation. Horizon 2020-SC5 supports actions where ICT are always embedded: complex climate change models, earth observation (from space and in-situ, for example through citizens observatories apps), water and waste treatment that include monitoring, air quality, etc. The very low score mentioned above looks unrealistic.

N.2.4. Focus on Raw Materials

As explained in section N.1.2, the raw materials part of Horizon 2020-SC5 presents relevant specificities in its logic of intervention and functioning. A closer look at its participation statistics is therefore necessary. In sum, 40 projects gathering 593 participants from 43 countries have been selected in the 23 topics on raw materials under SC5 or Waste Focus Area, with and EU contribution of EUR 211 million. The average project budget is therefore EUR 5.3 million. Average EC contribution per participation is EUR 355.000 and there are around 15 partners per project in average. These data do not include the projects selected under SME Instrument, as this is analysed at Societal Challenge level.

Table 197 shows that, in line with the other parts of Horizon 2020-SC5, the raw materials part of the programme has so far been implemented mainly through Research and Innovation Actions. However, in line with the target of Pilot Actions of the EIP on Raw Materials for 2020, Innovation Actions will take more and more importance with the time, both in funding and participants. For example, 74% of the budget allocated to raw materials topics under SC5 in 2017 is dedicated to Innovation Actions (on Processing and Metallurgical processes). The ratio EC Contribution/Total project costs is close to 90%. The planned increase in the use of IAs for next years will make this value closer to the average of SC5.

36 Communication from the Commission (2010) Digital Agenda for Europe, COM(2010)245 final i

Table 201 - Number of projects, EC funding, oversubscription rate and success rate per type of action (raw materials projects under Horizon 2020-SC5, SME Instrument excluded)

Instrument No. of No. of Share of Total Actual Share Over Suc Avera eligible projects total EC EC of subscr cess ge EC

proposa projects contrib contri total iption rate contri ls (%) ution bution EC rate (%) bution requeste (€ contri per d (€ millio bution projec

million) n) (%) t (€ millio n)

CSA 33 13 32.5 55.8 25.3 12.0 2.2 39.4 1.9

ERA-NET- 1 1 2.5 5.0 5.0 2.4 1.0 100. 5.0 Cofund 0

IA 4 2 5 36.5 22.0 10.4 1.7 50.0 11.0

RIA 156 24 60 920.2 158.5 75.2 5.8 15.4 6.6

Total 194 40 100 1017.5 210.8 100 4.8 20.6 5.3

Source: CORDA, extraction: 1/01/2017.

Table 202 - Participation per type of organisation (raw materials projects under Horizon 2020-SC5, SME Instrument excluded)

No. of % Of No. of % of No. of Average EC % of partici which: newco newco indivi particip contrib total

pants coordi mers mers dual ation per ution(€ fund nators partic beneficia million) ing ipants ry

HES: Higher or 128 21.6 11 3 2.3 81 1.6 52.4 24.9 Secondary

Education

Establishments

OTH: Other 42 7.1 4 19 45.2 34 1.2 7.7 3.7

PRC: Private 248 41.8 10 127 51.2 218 1.1 93.9 44.5 for Profit

Entities

(excluding HES)

PUB: Public 33 5.6 1 8 24.2 30 1.1 5.1 2.4 bodies

(excluding REC and HES)

REC: Research 142 23.9 14 5 3.5 67 2.1 51.7 24.5 organisation

Total 593 100 40 162 27.3 430 1.4 210.8 100

Of which, SMEs 122 20.6 2 41 33.6 105 1.2 35.8 17.0

Source: CORDA, extraction: 1/1/2017.

Private for-profit entities clearly present the highest share of participation in the proposals (near 50% of applications and requested funding being for private entities). Similar figures are seen in the selected raw materials projects under SC5 (Table 16), with 42% of participations and near 45% of funding received, significantly higher figures than the average of SC5. Higher or Secondary Education institutions (mainly universities) and Research organisations present significantly lower figures (with 22-24% participations each and near 25% in terms of funding). Meanwhile, Public bodies and “Others” are below 8% in participations and 4% in funding. Overall, 27.3% of participations (34.7% of individual beneficiaries) are newcomers. Among the newcomer organisations, 78% are private companies (51% of the private companies are newcomers). These figures exemplify how Horizon 2020 is successfully helping to consolidate a community of public and particularly private stakeholders working on research and innovation on raw materials, in line with the objectives of the EIP on Raw Materials. On average, private companies, public bodies and "others" participate much less times than academic and research institutions.

Interestingly, 122 SMEs (105 individual beneficiaries, which represent half of the private companies) have received almost 36 million euro of funding under the budget of general calls of SC5 for raw materials. This already represents a level of participation of 24% in terms of individual participants and a funding of 17%. Taking into account the budget corresponding to raw materials dedicated to SME instrument in 2014-2016 (EUR 15.5 million), the percentage of budget dedicated to fund research and innovation activities carried out by SMEs on the raw materials field between 2014-2016 is close to 23%, i.e. above the target of 20%.

The leaders in terms of number participations in the projects on raw materials are mostly research and academia organisations (with Bureau de Recherches Géologiques et Minières first). In terms of funding, Lulea University leads, mainly thanks to its participation on an Innovation Action. Major industrial companies are participating, often in more than one project (sometimes coordinating) and receive relevant funding. Upstream sector (such as mining companies Cobre las Cruces, Eramet, KGHM) and downstream industrial companies (e.g. equipment providers such as Soil Machine Dynamics, Atlas Copco, Outotec) have significant participation (mostly in RIAs/IAs) and are ranked in the top 5% in terms of funding received. End-users (e.g. Ford, Airbus, Johnson Matthey) are also involved in projects. Consequently, representatives of all the relevant steps in the whole raw materials value chain from exploration to end-use and recycling and substitution are participating, as requested in most RIA/IA topics.

Partners from 43 different countries participate in the projects on raw materials. The geographical distribution of beneficiaries and EC funding (with 90% of funding allocated to EU-15 countries and Germany and United Kingdom leading) is similar to the general trends of Horizon 2020-SC5.

It should be noted that 15 projects with 218 individual participants are clearly linked with raw material commitments. The comparison of the distribution by country with the statistics of participation in raw materials commitments of the EIP shows common trends, such as the good position of Germany (and the first position of Poland in EU-13). But there are also differences: Spain and particularly Italy (the leaders in participation in commitments) have lower presence in selected Horizon 2020-SC5 projects compared to their participation in commitments. On the other hand, United Kingdom and Netherlands have good results in Horizon 2020-SC5 despite their moderate participation in commitments. The relatively low participation from EU-13 Member States is also observed in the Calls for Commitments of the EIP (with a number of unique partners from "new" Member States below 10%). Additional dissemination activities are being put in place to encourage the participation of stakeholders from those countries.

Around two thirds of participants (gathering almost 90% of funding granted) are working in subjects covered by the first Pillar of the EIP on Raw Materials (Technology Pillar). On the other hand, one fourth of participants which receive 8% of funding are working on framework conditions or knowledge base for primary and secondary raw materials (Second Pillar of the EIP). Finally, participants working on issues related to International Cooperation (Third Pillar) represent 8% in number and 3% in funding. This is perceived in the distribution of type of actions and their funding allocated, much higher for RIAs and IAs (covering the first pillar) than for CSAs (covering the other two pillars). This trend will consolidate in the future with the introduction of more IAs on raw materials. Finally, 26 projects (two thirds of the total) address to some extent challenges related to critical raw materials.

N.3. R ELEVANCE

This section analyses to what extent the original objectives of Horizon 2020 and the intervention logic are still relevant, taking into account new socio-economic and/or scientific developments.

N.3.1. Is Horizon 2020-SC5 tackling the right issues?

N.3.1.1. The relevance of Horizon 2020-SC5 given the challenges to address

The Horizon 2020 Impact Assessment explained that “on the backdrop of a changing world order, Europe faces a series of crucial challenges: low growth, insufficient innovation, and a diverse set of environmental and social challenges”. These problems are inter-connected and require a common solution: “it is precisely by addressing its environmental and social challenges that Europe will be able to boost productivity,

generate long-term growth and secure its place in the new world order”. 37 Five years

after the publication of this analysis, the symptoms described persist. The European economy seems to emerge little by little from the most severe crisis in decades, but growth rates remain moderate and insufficient to substantially reduce unemployment. Despite being in its fourth year of slow recovery, the European economy faces risks (e.g. slowdown in emerging countries) and presents weaknesses (e.g. excessive debt in some

countries, low investment, economic uncertainty). 38

The report The European Environment – State and outlook 2015 39 explains that in the

last decades, environmental and climate policies have delivered substantial benefits for European ecosystems and living standards. They have also created economic opportunities (e.g. the environmental sectors grew by 50% between 2000 and 2011), but there are still challenges: “European natural capital is being degraded by socio-economic activities such as agriculture, fisheries, transport, industry, tourism and urban sprawl. And global pressures on the environment have grown at an unprecedented rate since the 1990s, driven not least by economic and population growth, and changing consumption patterns”.

Climate issues and their consequences are one of the major threats for humanity. As

explained by the 5 th Report of the International Panel on Climate Change (IPCC), “many

of the observed changes are unprecedented over decades to millennia. The atmosphere and ocean have warmed, the amounts of snow and ice have diminished, sea level has

37 COM(2011)808 final i, op.cit., p.6.

38 European Commission (2016) European Economic Forecast, Winter 2016. Institutional Paper 020, February 2016.

Luxembourg: OPOCE.

39 European Environment Agency (EEA, 2015) The European Environment – State and outlook 2015. Synthesis Report.

Luxembourg: OPOCE, p.9.

risen, and the concentrations of greenhouse gases have increased” 40 . The IPCC concludes

that “a large fraction of anthropogenic climate change resulting from CO2 emissions is irreversible on a multi-century to millennial time scale, except in the case of a large net

removal of CO2 from the atmosphere over a sustained period”. 41

Climate change worsens other environmental issues, like water scarcity – with impacts on food and energy supply. Water scarcity is seen as one of the major global risks, source of economic, social (poverty, migration), political (instability, conflicts within and between countries to access the same resources), on health (quality of available drinkingwater)

 and environmental issues (biodiversity) 42 . Water is one of most important

traditional areas of intervention of the Framework Programmes concerning environmental issues.

The international community is increasingly aware about the magnitude and impacts of climate change and environmental issues for the future of humanity. In 2015, there were two major international agreements: the UN's 2030 Agenda for Sustainable Development, with the approval of the Sustainable Development Goals (SDGs), and the COP21 Paris Agreement on climate change (see section 2.2). These two events have put climate and sustainable development, areas of intervention of Horizon 2020-SC5, at the top of the international political agenda. Ongoing projects, like the FP7 Helix, are already adapting their analysis to those agreements.

Concerning raw materials, despite the progress made in several areas since the launching of the Raw Materials Initiative in 2008, projections indicate that resource use could

double between 2010 and 2030, mostly driven by demand in developing regions 43 . This

rising demand for raw materials has significant consequences for the EU’s security of supply. Planning cycles — i.e. the time between the exploration of a mineral deposit and the development of a mine — can take up to 10 years or more. Because of such long cycles, raw materials supply cannot always be increased in the short term. Similarly there are also limits to increasing raw materials production from secondary raw materials, as this depends, amongst others, on the amount of products that reach their end of life and become available for recycling. As a result, when demand exceeds supply, prices spike. This drives up production costs for downstream industries. High prices can also be aggravated by export restrictions and trade barriers put in place by supplying countries. This was the scenario when Horizon 2020 was designed, with recent sharp price increases of the rare earth elements caused by the export restrictions introduced by China

in 2011 44 . Currently, commodity prices have dropped which lead to other issues: longterm

 investments are put on hold, which negatively affects future production capacity. On the other hand, the list of Critical Raw Materials according to the Commission has passed from 14 in 2010 to 20 in 2014, and substitution for many applications remains extremely difficult. These elements support public intervention in the field of raw

40 IPCC (2013) Summary for Policymakers. In: Climate Change 2013: The Physical Science Basis. Contribution of

Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate. Cambridge (UK) and

New York: Cambridge University Press, p. 4.

41 Ibid. p.28.

42 World Economic Forum (2015) Global Risks 2015. Geneva: World Economic Forum; Pitman, G.K. (2002) Bridging

Troubled Waters. Assessing the World Bank Water Resources Strategy. Washington D.C.: The World Bank; UN’s factsheets: http://www.unwater.org/water-cooperation-2013/water-cooperation/facts-and-figures/en/; European

Commission (2011) EuroMed 2030. Long term challenges for the Mediterranean area. Report of an Expert Group.

Luxembourg: OPOCE; Overseas Development Institute et al. (2012) Confronting scarcity: Managing water, energy and land for inclusive and sustainable growth. Luxembourg: OPOCE; IPCC (2013) Climate Change 2013. The

Physical Science Basis. New York: Cambridge University Press.

43 EIP on Raw Materials (2016) op.cit.

44 Ibid., Indicator 5 ‘Export restrictions’

materials, but also a strong coordination with stakeholders – because raw materials supply is characterised by interlinked complex value chains.

However, this intervention must be coordinated with stakeholders. As seen in the Figure below for the example of neodymium (an element used in permanent magnets and lowcarbon technologies and without significant production within the EU), raw materials supply is characterised by interlinked complex value chains. Therefore, solutions to the challenges on raw materials come from the involvement of various players working on diverse dimensions: improved knowledge base, technological innovations, better framework conditions, proactive international cooperation. Indeed, according to a recent

report 45 , the mining industry has the opportunity and potential to positively contribute to

all 17 SDGs.

Figure 245 - Simplified Sankey Diagram of Neodymium’s value chain

Source: Study on Data Inventory for a Raw Material System Analysis. 46

N.3.1.2. The relevance of SC5 to address European objectives

President Juncker stated in his political priorities that “(…) after spending several years concentrating on crisis management, Europe is finding it is often ill-prepared for the global challenges ahead, be it with regard to the digital age, the race for innovation and skills, the scarcity of natural resources, the safety of our food, the cost of energy, the impact of climate change, the ageing of our population or the pain and poverty at

Europe’s external borders”. 47

The expected impact of Horizon 2020– SC5 on President Juncker’s political agenda is substantial, in line of the logic of the Europe 2020 strategy and of the Horizon 2020 Impact Assessment: R&I activities that address societal challenges are drivers of future growth and competitiveness.

A new boost for growth, jobs and investment

45 VV.AA (2016) Mapping Mining to the Sustainable Development Goals: An Atlas. Columbia Center on Sustainable Investment (CCSI), UN Sustainable Development Solutions Network (SDSN), United Nations Development Programme (UNDP), World Economic Forum. At: http://unsdsn.org/wpcontent/uploads/2016/11/Mapping_Mining_SDGs_An_Atlas.pdf

46 Study on Data Inventory for a Raw Material System Analysis, DG GROW, November 2015.

https://ec.europa.eu/growth/tools-databases/msa/raw-material/neodymium#

47 op. cit, https://ec.europa.eu/priorities/sites/beta-political/files/juncker-political-guidelines_en.pdf

The so-called “environmental goods and services sectors” 48 are amongst the very few

that have flourished in terms of revenues, trade and jobs during the economic and financial crisis that started in 2007. Employment rose by 48 % between 2000 and 2012 (from 2.9 million full-time equivalents in 2000 to 4.3 million full-time equivalents in 2012). In parallel, sectoral value added almost doubled in real terms to reach EUR 271 billion in 2012. The environmental economy’s contribution to overall GDP increased from 1.5 % to 2.1 % during that period.

Europe is the world leader in environment-related technologies. In 2012, European Union countries represented 37% of the environment-related technologies developed worldwide, with 6,778 claimed priorities patent applications, against 22% for Japan, 17% for the United States, 10% for Korea and 4% for China. The EU is particularly well performing in the field of environmental management technologies, with 43% of the world's patents having a Europe-based inventor in 2012, against 37% for water-related adaptation technologies and 35% for climate adaptation technologies.

The OECD is calling for “putting sustainable development at the core of business

models”, arguing that investments in sustainability are smart ones 49 : The global water or

waste markets, traditional areas of intervention of Horizon 2020-SC5, are estimated at $400 billion and EUR 250 billion per annum respectively, with global investments over

EUR 33 billion only for water 50 .

Concerning raw materials, section N.1.2 has already described how sectors such as mining and metallurgy contribute to the EU economy with hundreds of billions of Euros in added value and several million jobs, both directly in mining and metal industries or indirectly in downstream, manufacturing industries. Despite being an industry of relative low R&D intensity, top investor companies in the raw materials sector have almost doubled their annual R&D expenditure since 2003, growing more than twice as fast as

public R&D investments between 2003 and 2013 51 . Patent applications are mainly filed

by companies. Even though the EU proportion of patent applications in the raw materials sector is on a decreasing trend, it still accounted for almost 36 % of patent applications filed in the same sector by the EU, Australia, Canada, Japan, Russia and the USA altogether. The number of EU patent applications increased in some sub-sectors such as

‘mining and mineral processing’ with a rise of 35 % between 2000 and 2011 52 .

A Resilient Energy Union with a Forward-Looking Climate Change Policy

Horizon 2020-SC5 is also expected to contribute to the third priority of President Juncker’s agenda. Societal Challenge 5, like its predecessors, is expected to provide a major contribution to the IPCC reports, increasing the knowledge base about climate change, its processes and consequences. In addition, it is has to support and assess mitigation and adaptation measures and policies, as requested in the Specific Programme.

48 Defined by Eurostat as “those produced for the purpose of environmental protection (i.e., preventing, reducing and

eliminating pollution and any other degradation of the environment) as well as resource management (i.e., preserving and maintaining the stock of natural resources and hence safeguarding against depletion)”. See: http://ec.europa.eu/eurostat/statistics-explained/index.php/Environmental_goods_and_services_sector

49 OECD (2016) Development Co-operation Report 2016: The Sustainable Development Goals as Business

Opportunities. Paris: OECD Publishing.

50 Horizon 2020 Work Programme 2014-2015, part 12. Original source: UNEP, quoted by World Watch Institute, and

Acqueau (EUREKA Cluster in the environmental Sciences).

51 EIP on Raw Materials (2016) op.cit, Indicator 8 'Corporate R&D investment'

52 Ibid, Indicator 9 'Patent applications'

Several promising low carbon technologies strongly rely on a sustainable supply of certain raw materials (including Critical Raw Materials – CRMs), whose demand is expected to escalate in the future. Supply disruptions might endanger the market uptake of these technologies and the fulfilment of climate and energy targets. In this regard, Horizon 2020-SC5 is contributing with a multilateral approach. First, promoting a sustainable supply of these raw materials mostly through RIA and IA focused on production of raw materials and Coordination and Support Actions focused on nontechnological issues (e.g. International cooperation to forecast the Demand-supply forecast and raw materials flows at global level, with particular attention to CRMs used in low-carbon technologies). In parallel, the substitution of critical raw materials by viable alternatives is stimulated via Research and Innovation Actions under SC5 and other parts of the Programme.

A Deeper and Fairer Internal Market with a Strengthened Industrial Base

The overall objective of the EIP on Raw Materials is to contribute to several of the flagship initiatives of the Europe 2020 Strategy, and especially to the EU Industrial Policy. Well aligned with this objective, Horizon 2020-SC5, particularly the part focused on the sustainable supply of the raw materials, contributes to strengthen the industrial base of the EU. The EU is highly dependent on imports of many metal ores and natural rubber, but it is the security of supply (i.e. its diversification) that is crucial for a strong European industrial base. Raw materials are an essential building block of the EU’s economy, with many sectors relying on raw materials supply. The metal industry, and particularly the steel sector, is a good example of the complex interdependencies between the different stages of the value chain. The State of the Union speech on 14 September 2016 called several times for a stronger support to European steel industry, currently

suffering strong market disruption. 53

A Stronger Global Actor

Horizon 2020-SC5 plays a role to make the EU a stronger international actor (priority 9), through the active participation of projects and the Commission services in international fora like the IPCC, GEO/GEOSS, the Belmont Forum, United Nations (the Multi Stakeholder Forum on Science, Technology and Innovation for the SDGs) or the White House Arctic Science Ministerial Meeting. Taking into account the important place that issues like climate change or sustainability have in the international political agendas, and the strong R&I component behind, science diplomacy looks a necessary factor to enhance European position in the international scene.

Towards a New Policy on Migration

Last but not least, migration (priority 8) is not at the core of Societal Challenge 5 activities. However there is more and more evidence that alerts about the current and

future impacts of climate change and environmental pressures on migration. 54 The

forthcoming PRIMA initiative (see section N.3.2) responds indirectly to the migration political priority.

53 http://ec.europa.eu/priorities/state-union-2016_en

54 See for instance: Baker, A. (2015) “How Climate Change is Behind the Surge of Migrants to Europe”, in Time, 7

September; Raleigh, C.; Jordan, L. and Salehian, I. (2008) “Assessing the impacts of climate change on migration and conflict”, Paper commissioned by the World Bank Group for the Social Dimensions of Climate Change workshop,

Washington DC.

N.3.2. Adaptation to new scientific and socio-economic developments

Since the preparation and approval of Horizon 2020 Impact Assessment, Regulation and Specific Programme, there have been some major social, political and economic developments, now high in the political agendas. The impact of current or starting actions is indirect, for example, on migration through the forthcoming PRIMA initiative.

Box 21 - PRIMA and its expected impact on migration

PRIMA stands for “Partnership for Research and Innovation in the Mediterranean Area”. It is an

EU initiative originally proposed on 23 December 2014 by a set of Member States and

Neighbouring countries. Its aim is to establish a joint programme “for the development and adoption of innovative and integrated solutions for improving the efficiency, safety, security and

sustainability of water provision and food systems in the Mediterranean area”, pursuant to Article

185 of the Treaty on the Functioning of the European Union (TFEU).

The PRIMA Impact assessment, that obtained the placet of the Regulatory Scrutiny Board (RSB)

on 22 July 2016, considers that the initiative will have “unintended impacts” on mitigating

migration through its expected positive impact on the wellbeing and quality of life in

Mediterranean countries. 55

The inception impact assessment 56 went even further. It argued that literature explains migrants tend to be better educated than the average population in source countries. 57 This includes tertiary

educated people, like scientists or engineers, and provokes the well-known "brain drain" phenomenon. The main reasons to migrate are socio-economic, i.e. unemployment and,

especially, the aim of improving standards of living. Being employed does not prevent migration.

Fieldwork studies have demonstrated that although unemployed people are more likely to

consider migration, those who have a job, including theoretically good ones like managers or

professionals, present also high levels of intentions to leave their country. 58

Therefore, according to this rationale, an enhanced cooperation on R&I in agro-food and water,

sectors that are critical for the Mediterranean countries, is likely to create opportunities for young

and educated people, reducing their propensity to migrate. Enhanced R&I cooperation is also likely to improve the skills of potential educated migrants and increase the knowledge about the

situation in Southern Mediterranean for researchers and technicians from the Northern Europe, in

a context where climate change pressures will also affect Europe relatively soon.

New paradigm for a green economy and society

Horizon 2020-SC5 5 proposes and establishes the bases of a new green economy and society. In this vision, consistent with the Circular Economy narrative, “green growth” is not only a market niche that includes trends in energy conservation, renewables and sustainable products. It is about a radical “innovation in the productivity of resources, the shift from products to services and tangibles to intangibles an increase in the use of bio

55 SWD(2016)332/F1

56 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_rtd_009_prima_en.pdf

57 Docquier, F. and Marfouk, A. (2006) "International Migration by Educational Attainment", in Özden, C. and Schiff,

• L. 

  (ed.) International Migration, Remittances and the Brain Drain, Washington: World Bank and Palgrave Macmillan; te Velde, D.W. (2005) "Globalisation and education: what do the trade, investment and migration literatures tell us?", ODI Working Paper, London.

58 Alquézar, J. et al. (2010) Migration and Skills. The experience of migrant workers from Albania, Egypt, Moldova

and Tunisia. Washington: World Bank-European Training Foundation.

materials and bio-chemistry; healthy eating, exercise and preventative medicine; reuse

and recycling; and so on” 59 .

There are already “weak signals” about the potential of ‘a new (European) green lifestyle’, in the current growing demand for personal services, health, education, training, coaching, “quality of life” goods and services, creative industries, information intermediation, maintenance, rental services, energy conservation, recycling and other climate and resource related activities, all with a “green’ direction”. There is an unexploited demand for greener and sustainable cities. According to a Special Eurobarometer carried-out in December 2015, 43% of European citizens would like to have more natural features in their area. This percentage reaches 57% in large towns. For

the majority of Europeans (53%) natural features lead to better quality of life 60 .

Changing trends in raw materials issues

The raw materials part under Horizon 2020-SC5 is adapting with a systemic approach to several relevant socio-economic changes that have occurred since the time when Horizon 2020 was being planned. First, the prices of raw materials have severely decreased in recent years, which may put at risk long-term investments, future production capacity and jobs. In response to this, the topics on raw materials explicitly call for solutions with high economic viability and reduced costs (keeping the highest environmental, health and safety standards) in order to promote long-term competitiveness and creation of added value and new jobs in raw materials and downstream industries. Taking advantage of an ample catalogue of RIAs launched in 2014-2017 and thanks to higher budgets planned, Innovation Actions at high TRLs will have a major role in the Work Programme 2018- 2020, in line with the EIP targets. Additionally, extractive industries must responsibly tackle the more and more important question of social acceptance and address the socalled

“not in my backyard syndrome”. 61 Consequently, as a novelty RIAs and IAs in the

Work Programme 2016-2017 require proposals to include a plan to communicate the added value to the local communities and society for improving public acceptance and trust. Participation of civil society in a process of co-design, co-development and coimplementation is strongly encouraged. On the other hand, as a new list of Critical Raw Materials (CRMs) is expected for 2017, proposals submitted to CSAs focused on CRMs should demonstrate the flexibility of incorporating new CRMs in the scope of the project. More on a technological side, actions are (and will be) paying due attention to the incorporation of new technologies such as automation or robotics in fields such as mining or processing. Finally, the main goals and guidelines of the Circular Economy package published in the end of 2015 are taken into account in most raw materials topics. For example, it should be noted that Circular Economy communication mentions "Critical Raw materials" and "Construction and Demolition Wastes" as two of the five priority sectors that need to be addressed in a targeted way.

Horizon 2020 – SC5 aims at establishing the bases for a green transition. This requires a systemic approach, going beyond funding the development and commercialisation of

specific innovations. 62 To implement this socio-economic transition, Horizon 2020-SC5

support has a strong focus on large-scale demonstrations, which impacts (social,

59 Pérez, C. et al. (2016) Changing gear in R&I: green growth for jobs and prosperity in the EU. Report of the

European Commission Expert Group “R&I policy framework for Green Growth & jobs”. Luxembourg: OPOCE, p. 4.

60 Special Eurobarometer EB 444.

61 EIP on Raw Materials (2016) op.cit., Indicator 14 'Public Acceptance'

62 European Commission (2015) From niche to norm. Suggestions by the Group of Experts on a 'Systemic Approach to

Eco-Innovation to achieve 2015 Directorate-General for Research and Innovation a low-carbon, Circular Economy'. Luxembourg: OPOCE.

economic, on behaviours) are duly monitored, in order to unlock further investments for an uptake. In addition, the Commission tries to improve the framework, testing initiatives like the Innovation Deals.

N.3.3. Addressing specific stakeholder needs

The Commission services implementing Horizon 2020-SC5 are in contact with stakeholders through different channels:

 International fora, like the IPCC, the Belmont Forum, GEO/GEOSS, United

Nations, etc.

 Through the relationship with relevant European associations, like the European

Innovation Partnership (EIP) on Water (EIP Water), the European Innovation Partnership (EIP) on Raw Materials, the Joint Programming Initiative (JPI) on Water “Water Challenges for a Changing World”, the Water Supply and Sanitation Technology Platform (WssTP), the European Institute for Innovation

and Technology’s Knowledge and Innovation Community on Climate (KIC- Climate) and Raw Materials (KIC-Raw Materials), SPIRE (Sustainable Process Industry through Resource and Energy Efficiency), the High-level Group on Energy-Intensive Industries, etc.

 Through the active participation and involvement in relevant conferences, highlevel

or dissemination events;

 Through public consultations and calls for ideas launched to prepare each Work

Programme.

The Work Programmes include explicit references to relevant European initiatives and their Strategic Research Agendas (SRAs), like the EIP Raw Materials, the IPCC, the Belmont Forum, the JPIs Climate and Water, the EIP Water, the EIT, etc. Both the National Contact Points and the Commission services insist during events like Info Days on the need of reading such strategies and align to them to respond to the calls specifications.

As regards to raw materials, the EIP gathers in its Governance Groups (High level Group, Sherpa Group and Operational Groups) public and private stakeholders from Industry, Academia, Research, NGOs, Member States, etc. Thanks to its diverse composition, the EIP is the most important instrument to address stakeholder needs and to set priorities in the raw materials field. First, the Strategic Implementation Plan of the EIP was drafted in close collaboration with the wide range of stakeholders gathered in the governance groups of the EIP and is a strong source of inspiration of the topics of Horizon 2020 on raw materials (under SC5 and beyond). Secondly, the meetings of the operational groups of the EIP help to identify gaps, fine-tune priorities and optimise implementation. Finally, the "Raw Material Commitments" help the Commission to identify the most active areas and target fields for priority action. The commitments are essential to achieve the objectives set out in the Strategic Implementation Plan, and the calls for commitments are a powerful clustering and outreach tool: approximately 980 unique partners from very different sectors and from more than 50 different countries work in some commitment. A substantial and increasing amount of these partners are already involved in Horizon 2020 projects on raw materials.

N.3.4. Other issues related to relevance

The preparation of the Horizon 2020 Work Programme 2018-2020 included a gap analysis to check to what extent the objectives and activities stated in the legal base were covered by the two first Work Programmes. It is the occasion of making a balance, based on the analysis of the Work Programmes and ongoing projects. The analysis shows that the projects funded to-date from the 2014-2015 Work Programme together with the topics designed in the 2016-2017 Work Programme are in line with the overall aims of the Specific programme and provide great progress towards fully addressing its objectives. No major gaps have been revealed by the analysis; only minor areas require additional efforts. To optimise the use of resources and to create synergies between activities, a special attention has been brought in the design of projects and topics to address more than one activity contributing to the good coverage of the Specific programme. This explains why there are areas which have not been addressed exclusively with dedicated topics, in line with the systemic approach that Horizon 2020- SC5 follows.

N.3.5. Lessons learnt/Areas for improvement

In 2016, environmental and climate problems are even higher in political agendas than in 2011, when the Horizon 2020 Impact Assessment pointed-out them amongst the grand societal challenges that R&I should address. The Sustainable Development Goals and the COP21 Conferences have established, for the first time ever, global and compulsory objectives to better protect the planet. At EU level, the Circular Economy package defends a transition towards a sustainable production and consumption paradigm.

There is also a growing social awareness and citizens’ concerns about environmental issues, which translates in growing market demand for “green” products and services. This implies a business and investment case to move towards a sustainable economy, which includes sectors that create jobs and where Europe has competitive advantages. This requires also a strong commitment from extractive industries, not always the most environment-friendly. Therefore Horizon 2020-SC5 looks connected with the first priority of the President Juncker’s agenda: creating jobs, growth and investments.

To implement its vision for a greener economy, Societal Challenge 5 focuses on largescale pilots and demonstrations, with a monitoring of their impacts (social, economic, on behaviours), in order to unlock further investments for mainstreaming. In addition, the Commission services try to improve the framework, testing initiatives like the Innovation Deals.

The Commission services work closely with stakeholders, both through direct contacts and through public consultations and calls for ideas. Stakeholders have so far supported the actions carried-out under Horizon 2020-SC5, also alerting about the risks of some strategies. However the involvement of some kinds of stakeholders, like NGOs and Civil Society Organisations (CSOs) looks still low. The most active stakeholders remain the traditional R&I actors, like academia and industry. Despite the efforts to open Horizon 2020 to new players and to empower citizens (e.g. citizens science/citizens observatories), the risk is to remain disconnected from people.

N.4. E FFECTIVENESS

It is too early to assess the outputs of Horizon 2020-SC5. Only a few ongoing projects have reported publications and patents, and the quality of those data is arguable. Therefore this section mainly analyses outputs and outcomes from FP7-Environment and tries to extrapolate such results to Horizon 2020-SC5. It presents also the expected impacts, uncertainties and risks of the new approaches that are being implemented.

N.4.1. Short-term outputs from the programme

On 1 st January 2017, a few ongoing projects have reported results, i.e. publications and

patents. Only four patents and one trademark can be considered as actual IPR outputs of Horizon 2020-SC5 projects so far.

Data on publications present several inconsistencies (e.g. non-peer-reviewed peerreviewed papers, non-peer-reviewed thesis dissertations). Only 65 publications present a Digital Object Identifier (doi) and 148 an International Standard Serial Number (ISSN). It is therefore rather difficult to assess the reliability of the figures provided.

Table 203 - Ongoing Horizon 2020-SC5 projects publications, as reported in CORDA

Project Instru Peer Monog Book Conference Other Thesis Total Acronym ment reviewed raphic chapt proceedings dissert

articles book er ation Peer Peer Peer Not peer Peer Not peerreview

revie revie reviewed revie reviewed ed wed wed wed

AQUACRO RIA 4 7 11 SS

AtlantOS RIA 11 1 1 13

BAMB IA 1 1

BINGO RIA 1 6 7

BioMOre RIA 4 4

BlueSCities CSA 2 2 1 5

CABRISS IA 4 4

CARISMA CSA 1 1 2

CloseWEEE RIA 2 2 4

CRESCEN RIA 1 1 DO

ECOPOTE RIA 12 1 13 NTIAL

ESMERAL CSA 1 1 DA

EU- RIA 52 13 65 CIRCLE

FREEWAT CSA 1 1

green.eu CSA 1 1

GREEN- RIA 1 1 WIN

HISER RIA 1 3 4

IMPREX RIA 5 1 3 9

INREP RIA 1 1 11 1 14

INSPIRATI CSA 5 1 1 7 Project Instru Peer Monog Book Conference Other Thesis Total Acronym ment reviewed raphic chapt proceedings dissert

articles book er ation

Peer Peer Peer Not peer Peer Not peerreview revie revie reviewed revie reviewed

ed wed wed wed ON

INTRAW CSA 2 3 5

KINDRA CSA 1 1 2

MERCES RIA 2 2

METGRO RIA 1 1 W PLUS

MICA CSA 6 6

MINATUR CSA 6 5 11 A 2020

OptimOre RIA 8 2 10

PIANO CSA 2 2

POWERST IA 1 2 2 10 1 16 EP

PRIMAVE RIA 5 5 RA

Real-Time RIA 1 5 7 2 15 Mining

RESYNTE IA 3 2 5 X

RIBuild RIA 1 1

SWOS RIA 1 1 1 3

WaterWork ERA- 2 2 s2014 NET

WIDEST CSA 1 1 2

WINTHER SME 1 7 4 12 WAX

Total 118 1 1 50 27 19 46 6 268

Source: CORDA. Extraction: 1/1/2017.

There are so far 268 publications, of which 188 are peer-reviewed (thesis dissertation not included. Scientific papers represent 44% of all publications – 62.8% of the peerreviewed ones. There is also a relatively high number of conference proceedings, which is to be expected considering the short time between the beginning of the projects and their first results. It takes time to publish peer-reviewed papers, while presenting research results in conferences is generally faster – and those initial results use to lead to more peer-reviewed articles.

The most productive project so far is EU-CIRCLE, with 65 reported peer-reviewed publications. However, a close look to those publications shows that the vast majority have been issued in the same journal, which mainly publishes the papers and lectures accepted for and presented at some specific summer seminars in Poland, by almost the same authors.

Apart of EU-CIRCLE, ECOPOTENTIAL (a large RIA on earth observation to improve ecosystems) and AtlantOs (another large RIA that brings together the main actors on North Atlantic Earth Observation) are so far the project with more peer-reviewed articles.

Only 46 publications are published in gold open access, and 144 more in green open access, according to projects’ declarations. These data, if confirmed, are negative, because all Horizon 2020 publications are supposed to be in open access by default (the participants can decide to opt out). In FP7-Environment, the share was 44%. Open access was not compulsory, but the Cooperation Theme Environment (including Climate Change) participated in the pilot. FP7-Environment participants were invited to publish in open access.

Forty-one publications are the fruit of joint public-private efforts.

The ongoing study “European added-value and economic impact of the Framework

Programmes” 63 carried-out a survey between June and August 2016, in which Horizon

2020 beneficiaries gave their opinions about what would happen without EU support (different sources of funding, different level of ambition) and the expected impacts of their projects. The sampling was random, meaning that the margin of error is +/- 15.5%

(p=q=0.5 and level of confidence =95%) 64 . The main caveat of the study is that

respondents provide just opinions, most probably with a positive bias vis-à-vis Horizon 2020, because they are beneficiaries. There is no evidence so far to demonstrate that those hopes were actually accomplished. According to this survey:

57% of the respondents consider that without Horizon 2020 funding, the project would have had not gone ahead, while 27% say that they would have had to introduce no or minor modifications and 12.5% would have been able to carry-out their project with major changes.

 The main declared reasons why the EU funding was absolutely necessary are the

magnitude of the necessary funding and the lack of alternative forms of financing. A minority of respondents pointed-out the lack of the required expertise at national level, the impossibility of addressing pan-European issues nationally or

the need of infrastructures or databases.

 In terms of expected commercial outcomes (i.e. 15 respondents), amongst

respondents who said that the project could run without EU funding, 80% consider that it is likely that this would decrease their position internationally, 73% think that their access to new markets would be challenged and 53% estimated that it would be harder to keep or increase their market share in their

existing markets.

 Interestingly, only 30% of Horizon 2020-SC5 respondents foresaw a reduction of

emissions thanks to their project, 38% expect a reduction of waste and 43% a reduction in the use of primary resources. These percentages are nevertheless biased, because they are very strongly influenced by the instrument: CSAs, that

are not supposed to directly lead to positive environmental benefits, are overrepresented in the sample.

N.4.2. Expected longer-term results from the programme

This section analyses the results of FP7 – which is still ongoing – and then tries to estimate the expected outcomes and impacts of Horizon 2020.

Scientific results

63 Conducted by PPMI, specific contract under Framework Contract n°2012/S 144-240132.

64 SME Instrument Phase 1 projects are not included.

According to FP7 projects’ reporting 65 , the Cooperation Theme “Environment (including

Climate Change)” has delivered 7,166 scientific publications, of which 3,174 (44%) in high ranked peer-reviewed journals. This means that on average each project published 18.1 scientific papers, with peaks in areas like “climate change, pollution and risks” (37.2 publications per project), “protecting citizens from environmental hazards” (34 publications per project on average), “sustainable use and management of land and seas” (33.2) and “sustainable management of resources” (30.1).

On average, there are 59 publications per EUR 10 million invested, or 26 in high ranked journals. The area “climate change, pollution and risks” is the most performant when taking into account the excellence criterion: 54 papers in high ranked journals per EUR 10 million. The scientific excellence of the climate change research funded by the EU is also demonstrated by the relevant role of FP-funded projects in the IPCC reports, with

around 1,000 quotes in the 5 th one.

These data include publication in the highest rates journals, like Nature (46 papers, SRJ 66

14.5) or Science (44 papers, SRJ 11.2). However, most papers were published in journals with a SJR between 0 and 5, as shown in Figure 246.

Figure 246 - FP7-Environment publications, by journal rank (SJR)

Source: RESPIR, extraction 28/03/2017. SJR reference: 2011.

T he Ex Post Evaluation of FP7-Environment concluded that “FP7-Environment funding was more successful in supporting the creation of critical mass of good research, rather than in creating a worldwide lead for Europe in excellence in particular domains”. It mentioned the examples of Professors Novoselov and M.B. Moser and E.I. Moser, who were FP grantees before becoming Nobel Prizes. For the authors of that report, “FP collaborative projects create high level career opportunities for early-stage researchers,

supporting knowledge creation and sharing, international networking, etc” 67 . These

conclusions are still valid when most FP7-Environment projects are finalised.

RESPIR data show that the number of publications significantly depends on factors like:

65 RESPIR database, extraction: 28/03/2017. Projects with a processed final report: 397. Total number of projects:

494.

66 SJR - Journal Rank Indicator, it is a measure of journal's impact, influence or prestige. It expresses the average

number of weighted citations received in the selected year by the documents published in the journal in the three previous years (2011).

67 op.cit., p.33.

 The financial instrument: Collaborative projects performed much better than

CSA.

 The area of intervention: There are significant differences between priority areas,

in terms of publications per project, publications per EUR 10 million invested and high ranked publications per EUR 10 million of EC contribution.

 The scale of EC contribution.

 The number of researchers involved and the number of young researchers 68 .

Based on these trends, and considering the stronger Horizon 2020-SC5 focus on innovation, in particular through innovation actions that in principle are not expected to be highly productive in terms of publications, it is possible to anticipate a decrease in the

number of publications 69 . This trend may be mitigated by areas like Climate Change or

Earth Observation, which still include a number of research and innovation actions. These areas are characterised by very well-established research communities at international level, with strong structures (i.e. IPCC, GEO). Scientific advances are indeed critical to understand and fight climate change and other environmental pressures.

Patents

According to the RESPIR database, FP7-Environment, which is still ongoing, has produced 54 Intellectual Property Rights, of which 35 patent applications. Most of them came from the “Environmental technologies” area (25). The most performant organisations were the University of Stuttgart (5 patent applications), Acciona Infrastructuras (Spain) and the Dutch Stichting Energieonderzoek Centrum (4 each), the Italian CNR and the National Environment Research Council (3 each).

On average, there is one patent every EUR 34.9 million of EC contribution.

Innovation results and impacts 70

Neither FP7, nor Horizon 2020 collects adequate data on actual innovation results. The main traditional innovation proxy is indeed patents, but this vision is nowadays

challenged. 71 There is no systematic and comprehensive instrument to identify innovation

outputs and their economic (e.g. sales) and resource-efficient (e.g. energy or raw materials savings) impacts.

The Community Innovation Survey (CIS) carried-out by Eurostat 72 includes an item that refers to funding from the Framework Programme. 73 The analysis of the data from the

68 Ibid., p. 35.

69 The main objective of IA is not increasing knowledge, but demonstrations and pilots can lead to the collection of

many data to support scientific publications on specific eco-innovative solutions.

70 What follows is based on Alquézar, J. and Kwiatkowski, C. (2016) “The Community Innovation Survey and the

innovation performance of enterprises funded by EU’s Framework Programmes”, Open Innovation 2016 Conference Proceedings, Vienna, 24-25 November. Forthcoming.

71 See: Griliches, Z. (1990). “Patent statistics as economic indicators: A survey”. Journal of Economic Literature 28,

1661–1707, at: http://www.nber.org/chapters/c8351.pdf, Hall, B.; Thoma, G. and Torrisi, S. (2007) “The market value of patents and R&D: Evidence from European firms”. NBER Working Paper No. 13426, p.2. At: http://www.rady.ucsd.edu/faculty/seminars/2010/papers/hall.pdf , as quoted by op. cit., p.36

72 This bi-annual large scale survey (143,669 enterprises responded to the 2012 edition) provides harmonised data on

enterprises’ innovation activities and results by sector, size of company, type of innovation and the various stages of the innovation process: objectives, sources of information, investments, public funding, etc. The 2012 edition covered 13 EU Member States plus NorwaySee: http://ec.europa.eu/eurostat/web/microdata/community-innovation-survey . The CIS 2012 methodology is explained in detail at: http://ec.europa.eu/eurostat/cache/metadata/en/inn_cis8_esms.htm

CIS 2008, 2010 and 2012 (covering 2006 to 2012, almost the FP7 timeframe – 2007 to 2013) shows that innovative enterprises supported by FP7 perform better than those not financed by the programme:

 Between 73% and 80% of them introduced new products or services to the

market, compared with 32% to 43% of innovative companies not financed by

FP7.

 Between 67% and 71% introduced products or services new to the firm only,

compared with 42% to 59% of innovative companies not financed by FP7.

 Between 39% and 49% introduced processes new to the market, while the figures

are 13% to 20% amongst of innovative companies not financed by FP7.

These data also indicate that FP7-funded innovative enterprises performed best in exploiting new to the market products or services – especially compared with products and services that are just new to the firm. The significant correlations between FP7 participation and innovation performance do not necessarily mean causality. It could be assumed that the Framework Programmes attract R&I-intensive organisations, which are expected to be more innovative than the average. The innovation logic is likely to be circular, not linear: The Framework Programmes attract the most R&I intensive enterprises, which in turn improve their capacities thanks to collaboration in R&I at international level. The CIS shows also that participation in FP7 has a largely positive economic impact, measured in terms of turnover. Innovative firms supported by FP7 present a proportion of sales of new to the market products twice or three times higher than companies not funded by the Framework Programme. The difference is heavily influenced by the very large size of (some) enterprises that received FP7 support.

The CIS data permits also to characterise the FP7-funded enterprises that exploited their innovations in the market:

 Large innovative firms use to perform best in introducing products and services

that are new to the market. Differences are in most cases statistically significant, but not that high (e.g. in CIS 2008, 79% of large innovative enterprises funded by FP7 introduced a new product or service to the market, while the figure for SMEs

was around 70%; the gap is lower in successive CIS editions).

 The country where enterprises are based influences more the introduction of new

products, services or processes to the market or the firm than the size of enterprises. The Czech Republic, Spain, France or Portugal use to perform best in terms of introduction of new products or services to the market. Participating in FP7 has always a statistically significant positive impact, with sometimes a very strong influence. In Germany, for instance, the gap between FP7-funded and not funded enterprises is very significant, going from 23%-27% to 66% to 74%. In most cases, FP7 participation increases the performance of countries by 25% to

50%.

 The differences in terms of innovation performance between countries are lower

when they are supported by FP7. This suggest that the Framework Programme would play a cohesion role for innovative enterprises, as a consequence of

collaborative R&I activities.

73 Question 5.3 in CIS 2012.

 The Manufacturing sector (NACE C) provides the majority of new to the market

product innovations of the European economy, followed by Information and Communication (NACE J, around 10-14%) and by “Wholesale, retail and repair of vehicles” and “Professional, scientific, technical activities” (NACE G and M respectively, slightly less than 10% each). Within innovative enterprises supported by FP7, Manufacturing, Professional and scientific activities, and ICT cover alone 90% of the new products to the market. The manufacturing sector represents a share of more than 50%, Scientific and technical services around 25% and ICT 12-14%. However ICT enterprises use to be the most successful in

terms of introduction of new products to the market.

 Based on the CIS 2008 only, innovative companies supported by FP7 use to

deliver more environmental-friendly products and services. They introduced almost three times more eco-innovations (+183.5%) than enterprises that did not benefit from FP’s support. This is very relevant, considering that the societal

challenges orientation has increased with Horizon 2020.

These results correspond to the whole FP7. The CIS does not allow to identify projects by FP7-Cooperation Theme/Horizon 2020-Societal Challenge. Taking into account these caveats and under different assumptions, it can be calculated that FP7 is catalysing a leverage effect (defined as “increase of sales per euro invested”) between 27 and 68,

depending on the chosen scenario (“prudent” or “optimistic”) 74 .

According to the Horizon 2020-SC5 continuous reporting of projects, there would be already 109 prototypes and 101 innovations introduced in the market. There is no possibility to assess the reliability of these numbers, since projects provide just numbers and no details about those innovations or prototypes.

In terms of direct crowding-in 75 of funding, Horizon 2020-SC5 is directly mobilising

EUR 325.3 million (i.e. 1 euro per each 3.4 of EC contribution), of which 29.6% from the private sector (private for profit organisations). Unsurprisingly, the maximum leverage appears in ERA-NET Cofund actions (EUR 1 for each EUR 2.4 of EC contribution), followed by Innovation Actions (EUR 1 for each EUR 5.3 invested by the EU). This reflects to maximum rate of reimbursement of eligible costs identified in the Work Programmes 2014-2016.

This leverage is just the difference between the cost of projects and the EC contribution. It does not include further investments that solutions developed by projects are expected to obtain in order to exploit their innovations in the market. The general objective stated in article 5 of the Horizon 2020 Regulation (see section N.1.2: “[…] by leveraging additional research, development and innovation funding […]”) can hardly be limited to this direct crowding in. The Commission services do not collect data about further investments committed to further develop innovations, even if this factor is critical to ensure their commercial exploitation.

N.4.3. Progress towards attaining the specific objectives

The detailed specific objectives of Horizon 2020-SC5 are presented in section N.1.2. As mentioned above, it is too early to describe actual results. What follows is further

74 See the forthcoming Interim Evaluation of Horizon 2020-SC5.

75 Defined as the amount of funding contributed by the stakeholders to the project, matching the EC contribution. In

other words, the difference between the project cost and the EC contribution. The data includes the SME Instrument Phase 1.

developed in sections N.1.2, N.2.1 and N.2.2 (intervention and implementation logic). So far, Horizon 2020-SC5 is being able to change traditional R&I approaches, making more links between science and innovation through the development of new markets (e.g. climate change services, nature-base solutions) through a systemic approach (implying multi-disciplinarity and a challenge-driven, solutions-oriented vision).

Fighting and adapting to climate change: The current implementation strategy has three main objectives: (i) continue to increase knowledge on climate change, (ii) transfer this knowledge to policy through the IPCC and international negotiations like COP21 and COP22, while increasing public awareness; and (iii) develop solutions for climate mitigation and adaptation. In terms of concrete results, Horizon 2020-SC5 projects are expected to deliver high quality publications, quoted in the IPCC reports, and to develop the emerging market of climate services.

Protecting the environment, sustainably managing natural resources, water, biodiversity and ecosystems: The Horizon 2020-SC5 implementation strategy tries to go beyond the traditional focus on the water and waste sectors. It is in particular promoting nature-based solutions, both in cities and for territorial resilience, through large-scale demonstrations and pilots. Water has been mainstreamed in different calls in the Work Programme 2016-2017, which is consistent with the systemic innovation approach but is decreasing the visibility of the topics. The current approach is expected to strength links between different stakeholders (innovators, researchers, public authorities, NGOs), to demonstrate the positive impacts of the solutions funded and to improve the competitive position of European actors in nascent markets. In addition, the Commission is still supporting the development of knowledge in areas like natural capital and ecosystem services, where there is already policy impact (see section N.6.3)

Ensuring the sustainable supply of non-energy and non-agricultural raw materials: Horizon 2020-SC5 actions in raw materials are expected to contribute to the objectives of the Raw Materials Policy and the EIP on Raw Materials. In this regard, the most important goals are: to push Europe to the forefront in raw materials sectors, to reduce import dependency, to improve supply conditions from European and other sources, to provide alternatives in supply; and to mitigate negative environmental and social impacts of the raw materials activities. Other important objectives are to improve the awareness, acceptance and trust of society in a sustainable raw materials production in the EU and to increase the EU knowledge base of primary and secondary raw materials. Furthermore, most actions aim to improve the competitiveness of raw materials and/or downstream and related industries and creating added value and new jobs.

Enabling the transition towards a green economy and society through ecoinnovation: The Horizon 2020-SC5 implementation strategy is not focused on specific (technological) innovations. It follows a systemic approach, mainly through large-scale demonstrations that are expected to be replicable. This approach is expected to prove on the ground the economic and environmental feasibility of eco-innovative solutions and increase their societal impact. It contributes to relevant policy areas, and in particular to the implementation of the Circular Economy Action plans.

Developing comprehensive and sustained global environmental observation systems: Horizon 2020-SC5 actions should lead to (i) the continuous development of technologies and knowledge on earth observation, and (ii) to maximising the benefits of those technologies and the cumulated knowledge for European citizens. They are expected to develop new services (e.g. climate services, disaster risk management services, water or land management) that capitalise on the current scientific and technological state-of-the-art. Special emphasis is given to empowering citizens through “citizen science” tools (e.g. apps to measure, share and communicate on air quality, noise, alien invasive species or other environmental issues).

Cultural heritage: Actions are expected to demonstrate that cultural heritage is a driver of sustainable growth factor and not a cost. In terms of concrete outcomes, projects are expected to deliver urban and rural regeneration examples, demonstrating the economic, social and environmental benefits of the interventions.

Specific implementation aspects (i.e. coordination with IPCC, IPBES, GEO, EIPs, EIT): See section N.6.

The Commission services have not identified specific factors hindering dramatically progress towards the expectations. However, they are aware that the proposed new approaches have to be well explained to traditional stakeholders, for instance in Info Days, workshops and through collaboration with the National Contact Points.

Traditional academic stakeholders continue to play an important role in many of the funded actions. This is reflected, in particular, in the innovation ones (IAs) where many proposals and actions still include a considerable number of research activities. Participation data seem to confirm that academia continues to receive play a central role, but they also show an increasing trend in the participation of enterprise compared with FP7-Environment.

Several proposals to the first Horizon 2020-SC5 calls still include attempts to force research activities in the form of an Innovation Action, with different degrees of success. However, in the most recent calls, it appears that the traditional Horizon 2020-SC5 community is better responding to allocate considerable parts of the budget of proposals and actions to clear innovation activities, such as large scale demonstrators, and to realscale applications, looking at potential market uptake. Indeed, despite its novelty, the concept of Innovation Actions becomes quite successful, in terms of response by applicants (huge oversubscription) and ambition of the actions proposed. This positive trend could be explained by an initial inertia of the traditional FP7 community to go beyond business as usual.

EASME is now implementing the 4th year of Horizon 2020 Work Programmes, and the efforts in communicating its innovation dimension are giving their results. Despite this positive trend, there is still room for improvement for Horizon 2020 to attract new kinds of innovators, e.g. social innovators, and the large industries and SMEs that are key for the implementation of large-scale demonstrators. In addition, for some topics, the R&D maturity of the field might not be enough to support an IA and further research could be needed.

Another issue that is constantly raised by stakeholders is the complexity of the Work Programme – and even of Horizon 2020 itself. It would be difficult to find relevant topics and to understand the rationale of the programme, its instruments, JTIs, PPPs, Article 185 initiatives, management modes, responsibilities, etc.

N.4.4. Progress towards the overall Horizon 2020 objectives

N.4.4.1. Fostering excellent science in scientific and technological research

The Horizon 2020-SC5 Work Programmes try to reach a balance between innovation and research. There are still topics and domains where “pure science” is required, like climate change or earth observation. It is not by case that projects like ECOPOTENTIAL or AtlantOS are already delivering many papers.

Box 22 - Contribution to the achievement and functioning of the ERA

Horizon 2020-SC5 actions are contributing to support the achievement and functioning of the

ERA, through:

• More effective national research systems: See section N.6.2. Also the project NCPs CaRE

(CSA, EC contribution: EUR 2 million) aims at enhancing cooperation and networking

between National Contact Points (NCPs), in order to increase the quality of their consulting

services and thus of proposals and projects. NCPs CaRe facilitates knowledge sharing between these key national entities, which is particularly relevant for countries which R&I

organisations are not particularly successful in Horizon 2020. This should ensure a more

efficient use of resources and R&I developments by improving the workflow between NCPs,

applicants, the Commission, and other parties with a stake in SC5.

• Optimal transnational co-operation and competition: See section N.6.1. The forthcoming

Article 185 initiative PRIMA (see Box 23 , section N.3.2) is another example of

dissemination of knowledge, beyond the European borders, through co-design and codevelopment

of solutions.

• Optimal circulation and transfer of scientific knowledge: All Horizon 2020 publications must

be in open access, and Horizon 2020-SC5 participates in the pilot on open data. Beneficiaries are invited to share their research data also in open access.

• There are no specific actions to support an open labour market for researchers. Indeed, data

are not collected to measure systematically the impact of Horizon 2020 of researchers’ mobility, as it used to be the case under FP7.

• The same applies to the ERA objective “gender equality and gender mainstreaming in

research”: FP7 collected detailed statistics about women participation in projects. Under

FP7, women represent 37.1% of the staff only, and indeed, they are less likely to occupy

posts of responsibility (e.g. coordinators, work package leaders) than men. Right now, under

Horizon 2020, only the proportion men/women is measured. This is a big caveat to address

gender issues in EU-funded R&I.

N.4.4.2. Boosting innovation, industrial leadership, growth, competitiveness and job creation

The approach to increase innovation, industrial leadership, growth, competitiveness and job creation has been explained, for each specific objective of Horizon 2020-SC5, under section N.4.3. Participation of private-for profit organisation is increasing compared with FP7-Environment.

An estimate of the possible impact of Horizon 2020 on innovation, based on the CIS 2008, 2010 and 2012 data and under several assumptions, has been presented in section N.4.2.

N.4.4.3. Addressing the major societal challenges

The (expected) contribution to Horizon 2020-SC5 has been explained in section N.4.3.

In addition, as described in section N.1.1, Horizon 2020-SC5 contributes to other societal challenges, mainly through its contribution to cross-cutting Focus Areas.

N.4.4.4. Spreading excellence and widening participation

The participation data show that the systemic approach that Horizon 2020-SC5 promotes is being successful in enlarging the involvement of stakeholders like enterprises or NGOs. The Ex Post Evaluation of FP7-Environment considered their participation too low (see section N.2.3.1).

N.4.4.5. Science with and for society

The systemic approach that Horizon 2020-SC5 is promoting is based on co-creation. New markets, like nature-based solutions, cannot be successful is they are not jointly developed by all socio-economic players, including the research community, industry,

public authorities and, of course, citizens. 76 This is particularly important in calls on

nature-based solutions, like in cities. The history of architecture and urbanism is full of false good ideas that did not correspond to citizens’ needs and desired life-styles.

Horizon 2020-SC5 continues to support citizens’ science actions, capitalising on the results of FP7 projects (i.e. MyGeoss, Citizens Observatories). The goal is to empower citizens, providing them tools to measure and share, through apps, environmental parameters like air quality, noise, alien invasive species, etc. Citizens’ science is not something new. Statistics on birds are using citizens’ observations for decades. However, digital tools open new perspectives. Any person with a smart phone can make a geolocalised environmental measure, or share the photo of an animal or plant, and this information can immediately be checked centrally and/or by other users, to elaborate statistics and take measures to address the problem. This approach has a strong potential for reducing the environmental monitoring and reporting burden of public

administrations and enterprises – in the framework of the ongoing Fitness Check. 77 It is

also relevant to facilitate the compliance of legislation, with a two-ways relationship between citizens and public administration. It is gaining a political momentum, with a

very active European Citizen Science Association (ECSA). 78

N.4.4.6. Science for policy

The Commission services in charge of Horizon 2020-SC5 have strengthened their links with traditional policy DGs in order to increase the policy impact of R&I projects. For this, the Commission is implementing a new approach: the Environmental Knowledge Community (EKC). The evaluation of FP7-Environment showed that R&I results were not always directly exploited in policy, for a series of reasons (timing, changing priorities, new staff, etc.). To address this challenge, DG Environment, DG CLIMA, JRC, DG RTD, Eurostat and the EEA launched in 2014 the EKC, with the aim of "working in a more structured, strategic and collaborative way for the development of

76 See Horizon 2020 Work Programme 2016-2017, chapter 12, page 5.

77 European Commission (2016) Staff Working Document: Towards a Fitness Check of EU environmental monitoring and reporting: to ensure effective monitoring, more transparency and focused reporting of EU environment policy, 27/05/2016, SWD(2016) 188 final.

78 http://ecsa.citizen-science.net/

knowledge would help deliver better results in a more timely way, using fewer resources". Within the EKC, each participant is at the same time knowledge user and a knowledge provider. There is not anymore the traditional distinction between “policy DGs” (i.e. DG ENV, DG CLIMA), knowledge funders (DG RTD) and knowledge providers (JRC and EEA). The EKC is further described in section N.6.3.

N.4.4.7. Early success stories

Even if it is too early to present success stories, some promising projects can be presented. Other examples are described in section N.7.2.

 SMART-Plant (Scale-up of low-carbon footprint material recovery techniques in

existing wastewater treatment plants): Innovation Action; topic: WATER-1b 2015; EC contribution: EUR 7.5 million; total budget: EUR 9.7 million; 25 partners. This project will enable the recovery from water of 3 million tons per year feedstock for chemicals (e.g. phosphorous) and its downstream processing industry with a value higher than EUR 500 million. The figure could increase to more than 50 million tons per year extending SMART-Plant concept to biowaste processing. The expected clients are mainly the water utilities in Europe and world-wide, as well as the chemical industry. This project contributes to the circular economy, creating new business models and partnerships. The chemical industry see traditionally the water sector as a customer, while with the recovery of raw materials, the two sectors will become both customers and providers.

 POWERSTEP (Full scale demonstration of energy positive sewage treatment

plant concepts towards market penetration): Innovation Action; topic: WATER- 1a-2014; EC contribution: EUR 4 million; total budget: EUR 5.2 million; 12 partners. The objective of this project is to convert sewage treatment plants (STEPs) into power production facilities (POWER). For this, the partners will design and demonstrate energy positive wastewater treatment plants with available technologies in 6 full-scale case studies located in four European countries. The estimated benefits are energy savings: EUR 1,7 billion /annum; CO2 – equivalent emission savings: 5,9 million tons; and global market value: $ 30 billion/annum.

 RESYNTEX (A New Circular Economy Concept from Textile Waste towards

Chemical and Textile Industries Feedstock): innovation action: topic WASTE-1- 2014; EC Contribution: EUR 8.8 million; total budget: EUR 11.4 million; 20 partners. This innovation action will demonstrate in a pilot textile recycling plant of 500 ton/year how to avoid the incineration and landfilling of textile waste through chemical recycling of unwearable blends. It will create industrial symbiosis with the production of usable feedstock and it will ensure market acceptance for products made of textile waste. It aims to reach a 50% collection rate all over Europe, to convert 6.2Mt of fibres per year into new feedstock by 2020, to reduce final waste disposal at around 5%, and pollutants emissions, contributing to reduce the total average global warming potential by 1.5%.

 ESMERALDA (Enhancing ecoSysteM sERvices mApping for poLicy and

Decision mAking): Coordination and Support Action; topic: SC5-10a-2014; EC contribution: EUR 3 million; total budget: EUR 3.1 million; 25 partners. ESMERALDA is an example of success in terms of policy impact. It is already

delivering a flexible methodology to provide the building blocks for pan European and regional assessments of ecosystem services. The work ensures the timely delivery to EU Member States in relation to Action 5 of the Biodiversity Strategy, supporting the needs of assessments in relation to the requirements for planning, agriculture, climate, water and nature policy. It capitalises on previous FP7 projects (OpenNESS, OPERAs), national studies and supports the work of the MAES group. The ongoing work creates the basis for a future European natural capital and ecosystem services accounts system, in line with the System of

Experimental-Economic Accounting (SEEA-EEA) 79 .

 ¡VAMOS! (¡Viable and Alternative Mine Operating System!) is a Research and

Innovation Action funded under topic SC5-11a-2014, "New solutions for sustainable production of raw materials". Estimates indicate that the value of unexploited European mineral resources at a depth of 500-1,000 metres is ca EUR 100 billion, however, a number of physical, economic, social, environmental and human constraints have until now limited their exploitation. ¡VAMOS! is developing a new Safe, Clean and Low Visibility Mining Technique and will prove its economic viability for extracting currently unreachable mineral deposits, thus encouraging investment and helping to put the EU back on a level playing field in terms of access to strategically important minerals. The design of the novel underwater mining system has already been completed and the assembling part has started well ahead of schedule, the advanced virtual reality remote piloting system is being tested in a nearly industrial environment. The project directly supports the EIP on Raw Materials and its Strategic Implementation Plan by developing equipment able to unlock access to a substantial volume of various raw materials within the EU, including critical raw materials, in a sustainable way. The project is also dedicated to disseminating results among various stakeholders and increasing the social acceptance of the

new extraction technique via public demonstrations in EU regions.

N.4.5. Lessons learnt/Areas for improvement

It is still too early to assess the actual effectiveness of Horizon 2020-SC5. There are few available data on outputs from ongoing projects, because they are all in their initial phases. Indeed, the analysis of information available in CORDA shows a number of inconsistencies, both with output and input data. Having reliable and easy to access data is a key challenge for the monitoring and evaluation of Horizon 2020 in its last years. Under these conditions, effectiveness has been estimated based on the extrapolation of FP7 data, or qualitatively (e.g. “expected outcomes and impacts”, “perceived problems”).

The implementation strategy of Horizon 2020-SC5 may lead to:

 Proportionally less publications than under FP7-Environment, as a consequence

of the strong focus on innovation and the higher involvement of non-academic

beneficiaries.

 Exceptions to this trend can be areas like climate change and earth observation,

which still include a number of mainly scientific topics. They are also characterised by very well-established research communities at international level, with strong structures (i.e. IPCC, GEO). Scientific advances in these areas are critical to understand and fight climate change and other environmental

pressures.

79 United Nations, European Commission, FAO, OECD and the World Bank (2014) System of Environmental

Economic Accounting 2012. Experimental Ecosystem Accounting. New York: United Nations.

 A likely decrease in patent applications, because topics are not anymore focused

on developing specific technologies, but on large-scale demonstrations. This

expected decrease could be mitigated by SME Instrument Phase 2 projects.

 A higher innovation impact than FP7-Environment, as a result of the increasing

participation of enterprises.

 A stronger policy impact than previous Framework Programmes, especially of

those actions that are embedded in policy processes at European level (e.g. EKC, MAES, Circular Economy) or internationally (e.g. IPCC, Climate Change

Conferences of the Parties).

 An expected higher societal impact, as a result of large-scale demonstrations and

pilots. This approach use to be considered a way to deploy business models and bring R&I results to the market, e.g. by facilitating the risky up-scale of technology, and/or supporting “launching” customers. The Horizon 2020-SC5 demonstrations not only encompass technological pilots or demos. As they are based on co-creation and multi-disciplinarity, they should benefit from other forms of innovations (organisational, social, etc.). Therefore their impact is

supposed to be potentially wider.

This rationale is already encountering some obstacles that cannot be considered critical (yet) to reach the objectives: initial resistance to change of traditional (academic) stakeholders, the difficulty to attract new kinds of innovators (despite the fact that participation is widening), or the insufficient maturity of the “markets of the future” that Horizon 2020-SC5 tries to promote. These are nevertheless hypotheses based on anecdotal evidence, and trends show a positive evolution. They correspond to barriers typically encountered in R&I, when “first-of-its-kind” projects are financed.

More relevant is the stakeholders’ critique about the extreme complexity of Horizon 2020: too many intervention areas, too many funding instruments, too many objectives, too many references to specific policies, etc..

The structural complexity of Horizon 2020 is then translated to the Work Programme, the core operational tool for stakeholders interested in Horizon 2020 support. The Commission services are aware about this issue and are committed to address it in the design of the Work Programme 2018-2020 which is under preparation.

N.5. E FFICIENCY

Simplification is one of the keywords of the Horizon 2020 Regulation 80 : The need for

(further) simplification is a historical request to the Framework Programmes’ stakeholders. It is linked with the wider criterion on efficiency, i.e. the relationship between the resources used by an intervention and the changes it generates. As explained in section N.4, it is too early to evaluate the actual outputs, outcomes and impacts of Horizon 2020-SC5. All projects are ongoing and in their initial phases. There are few data on concrete results, basically some papers and patents. Under these conditions, this section focuses only on the observed or perceived results of the simplification measures that have been implemented. Basically:

80 Horizon 2020 Regulation, Recital (20).

 Simplification of procedures and timings (e.g. time to grant, time to pay), in line with Recital (43) of the Horizon 2020 Regulation. 81

 New governance structures, i.e. delegation of the projects management.

N.5.1. Budgetary resources

Horizon 2020-SC5 is jointly coordinated by DG RTD and DG GROW (for raw materials). Since the second stage evaluations of the Work Programme 2014, the vast majority of projects are evaluated and managed by EASME. The few exceptions are projects selected under topics explicitly excluded from delegation in the Work Programmes, because the activity supports the development and implementation of evidence-base for R&I policies and/or supporting various groups of stakeholders.

The budget has been executed so far in line with expectations. The only deviation concerned the SME Instrument Phase 1 in 2014 and 2015. The data presented in section N.2.2, show that, in 2014 and to a lesser extent in 2015, the actual funding of the SME Instrument exceeded the budget foreseen in the Work Programme. In 2014, the deviation was 13.5% (mainly coming from the Phase 1: 76.4%!), while in 2015 it was 3.9% (23.7%

for the SME Instrument Phase 1). 82 Such deviations from the budget allocated are within

the limits allowed by the legal base (+/- 20%),

The breakdown of the budget by financial instrument, sort of beneficiary and country is presented and analysed in section N.2.2.

The current average EC contribution to projects is EUR 5.02 million (SME Instrument Phase 1 excluded). In FP7-Environment, it was EUR 3.5 million. This represents an increase of 42.9%. If one assumed that the average EU funding per project under FP7- Environment corresponded to a typical project in 2010 (FP7’s central year), its net

present value 83 would be EUR 4.4 million, lower than the current average under Horizon

2020-SC5. This means that either Horizon 2020-SC5 projects are more ambitious with similar financial resources, or the simplification measures are not leading to a more efficient use of resources. This is a critical indicator to assess in the Ex Post Evaluation of Horizon 2020.

N.5.2. Programme's attractiveness

N.5.2.1. Mobilisation of stakeholders

The very low success rate (11% overall, under 10% for IA and RIA) indicates that Horizon 2020-SC5 is very attractive to stakeholders. Despite of its complexity, the programme is attracting 43.2% of newcomers. It shows an increasing participation of industry and “other” organisations, compared with FP7-Environment (see Table 198).

However:

81 “(…) The European Parliament, in its resolution of 11 November 2010 on simplifying the implementation of the

Research Framework Programmes, called for a pragmatic shift towards administrative and financial simplification and stated that the management of European research funding should be more trust-based and risk-tolerant towards participants. (…)”

82 Annex 1 shows the details of the SME Instrument evaluations for SC5.

83 Based on the annual inflation, as provided by Eurostat.

 There are external factors leading to a low success rate, like the decreasing

availability of national and sub-national R&I funds in several countries, as a consequence of the financial austerity measures. Horizon 2020, which budget instead increased despite of the crisis, becomes a solution for a number of R&I

teams.

 Such low success rates imply that a lot of resources are spent for preparing

unsuccessful proposals. This is a problem for the R&I community, especially in countries with limited national resources (e.g. EU-13).Section N.3.3 presents the main dissemination activities (conferences, workshops) carried-out to promote Horizon 2020-SC5, explain its objectives and procedures and to receive feedback

from stakeholders. There are no data to quantify the audience to these events.

Horizon 2020-SC5 is also active in social media (Facebook, Twitter, Yammer) where for instance success stories are presented. The web statics show:

 More than 150,000 visits to the DG RTD’s Directorate I, “Resource efficiency

and climate action”, and more than 230,000 page views per year.

 More than 45,000 annual visits to the Horizon 2020-SC5 page, with almost

65,000 page views in 2015.

 Almost 65,000 views of the Horizon 2020-Resource Efficiency and Climate

action page in 2015 (38,000 in 2014).

 Around 60,000 visits per month on average to the EASME website in the first 9

months of 2016 only.

As previously commented in this document, there is a high level of correspondence between the communities of EIP Raw Material Commitments and Horizon 2020 projects on raw materials. Besides, as expected from the published work programme, the selected RIA (and also the IA selected in mid-October) are all implemented by industrially driven consortia, with participation of downstream and end-user partners. Strong participation of the industry in proposals and funded projects is seen as very positive for this part of the programme.

N.5.2.2. Geographical dimension

The ongoing participation per country and group of countries on 1 st January 2017 is

presented in Table 202 and analysed in section N.2.3. The most relevant indicator is the number of participations, not biased by different national purchase parities and by the fact that some countries, in principle, do not receive EU funding. Compared with FP7- Environment, one may observe:

 An increase of EU-13 participation (8.3% in FP7-Environment vs. 9.4% in

Horizon 2020-SC5);

 A decrease of participation of Associated and Candidate Countries (8.2% in FP7-

Environment and 6.7% in Horizon 2020-SC5), partially due to the fact that

Switzerland was not associated until 2017 (2.7% in FP7-Environment);

 A decrease in the participation of Third countries (9.2% in FP7-Environment,

4.9% in Horizon 2020-SC5). This happens because some countries (e.g. China,

Brazil, India, Mexico) are in principle not funded anymore by Horizon 2020 84 .

There is also still room for improvement in terms of participation of partners from EU-13 and Third countries in raw materials projects. Additional dissemination activities are being put in place to encourage the participation of stakeholders from those countries.

N.5.2.3. Cross-cutting issues

Cross-cutting issues are discussed in section N.2.4. The main conclusions are:

 Horizon 2020-SC5 is one of the best performers on Sustainable Development and

Climate expenditure, as expected considering the nature of the subjects that are covered. The actual impacts of such investment, as well as the adequacy of the

tracking methodology, should be analysed at the end of Horizon 2020.

 There are reasonable doubts about the data for assessing gender, SSH and the

Digital Agenda. For example, gender issues identified under FP7, such as the low involvement of women in posts of responsibility, is harder to assess under

Horizon 2020.

 In any case, it seems that Horizon 2020-SC5 is being rather unsuccessful in

mobilising the SSH community to participate in excellent proposals, despite the

reasonable number of topics flagged as SSH relevant.

N.5.3. Cost-benefit analysis

A proper cost-benefit analysis cannot be conducted until there will be a significant amount of outputs from projects. However the ongoing experience indicates some issues and risks to be addressed.

If Horizon 2020-SC5 should finance all excellent proposals, its budget should increase very substantially. The very low success rate (“over-subscription”) is a proof of attractiveness, it is supposed to increase the projects' quality (through increased competition), but it also implies that huge resources are spent to prepare and assess unsuccessful proposals. Under these conditions, it is critical to guarantee the best evaluation of proposals, with the most adequate experts.

Another issue is the complexity of the programme, translated to the Work Programmes. However, despite this issue, Horizon 2020-SC5 is being able to attract a significant number of newcomers, including from enterprises – one of the weaknesses of FP7- Environment.

The 123 existing Raw Materials commitments (which cannot be directly funded by the EIP) have an indicative budget of close to EUR 2 billion. Considering these figures, the budget allocated for the seven year -period (near EUR 600 million) seems acceptable, but modest to reach the ambitious targets of the EIP (e.g. 10 large scale pilot actions by 2020). Indeed higher figures would be positive to consolidate the efforts made in previous years, to fund more high-quality projects and increase the effectiveness of actions on raw materials.

84 The Work Programme nevertheless allows exceptional funding, when the participation of organisations from those

countries is considered essential. This explains there rarely countries, without any EC contribution. Mexico is the main exception.

Supporting innovative approaches like Nature-Based Solutions or Climate Services implies a risk of deviating too many resources to conceptual development, instead of focusing on “core” R&I actions, with the expected socio-economic and environmental impact. On the other hand, accompanying measures are necessary, for example to create networks of practitioners. Finding the right balance is one of the challenges of this approach.

N.5.4. Other issues related to efficiency

For the Horizon 2020-SC5 management, the two main relevant delegation bodies are EASME and the Common Support Centre (CSC). The CSC plays a critical role for the monitoring and evaluation of Horizon 2020, since it is in charge of defining the business process and of the IT resources to collect and handle data from proposals and projects

(e.g. databases like CORDA). 85 The reliability of such data is very important to allow the

Commission to develop and implement an evidence-base policy, for its relationship with stakeholders (for instance, the Programme Committee) and for transparency vis-à-vis European citizens.

The high percentage of projects which time to grant (TTG) is signed before the target

(88% within 245 days 86 ) indicates that EASME is managing the “core” evaluations and

grants preparation according to the expectations. It is worth noting that the targets are very demanding and there is almost no margin of manoeuvre to reduce the time to grant, as explained in section N.2.2. :

 Even if there are less redress procedures than under FP7-Environment, more are

justified, with three re-evaluations so far (versus one for the whole FP7- Environment).

 There is a relatively high number of amendments, due to (i) the fact that proposals

are not anymore negotiated and (ii) the need to include corrections in the

contracts, signed under time pressure.

An additional concern relates to the information flow between EASME and the Commission services. The Commission needs information from projects to elaborate its policy, while the executive agency should be aware about policy developments in order to provide the most adequate information. This relationship could still be improved.

A fluid exchange of information is particularly relevant for DG GROW, in charge of raw materials within the wider EIP framework. Due to its current staffing conditions, DG GROW is able to follow relatively closely the CSAs, given their policy relevance. For other projects (RIAs, IAs), the DG relies almost completely on EASME.

Regarding the evaluation activities themselves, the High Level Expert Group for the Ex Post Evaluation of FP7 observed and recommended that "(…) evaluation activities have been considered as routine activities in recent years (…). Considering that the Framework Programme have consistently been the third largest budget of the European Union, a strategic and professional monitoring and evaluation system is required that

85 The whole Horizon 2020 is managed with the same IT tools. Application forms are similar and a common IT

infrastructure is in place: SEP for submission and evaluation, SYGMA and COMPASS for grant management, PDM for participant management; and CORDA for storage and reporting tool.

86 The figure includes projects managed by DG RTD. The actual figure for EASME would be 85.7% (or 238.4 days on

average). With two exceptions, the remaining 14.3% of contracts managed by EASME were signed as a maximum 10 days after the 245 days targets. These data do not include the SME Instrument.

increases transparency and serves as a comprehensive and trusted source of evidencebased

decision making". 87 This is however still not the case. Under Horizon 2020, the set

of output indicators is limited to those defined in the legal base, which are insufficient for a proper monitoring and evaluation of the programme. A better monitoring system is particularly important considering the reliability problems of current databases, the increasing complexity of the programme and the inexorable trend to reduce projects’ administrative and reporting burden.

N.5.5. Lessons learnt/Areas for improvement

The implementation of Horizon 2020 is characterised by a strong focus on simplification, implement through easier procedures for applicants and beneficiaries, and through new management modes (delegation to Executive Agencies). The net present average cost of Horizon 2020-SC5 projects, nevertheless, remains higher than under FP7-Environment. There are no project results yet to analyse the effectiveness of these measures in terms of outputs, outcomes and impacts (“cost-benefit analysis”).

In the case of Horizon 2020-SC5, the delegation of implementation tasks to the Executive Agencies proceeded well. EASME has been so far able to comply with difficult targets (e.g. time to grant) and procedures. However the formal information flow between the Commission and the executive agency could improve – in both directions.

The main efficiency concerns relate to over-subscription. Horizon 2020 is attractive, maybe too attractive in the current context. Huge R&I resources are spend to prepare and evaluate unsuccessful proposals, often excellent ones.

The monitoring system of the programme remains weak. This is due to factors like the complexity of Horizon 2020 and its governance structures, the low quality of data provided by applicants and beneficiaries and the only recent creation of the Common Support Centre. A monitoring system still under development implies difficulties for the Commission to capitalise of projects’ results to develop its policy.

N.6. C OHERENCE

The general objective of the Horizon 2020 Regulation (art. 5, §1, see section N.1.2) puts emphasis on the necessity of “leveraging additional research, development and innovation funding”. Horizon 2020 alone cannot tackle global issues like climate or environmental challenges; it must be coordinated to other sources of funding and policy actors. This is indeed further underlined in article 13 of the Horizon 2020 Regulation (“synergies with national programmes and joint programming”), article 20 (“complementarity with other Union programmes”) and in article 21 (“synergies with ESI Funds”, i.e. Structural and Investment Funds).

The alignment strategy of the Commission services in charge of implementing Societal Challenge 5 can be analysed at different levels:

 International: To what extent is SC5 consistent with international R&I actions?

 Alignment between SC5 and Member States actions.

 Synergies with other policies.

87 Martinuzzi et al. (2015) op.cit., p. 9.

 Synergies with other EU programmes.

 Synergies with other parts of Horizon 2020.

N.6.1. International coherence

Until 31 December 2016, the European Commission co-chaired the Belmont Forum, an international forum of agencies that finance environment-related R&I. It remains member of its Steering Committee and contributes to the secretariat. The partners select areas where common efforts are needed, in order to ensure synergies. For instance, in 2017 calls will start around the following themes: food-water-energy nexus for sustainable urban development and societal transformations to sustainability. Four more themes will follow in 2018 and 2019: biodiversity, e-infrastructures, climate change impact modelling and oceans. Eventually, three additional themes are under discussion for possible funding: food security, water and disaster risk reduction. For funding under Horizon 2020, CRAs can be implemented directly or via ERA-NETs, according to the normal Horizon 2020 rules. The Work Programme 2016-2017 includes three Belmont Forum topics: SCC-04-2016, “Sustainable urbanisation” (EUR 5 million); SC5-28-2016, “Transformations to sustainability” and SC5-32-2017 (EUR 3 million), “Biodiversity scenarios” (EUR 7 million). Box 23 provides more detailed information.

Box 23 - The Belmont Forum

The Belmont Forum is an international partnership of research funding organisations created in 2009 invested in interdisciplinary and transdisciplinary research to inform human action and adaptation to global environmental change. The Forum supports multi-national collaborative projects which combine approaches from natural sciences, social sciences and the humanities, as well as local knowledge to address grand research challenges in areas such as food security, ecosystem services, freshwater security, and coastal resilience. The underpinning consideration is that the advances needed to adequately address global environmental challenges benefit from international science cooperation and co-ordination – which corresponds with Horizon 2020 goals and logic. Its White paper revision of 2016 mainstreams science, technology and innovation supporting sustainable development and the Sustainable development goals.

The Belmont Forum gathers a (steadily growing) number of major research funding agencies and scientific councils. Today members are twenty five funding agencies from twenty three countries: MINCyT (Argentina), CSIRO (Australia), BMWFW (Austria), FAPESP (Brazil), NSFC (China, PR), MoST (Chinese Taipei), EC (EU), Allenvi and ANR (France), DFG and BMBF (Germany), MoES (India), IAI (Inter American Institute for Global Change Research), CNR-DTA (Italy), MEXT and JST (Japan), CONACyT (Mexico), NWO (The Netherlands), RCN (Norway), QNRF (Qatar), RFBR (Russia), NRF (South Africa), FORMAS (Sweden, replaces SSEESS as from 1/01/2017)), Tübitak (Turkey), NERC (UK), NSF (USA). Five international scientific associations are non-funding partners: GEO, ICSU, IIASA, ISSC and USGCRP.

The Belmont Forum funding mechanism turns around "Collaborative Research Actions - CRA", whose objectives are agreed upon by the participating funding Agencies.

Until 31 December 2016, the European Commission co-chaired the Belmont Forum with the Sao Paolo Research Foundation, FAPESP. It remains member of its Steering Committee.

For funding under Horizon 2020, Collaborative Research Actions (CRA) can be implemented directly or via ERA-NETs, according to Horizon 2020 rules. The current experience with ERA-NET Cofund shows that this instrument has the flexibility to contribute to CRAs, but some improvements would be welcome. For example, adjusting the calendar for scoping the actions and adapting the eligibility criteria of the joint calls. This is particularly important to open Horizon 2020 and EU R&I to the world, especially if one considers that the financial leverage effect of ERA-NET Cofund launched in connection with the Belmont Forum reach factor 5 and that so far more than 40 countries are involved in Belmont Forum Actions.

The European Commission also co-chairs the Group on Earth Observations (GEO). GEO is an intergovernmental partnership composed of 102 nations and the European Commission, plus 103 Participating Organisations with a mandate in Earth observation. Members of this partnership develop joint activities on a best effort basis to address Societal Benefit Areas (SBAs) of global relevance and coordinate their strategies on Earth observation. Societal Challenge 5 contributes through calls for proposals specifically addressed towards maximising the benefits for European citizens of the Earth observation infrastructure by developing innovative services that support more sustainable production and consumption patterns and resilient societies – something strongly demanded in the stakeholder consultations to design the Work programmes. SC5 actions in Earth observation aim to implement the Global Earth Observation System of Systems (GEOSS) and complete the in-situ and service components of Copernicus. They use a range of Horizon 2020 instruments: Innovation Actions, Research and Innovation Actions, Coordination and Support Actions and ERA-NETs.

Horizon 2020-SC5, like its predecessors, also plays a key role in the development and aggregation of climate change models, in coordination with the International Panel on Climate Change (IPCC). As the authors of the Ex Post Evaluation of FP7-

Environment 88 explained: “Models could be developed at national level, but [the

Framework Programmes are] unique because of [their] coordination role. [They] collate and ran models, ensuring the completeness of the systems. [FPs] allow an international co-development of climate change models, creating a process of mutual learning and an efficient knowledge creation. With its funding activities in this field, the Commission contributes to the creation of international standards that avoid fragmentation of research and funding. Something similar happens in other areas, like greenhouse gases (GHG) measurement or carbon in the sea, where the EU is leader thanks to its coordination and standardisation role – not to mention the impact of research in these field on policy (e.g. Directives)”.

The IPCC, which produces the probably largest international assessment of science, addresses knowledge gaps in the field of climate that are taken into consideration by the Commission to draft the Work Programmes. The link between the Commission and the IPCC is two ways: the IPCC feed the Work Programme and the results of the projects

funded by Horizon 2020-SC5 are then exploited in the assessment reports. The 5 th IPCC

assessment report included more than 1,000 references to outputs of projects supported by FP7 and FP6. The IPCC’s report provided the scientific evidence to the historical COP21 agreement.

The scientific cooperation between the EU, US and Canada is proceeding with mutual satisfaction in the Arctic in particular under the Transatlantic Ocean (and Arctic) Research Alliance launched by the Galway declaration in May 2013. Two Arctic Working Groups have been established in 2014 with the US and Canada. The activity of these Working Groups has triggered an improved cooperation and the decision to invest in a consistent package of Arctic research activities in the Work Programme 2016-17 focused on climate change issues, which has attracted further US and Canadian investments. In fact, in a recent trilateral EU-US-Canada meeting (15 March 2016 in Fairbanks) the both US and Canadian co-chairs confirmed that US and Canada will

88 op.cit., p. 8.

identify strategies to provide financial support to national institutions participating in the Horizon 2020-funded projects on the Arctic (see section N.7.2).

The raw materials part under SC5 is coherent with the overall strategy of the Commission regarding international cooperation in the field of Raw Materials. In this regard, Australia, Brazil, Canada, Chile, Colombia, Japan, Mexico, USA and South Africa are considered countries with which the EU must have bilateral relations as they are among suppliers of raw materials to the EU, have a likeminded approach to free trade, are technologically advanced and able to cooperate on a number of issues. Besides, China and Ukraine are among largest suppliers of raw materials to the EU, including critical raw materials, where bilateral relations are politically and economically important to ensure security of RM supply. All topics on the raw materials part are open to international cooperation partners' countries. Several CSAs focused on international cooperation where participation of third countries is compulsory have been launched, in some cases targeting particular countries or regions among those cited above (e.g. Japan, US). Furthermore, a topic addressing the idea of a World Forum on Raw Materials was launched in 2016, with a view to contributing to the fair and unrestricted access to raw materials worldwide. It is also worthwhile to mention the series of US-Japan-EU trilateral workshops on Critical Raw Materials, with a new edition taking place in end 2016.

The Commission services are represented in the Multi-stakeholder Forum on STI for the Sustainable Development, created as part of the Technology Facilitation Mechanism mandated by the 2030 Agenda and the Addis Ababa Action Agenda, and

organised by the United Nations 89 . In its first meeting on 6-7 June 2016 in New York, the

representative of the Commission read a statement on behalf of the EU and its Member States.

N.6.2. Coherence with Member States’ policies

The necessary coherence between Horizon 2020 and national programmes is explicitly requested in article 13 of the Horizon 2020 regulation. It also refers to Joint Programming Initiatives, with instruments defined in article 26 (“public-public partnerships”): ERA-NETs and Article 185 TFEU.

Historically, the main channel of information between Member States and the Commission services were the Programme Committees and the National Contact Points (NCPs). To deepen the knowledge of environmental and R&I situations, issues and policies at national level, the services in charge of implementing Horizon 2020-SC5 are developing “country fiches”, which collect and analyse information and data about Member States, mainly in environmental-related and R&I fields. This exercise is coordinated with other Commission services (e.g. DG ENV) and with Member States themselves (through the NCPs). The exercise is also supported by a CSA project focused on the macroeconomic and societal impacts of the circular economy (SC5-25-2016). Creating this knowledge base in a comprehensive manner is a long-term activity that is

89 This body’s goal is to "provide a venue for facilitating interaction, matchmaking and the establishment of networks

between relevant stakeholders and multi-stakeholder partnerships in order to identify and examine technology needs and gaps, including with regard to scientific cooperation, innovation and capacity-building, and also in order to help facilitate development, transfer and dissemination of relevant technologies for the sustainable development goals". See: https://sustainabledevelopment.un.org/TFM/STIForum

still ongoing. The country fiches provide the knowledge base to ensure the alignment of Member States’ and EU’s actions.

However the main instruments supporting complementarities with national initiatives are the ERA-NETs (EUR 122 million invested in the Work Programmes 2014 to 2017) and, to a lesser extent, the forthcoming Article 185 initiative PRIMA (see Box 23 in section N.3.2). Interestingly, the breakdown of ongoing ERA-NETs by group of countries (i.e. EU-15, EU-13, Associated and Third) is different than the distribution for the whole Horizon 2020-SC5. EU-13 and Associated countries participate proportionally more in ERA-NETs. Indeed, EU-13 and Associated countries receive proportionally more funding from ERA-NETs than for the whole programme. This means that one of the risks of this instrument (“the money goes to those who have more money”) has been so far avoided.

In the raw materials part, it should be noted that the first Action Area of the EIP is "I.1 Improving R&D&I co-ordination in the EU ". In line with this, a topic focused on this aspect was launched in 2015. The new ERA-NET Cofund on Raw Materials (ERA- MIN2) will strengthen co-ordination of national and regional research programmes in the field of non-energy non-agricultural raw materials, while building on the experience of the successful ERA-NET "ERA-MIN", funded by FP7 and recently finished. ERA-MIN was a network of European organisations owning and/or managing research programs on raw materials and comprising 21 partners from 15 European countries and 2 third countries. For an original EU contribution of 1.49 million euro and one call planned, ERA-MIN succeeded to mobilise a total budget of 18.4 million euro and launch 3 joint calls where 17 projects covering the whole raw materials value chain (from exploration to recycling) were selected.

Additionally, the Commission is aware of the important role played by the Member States, regions and municipalities for many issues related to the access of raw materials, and many policy-support CSAs are being launched on this regard. For example, two CSAs focused on linking land-use planning policies to national mineral policies and good practices in waste collection systems, and two CSAs promoting the creation of EU networks of mining and metallurgy regions and regions on sustainable wood mobilisation will be launched in 2017. Finally, actions will be launched to optimise collection of data in Member States in support of the EU Knowledge Base on Raw Materials ("EC Raw

Materials Information System" (RMIS) 90 ) and taking into account the INSPIRE Directive. 91

N.6.3. Synergies with other EU policies

The Ex Post Evaluation of FP7-Environment analysed to what extent R&I projects had an actual impact on EU policies. This is an uncertain exercise, in particular because there is not any monitoring system in place to measure how projects influence policy at EU,

national or sub-national levels. The report 92 explained that “the impact of research on

policy is often indirect; it generates original knowledge and/or facilitates knowledge exchange and mobilisation. To counterbalance this, policymakers such as the EC’s DG Environment often make use of other sources of evidence, so-called “knowledge brokers”, by commissioning studies from specialist consultancies, and/or working with

90 https://ec.europa.eu/jrc/en/scientific-tool/raw-materials-information-system

91 http://inspire.ec.europa.eu/

92 op.cit., p.54.

EC’s bodies like the Joint Research Centre (JRC) and the European Environment Agency (EEA)”.

There are however good examples of direct impact of projects to policy. The Horizon 2020 project ESMERALDA is completely embedded in a policy process, Mapping and

Assessment of Ecosystem Services (MAES) 93 , with involvement of Member States. Its

deliverables feed directly this process. The lesson is clear: when projects are embedded in policy processes, they are more likely to provide results that are valuable for policy. Similarly, FP7 projects like OPERAs or OpenNESS are helping Member States involved in MAES to develop their own ecosystem mapping.

The Environmental Knowledge Community (EKC) has been created with precisely this goal. The EKC was launched in 2014 on the initiative of DG Environment. It involves DG Environment (chair), DG CLIMA, JRC, DG RTD, Eurostat and the EEA with the aim of "working in a more structured, strategic and collaborative way for the development of knowledge would help deliver better results in a more timely way, using fewer resources". A key aspect of the EKC is that each participant is at the same time knowledge user and a knowledge provider. There is not anymore the traditional distinction between “policy DGs” (i.e. DG ENV, DG CLIMA), knowledge funders (DG RTD) and knowledge providers (JRC and EEA).

The core task of the EKC is to coordinate partners' planning for key environmental knowledge topics, i.e. harmonising their work programmes, identifying synergies and avoiding duplication. This is in line with Priority Objective 5 of the 7th Environmental Action Programme (7EAP), that calls for "coordinating, sharing and promoting research efforts at Union and Member State level with regard to addressing key environmental knowledge gaps".

In addition, the EKC partners work together on Knowledge Innovation Projects (KIPs), which cover knowledge gaps of strategic importance within the 7EAP. It is considered that such gaps can be best addressed by medium-/long-term cooperation among EKC partners, in order to achieve genuine knowledge breakthroughs by 2020. The most advanced KIPs are on Natural Capital Accounts and Ecosystem Services (KIP-INCA) and on Citizens’ Science, two domains where Horizon 2020-SC5 and FP7 have an extensive background.

The ongoing experience shows that this new form of cooperation within the Commission implies that former and ongoing projects are better exploited for policy. Projects like OPERAs, OPENNESS (FP7, on natural capital and ecosystem services valuation), ESMERALDA (Horizon 2020) or those funded under MyGeoss are extensively used and it is planned to capitalise in other FP7 and newly started Horizon 2020 projects. Citizen science is quoted as a solution for simplifying environmental reporting and monitoring in the related draft Fitness Check Staff Working Document.

Professor Giovannini et al. present the EKC as the example of “science-to policy task forces” that should be implemented “for each SDG in order to diagnose the science,

93 http://biodiversity.europa.eu/maes

technology and innovation needs along the innovation chain, and consider trade-offs and

possible conflicts for each goal/target”. 94

Another example of synergy is the Horizon 2020-SC5’s contribution to the Focus Area "Industry 2020 in the Circular Economy" implements one of the actions of the Circular Economy Action Plan. The topics and projects resulting from this call aim at demonstrating on the ground the feasibility of Circular Economy, responding to political priorities of the Commission and a several DGs (ENV, GROW, REGIO). The focus area "Industry 2020 in the Circular Economy" included also the pilot on Innovation Deals – that contributes to the better regulation agenda.

The Commission services in charge of Horizon 2020-SC5 have also bilateral and/or multilateral contacts with other services that deal with environmental and climate policy, according to more traditional formulas: Inter-service groups, tasks forces, periodic meetings on specific themes, etc. The actions on raw materials in Horizon 2020-SC5 follow the logic of the EIP on Raw Materials: RIAs and IAs focus mostly on actions covered by the first "Technology Pillar" and CSAs cover non-technological aspects, as well as international co-operation (Pillars II and III). In last instance, all these actions

help to implement the Raw Materials Initiative. 95

N.6.4. Synergies with other EU funding programmes

The Horizon 2020 Regulation calls for complementarity with other Union funding programmes (article 20), notably European Structural and Investment Funds (ESI Funds) “to contribute to the closing of the research and innovation divide within the Union” (art. 21).

ESI Funds

The Work Programmes 2014-2015 and 2016-2017 underline the possibility of complementing Horizon 2020 support with private or public funding, “including fro relevant national/regional schemes under the European Structural and Investment Funds

(ESIF), in particular under the European Regional Fund (ERDF)” 96 . The procedures to

obtain this additional financing (i.e. contacting ERDF managing authorities, look at Innovation Smart Specialisation Strategies) are also explained.

Until end 2016, looking at a sample of 75 proposals, 29 mentioned synergies. Amongst the 75 proposals, 19 have been granted, of which 15 referring to complementarity with ESI Funds. Interestingly, proposals arguing their potential synergies have a higher success rate – which may suggest a higher level of preparation, knowledge of potential funding sources or more systemic approaches. There is no strong commitment to deliver on synergies as a project outcome. Indeed, the actual complementarity between Horizon 2020 and ESI Funds is not reflected in periodic reporting, neither for Horizon 2020, nor for ESIF.

It is worth noting that the CSA SCREEN (Work Programme 2016) is expected to increase synergies. SCREEN aims at “defining a replicable systemic approach towards

94 Giovannini, E. et al. (2015) The Role of Science, Technology and Innovation Policies to Foster the Implementation

of the Sustainable Development Goals. Report of the Expert Group “Follow-up toRio+20, notably the SDGs”.

Luxembourg: OPOCE.

95 COM(2008) 699 final i

96 Horizon 2020 Work Programme 2014-2015, part 12, pages 7, 25 and 39.

the transition to the circular economy in EU regions within the context of the Smart Specialisation Strategy, through the identification and implementation of operational synergies between R&I investments from Horizon 2020 and the European Structural and Investment Funds, thus contributing to novel future eco-innovative and horizontal

business models across different value chains” 97 .

The current experience shows that there is little knowledge about ESI Funds within the traditional Horizon 2020-SC5 stakeholders. The Commission services are aware of these difficulties. They are trying to have a more pro-active approach, launching awarenessraising and training actions, providing ad hoc support, etc. For instance, executive agencies and National Contact Points are developing their expertise and experience on ESI Funds’ procedures. EASME has organised a public event on this subjectgahering Commission services and more than 15 regional authorities to discuss ESIF/Horizon 2020 funding on the circular economy

In addition, complementarity with Horizon 2020 funding is desirable, but not an

eligibility criterion 98 . Obtaining more money might be interesting for local authorities,

because they could increase the scale and impact of their action (e.g. under “Smart and Sustainable Cities” Focus Area), but for academia and/or industry, the incentives are not fully clear. More resources imply transactions costs, e.g. hiring more people, which is not necessarily advantageous.

LIFE 99

The LIFE 2014-2020 Regulation 100 establishes that the LIFE Programme should

encourage the uptake of the results of environmental and climate-related research and innovation of Horizon 2020. In this regard, since the 2014 LIFE Call for proposals, projects that foresee to take up the results of environmental and climate-related research and innovation projects financed by Horizon 2020 or by preceding Framework Programmes are granted one extra point during the evaluation process.

Provisional data based on the results of the 2014 and 2015 LIFE calls (“Natura and biodiversity” Theme, NAT) show that the number of successful projects linking their activities to the results of EU-funded research projects has increased from a share of 5% in 2014 (corresponding to 2 projects out of 41 funded) to a share of almost 32% in 2015 (corresponding to 13 projects out of 41 funded).

The uptake of Horizon 2020 projects is still relatively modest (14%) and mostly referred as future uptake during implementation. This can be explained by the fact that most projects under this Horizon 2020 have started only recently and have not delivered concrete exploitable results yet.

The Horizon 2020 ERA-NET BiodivERsA is mentioned in a relatively significant share (19%) of projects that proceed to the uptake of FP results. This is possibly explained by the fact that BiodivERsA aims at generating knowledge on biodiversity and ecosystem

97 See: http://cordis.europa.eu/project/rcn/205933_es.html

98 If actual complementarity was an eligibility criterion, there may be a de facto discrimination of organisations based

in areas where ESI Funds are less available.

99 What follows is based on the analysis done by the Executive Agency for SMEs (EASME).

100 Regulation (EU) No 1293/2013 i of the European Parliament and the Council of 11 December 2013 on the

establishment of a Programme for the Environment and Climate Action (LIFE) and repealing Regulation (EC) No

614/2007

services, an area that is strongly relevant to the type of projects financed within the NAT portfolio.

The desired complementarity between Horizon 2020 and LIFE presents some specificity. Horizon 2020 is a R&I programme that, by nature, is supposed to support first-of-its-kind projects. LIFE is more focused on implementation, but it can finance demonstrations and pilot projects, actions where there could be an overlap with Horizon 2020. In fact, supporting more than one project with similar objectives, or even developing similar technologies, is not a problem. R&I teams use to compete to discover the best solutions to the same problem. The risk to avoid is double-funding of a single R&I action by the same players. Avoiding this risk requires a strong monitoring from the Commission services. This monitoring is already in place, but, to be more efficient, it would require the implementation of IT tools to make a first screening of potential cases of double funding.

In addition, LIFE has been historically a bottom-up programme; the Commission

services could hardly orientate LIFE projects towards specific policy goals 101 . This

explains why DG ENV and/or DG CLIMA used to request the support of FP7/Horizon 2020 instead of further exploiting LIFE projects.

Raw Materials’ specificities

As for other parts of Horizon 2020-SC5, proposers on the raw materials part are encouraged to seek synergies and potential additional or follow-up funding, be it private or public, including from relevant regional/national schemes under the European Structural and Investment Funds (ESIF), in particular under the European Regional Development Fund (ERDF), or other relevant funds such as the Instrument for Preaccession Assistance (IPA II). Some project proposals (such as PLATIRUS or SCRREENN) make reference to this possibility.

Furthermore, there is a clear alignment between Horizon 2020 and the EIP on Raw Materials – which is not a funding scheme. There have been already two "Call for Commitments" of the EIP on Raw Materials in 2013 and 2015. At this moment, there are 123 commitments gathering almost 1000 unique partners and covering the three-pillar structure of the EIP with an indicative budget of close to EUR2 billion. The 40 Horizon 2020 projects on raw materials selected under SC5 gather 593 partners (430 individual participants) from 43 countries. Indeed, 15 projects with 218 individual participants are linked with EIP commitments. Although participation in a commitment is not taken into account as an evaluation criterion for Horizon 2020 calls, it is worthwhile to mention that, for example, they have had a higher success rate in many topics compared to proposals not related to commitments. This is largely due to the fact that the EIP commitments have been entered into and prepared in advance of the Horizon 2020 calls. The High Level Steering Group of the European Innovation Partnership on Raw Materials held on 14 July 2016 signed the Declaration of Support for the setting up of the European Investment Platform on Raw Materials and Recycling within the EFSI

framework and in co-operation with the EIT Raw Materials. 102

101 The LIFE Impact Assessment assessed the policy impact of LIFE projects, considering that the bottom-up approach

lacked strategic focus. The current LIFE programme has therefore included some top-down instruments. European

Commission (2011) Staff Working Document accompanying the document Proposal for a Regulation on the establishment of a Programme for the Environment and Climate Action (LIFE), 12/12/2011, SWD(2016)1542 final.

102 http://ec.europa.eu/DocsRoom/documents/17826/attachments/1/translations/en/renditions/native

N.6.5. Coherence with other Horizon 2020 intervention areas

The coherence of Societal Challenge 5 with other Horizon 2020 intervention areas is mainly ensured through cross-cutting focus areas, as described in section N.2.1. Around 47% of the budget foreseen in the Work Programme 2014-2015 and 41% of the 2016- 2017 budget have been allocated to such focus areas, jointly with SC2, SC3, SC7 and LEIT.

In addition, the Earth Observation calls include a cross-reference to other parts of the Work Programme, as already explained in section N.6.1. A similar approach was followed in the Work Programme 2015-2016, where water topics were mainstreamed all over the document, with cross-references.

Stakeholders are concerned about this approach, which according to them leads to a complex reading and understanding of the Work Programme, which could be considered contrary to the simplification principle. This critique can be illustrated by the (probably excessive) size of the Work Programme 2016-2017.

The EIT’s KIC-Climate plays a role in the Working Group on the Implementation of the Climate Services Roadmap. However little has been done so far by the KIC-Climate on climate services, a strategic priority of Horizon 2020-SC5 (see section N.2.1) but at the same time a very new concept. In any case, the Climate KIC is expected to implement climate services and concrete results are expected in 2-3 years, just before the official end of Horizon 2020.

In other to get the best level of coherence, reach critical mass and optimise budget implementation, some issues relevant to Raw Materials are targeted by other parts of Horizon 2020 beyond SC5. For example, in the Work Programme 2016-2017, Societal Challenge 2 and the Joint Undertaking for Bio-Based Industries (BBI) address biotic materials (wood-based and natural rubber). Substitution of critical raw materials was tackled by SC5 in 2014-2015 but is covered under Industrial Leadership Pillar in WP 2016-2017. Raw materials part in SC5 is complementary to the cross-cutting call on 'Industry 2020 in the Circular Economy', which focuses more on resource efficiency, reuse and product life cycles. There, the PPP SPIRE focuses on resource-efficient processes, including relevant Innovation actions. During Work Programme preparation there are inter-services discussions with other Units of the Commission to ensure a good coherence (and cross-link) in the topics relevant to raw materials launched in different part of Horizon 2020 and avoid overlapping. These are now extended to the KIC-EIT on Raw Materials, in order to ensure that the actions in the Raw Materials part of the Societal Challenge 5 are complementary to the KIC-EIT It is worthwhile to mention that projects selected in raw materials topics under SC5 are requested to include a workpackage to cluster with other projects financed under the same topic and – if possible – with other relevant projects in the field funded by Horizon 2020, in support of the EIP on Raw Materials.

N.6.6. Lessons learnt/Areas for improvement

Horizon 2020-SC5 is characterised by a strong coordination with international strategies – something logical considering the global nature of environmental and climate issues. The Commission services are trying to tackle some historical weaknesses, like the insufficient knowledge of national targeted policies, the direct exploitation of R&I results in policy-making, the coherence with other EU funding programmes or even the internal coherence within Horizon 2020. Advancement is so far uneven. In particular, more efforts are necessary to ensure synergies with ESI Funds, considered essential because tackling societal challenges require a strong financial leverage.

N.7. A DDED V ALUE

This section analyses the European added-value of Horizon 2020-SC5 intervention. The traditional questions to understand the EU added-value concept are: (a) Do the problems to be addressed require a public intervention? If yes, (b) is this public intervention preferable at European scale instead of at national or sub-national level? Three criteria are typically used to assess the EU added-value: (i) Effectiveness; (ii) Efficiency and (iii) Synergy.

N.7.1. Why a public trans-national intervention is needed for climate, environment, resource efficiency and raw materials

Environmental issues (e.g. pollution of lakes, rivers or air) are often mentioned as a clear example of negative externalities or market failures , provide a rationale for government

intervention”, like regulation or public ownership. 103 The state can decide to reduce cars

emissions, but technologies must be able to follow. This implies that, to address

environmental market failures, R&I activities are a sine qua non condition 104 . Without

cleaner technologies, environmental objectives of legislation could not be implemented. A good example is the paper sector, which used to be a very polluting one, in particular for air and rivers. This sector has been able to become more environment-friendly thanks to a combination of regulation and technology – and therefore R&I. Paper is, indeed, one of the products that are more commonly recycled.

These examples justify public intervention on environmental and climate-related R&I, but why should this intervention should be implemented at EU scale?

Climate and environmental issues are transnational and, indeed, global by nature. They have a complex and large-scale character that is acknowledged by the Horizon 2020 Regulation, which considers that in these fields “activities have to be carried out at the

Union level and beyond”. 105 In the previous examples, rivers can be shared by different

countries and air pollution does not stop in national borders. Similarly, primary raw materials (e.g. underground mineral deposits) do not respect borders, and the flows of both primary and secondary raw materials are usually transnational. Climate change and its consequences have an obvious global dimension.

The Horizon 2020 Regulation follows Title XX of the Treaty on the Functioning of the European Union (TFEU). Article 191 defines the objectives of the Union’s policy on environment, including “prudent and rational utilisation of natural resources” and “promoting measures at international level to deal with regional or worldwide environmental problems, and in particular combating climate change” (§1). Such policy is guided by the principles of preventive action, precaution, rectification of

103 See: https://stats.oecd.org/glossary/detail.asp?ID=3254

104 An analysis of enterprise’s innovation data in Germany shows that conventional regulatory tools alone are not

effective for triggering eco-innovation. The Porter Hypothesis is not confirmed per se. Regulations require other factors, such as R&I, to lead to eco-innovation. See Bitat, A. (2016) “Environmental regulation and eco-innovation: insights from diffusion of innovation theory”, Open Innovation 2016 Conference Proceedings, Vienna, 24-25 November. Forthcoming.

105 Regulation 1291/2013 i establishing Horizon 2020, Annex I, Part III, section 5.2.

environmental damage at the source and polluter pays (§2). It must take into account, amongst others “available scientific and technical data” (§3).

These objectives are currently implemented through the 7th Environmental Action

Programme (7EAP) 106 , in accordance with Article 192(3) of the TFEU. Article 1(38)

states that the EU has the right to adopt measures to achieve the objectives of the 7EAP, within the limits of subsidiarity and cost-effectiveness, when “by reasons of the scale and effects” they “cannot be sufficiently achieved by the Member States”. More specifically, the objectives of the 7EAP include increasing the knowledge-based for policy, securing investments, improving policy coherence and increasing the Union’s effectiveness to address international environmental and climate challenges.

Both the Treaty and the 7EAP raise the strong connection between the EU environmental policy and R&I. This is further underlined by article 7 of the 7EAP Annex, which mentions the important role that the EU’s Framework Programme on R&I plays: “Horizon 2020 will provide the opportunity to focus research efforts and to deploy Europe’s innovation potential by bringing together resources and knowledge across different fields and disciplines within the Union and internationally”.

More specifically, Horizon 2020-SC5 supports activities that could be hardly conducted

at national level. As explained in the Ex Post Evaluation of FP7-Environment 107 , this is

the case of Earth Observation (GEO/GEOSS) or the IPCC. Earth observation requires costly infrastructures. The fragmented action of single countries alone could not get results (effectiveness criterion). The contribution of Horizon 2020-SC5 to the IPCC is

another example. The 5 th IPCC report includes around 1,000 quotes to FP6 and FP7

projects’ outputs, which shows how the Framework Programmes are important for IPCC assessments. Moreover, very high impact is generated by large transnational projects that need big critical mass, such as for the development of very complex Earth System Models and the carrying out of multi-model inter-comparison exercises that require extensive supercomputing runs. (synergy criterion).

The IPCC has a huge social and political impact. It presents the global scientific consensus about climate change, its causes and consequences, illustrated by the success of the COP21 agreement.

Similarly, the involvement of Horizon 2020-SC5 in the Belmont Forum (see Box in section N.6.1) aims at increasing the consistency between national and international R&I funders, focused on interdisciplinary and transdisciplinary research to inform human action and adaptation to global environmental change.

EU intervention on aspects related to raw materials such as the definition of mineral deposits of public importance or the strengthening of raw materials knowledge base at EU level (and also at international level) is justified and has a clear added value.

Last but not least, evidence demonstrates that collaborative R&I produces great benefits in terms of scientific outcomes and innovation results (efficiency criterion). Collaborative R&I creates synergies and mutual learning between beneficiaries from different countries, increasing the overall knowledge and expertise. This leads to better results

106 Decision No 1386/2013/EU on a General Union Environment Action Programme to 2020 ‘Living well, within the

limits of our planet’. At: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D1386&from=EN

107 op.cit., pp. 8-9.

overall, both in terms of scientific outcomes (the majority of highest cited papers use to

be produced by collaborative teams) 108 and in terms of innovation (see section N.4.2).

The efficiency criterion is further supported by the data presented in Table 204, which compares the citation impact of publications from projects supported by FP7 and Horizon 2020 since 2007 with the EU, USA and world averages.

Table 204 - Citation impacts, comparative information – Horizon 2020 SC5 fields

All 2007 2008 2009 2010 2011 2012 2013 2014 2015

EC funded 2,45 1,78 2,04 2,46 2,5 2,52 2,47 2,34 2,67 3,63 publications

EU-13 0,9 0,86 0,81 0,83 0,85 0,87 0,89 0,94 1,03 0,97

EU-15 1,31 1,29 1,29 1,29 1,32 1,33 1,31 1,32 1,32 1,33

EU28 1,25 1,24 1,23 1,23 1,25 1,27 1,25 1,26 1,27 1,26

Source: Scopus, September 2015.

N.7.2. Horizon 2020-SC5 projects demonstrating EU Added Value

It is too early to determine which ongoing Horizon 2020-SC5 projects will actually demonstrate European added-value. There are however some promising cases. The list below is not comprehensive.

Examples contributing to the circular economy:

The main added-value of these projects comes from their scale and expected impacts, hardly to reach without Horizon 2020 support and without international cooperation.

 MASLOWATEN (MArket uptake of an innovative irrigation Solution based on

LOW WATer-ENergy consumption): Innovation Action; topic: WATER-1b 2015; EC contribution: EUR4 million; total budget: EUR4.9 million; 13 partners. This Innovation Action will demonstrate large scale photovoltaic pumping systems for irrigation using smart digital solutions in five farmlands in Spain, Italy, Portugal and Morocco – Mediterranean countries facing similar (increasing) issues related to water scarcity. The aim is to deliver efficient systems that are not dependant on daily variations of solar irradiation. Environmental benefits are significant as the systems are 100% powered by renewable energy and use 30% less water resources. At the end of the project it is expected to generate a real market of 6 GigaWatt of large-scale systems, i.e. a EUR 9 billion business.

Example contributing to Earth Observation:

 AtlantOS (Optimising and Enhancing the Integrated Atlantic Ocean Observing

Systems): Research and Innovation Action; topic: Horizon 2020-BG-2014-2; EC contribution: EUR21 million; total budget: EUR21 million, 62 partners. AtlantOS is a large scale research and innovation project that aims to improve and innovate Atlantic observing by using the Framework of Ocean Observing to obtain an

international, more sustainable, more efficient, more integrated, and fit-forpurpose system contributing to the Trans-Atlantic Research Alliance, the GEO

108 Halevi, H. and Moed, H. (2014) "10 years of research impact: top cited papers in Scopus 2001-2011", in Research Trends, Issue 38, September 2014. At: http://www.researchtrends.com/wp-content/uploads/2014/09/4135-Research Trends-Issue-38-v3-singles-online.pdf

global initiative Blue Planet, and GOOS (Global Ocean Observing Systems). It puts together the main oceanic actors, in Europe and beyond, in order to integrate

ocean observing activities across all disciplines for the Atlantic.

Examples contributing to the Arctic Science Ministerial meeting EU’s engagements:

 INTAROS (Integrated Arctic Observing System; Research and Innovation

action; topic: BG-2016-1): The project, with a EUR15.5 million budget, will involve scientists in 14 European countries (Belgium, Denmark, Finland, France, Germany, Greenland, Ireland, Italy, Norway, Poland, Portugal, Spain, Sweden, and the United Kingdom), as well as in a number of countries elsewhere in the world (Canada, the Peoples’ Republic of China, the Russian Federation, and the

United States, with other countries expected to join).

 APPLICATE (Advanced Prediction in Polar regions and beyond: modelling,

observing system design and LInkages associated with a Changing Arctic climate) (2016-2020; Research and Innovation action; topic: BG-10-2016; EUR 8 million budget) and Blue-Action (2016-2021; Research and Innovation action; topic: BG-10-2016; EUR 7.5 million budget) aim at understanding the impact of the changing Arctic on the weather and climate of the Northern Hemisphere. They will involve scientists in 13 European countries (Belgium, Denmark, the Faroe Islands, Finland, France, Germany, Iceland, Italy, Norway, Poland, Portugal, Spain, and the United Kingdom), as well as in a number of countries elsewhere in the world (Canada, the Peoples’ Republic of China, the Republic of

Korea, the Russian Federation, and the United States). 109

Example contributing to the Raw Materials policies and knowledge base:

 ProSUM (Prospecting Secondary raw materials in the Urban mine and Mining

waste) is a Coordination and Support Action (CSA) establishing a European network of expertise on secondary sources of critical raw materials (CRMs), vital to today’s high-tech society. This CSA is already contributing to policy-making activities in the field of circular economy and resource-efficiency. ProSUM directly supports the European Innovation Partnership (EIP) on Raw Materials and its Strategic Implementation Plan (SIP). The project is complementary to a few other Horizon 2020 and FP7 actions focused on primary raw materials and it is concurring on unlocking new possibilities for a sustainable supply of raw materials. It is contributing to the Raw Material Information System (RMIS) developed by JRC under DG GROW initiative. ProSUM deliverables are key for

the creation of a European raw materials knowledge base.

N.7.3. Other aspects related to EU Added Value

The collaborative research supported by the Framework Programmes has been historically able to identify new environmental and climate-related problems, and even to open new areas of research. This is the case of ocean acidification, an issue that is

considered one of nice “planetary boundaries” by Rockström, Steffen et al. 110 The

planetary boundaries concept was endorsed by the United Nations in 2012, in the context of the so-called “zero-draft” of the outcome of the United Nations Conference on

109 https://www.whitehouse.gov/the-press-office/2016/09/28/fact-sheet-united-states-hosts-first-ever-arctic-science

href="https://www.whitehouse.gov/the-press-office/2016/09/28/fact-sheet-united-states-hosts-first-ever-arctic-science-ministerial">ministerial

110 See: Rockström, J.; Steffen, K. et al. (2009) ”Planetary Boundaries: Exploring the safe operating space for

humanity”, in Ecology and Society, 4, 32; and Steffen, W.; Richardson, K.; Rockström, J.; et al. (2015). "Planetary boundaries: Guiding human development on a changing planet"., in Science. 347 (6223).

Sustainable Development to be convened in Rio de Janeiro 20–22 June 2012 – but it was removed in successive versions. At EU level, the sub-title of the 7EAP is indeed “living well within the limits of the planet”. These two examples show the policy relevance of the concept and its nine boundaries, including ocean acidification. The knowledge on ocean acidification was very rudimentary when FP7 launched the project EPOCA (“European Project on OCean Acidification”), an Integrated Project bringing together 160 researchers from 32 institutes from 10 European countries, in 2008. EPOCA, together with the projects MedSea (2011-2014) and CARBO-CHANGE (2011-2015), were critical for the advancement of our understanding of the biological, ecological, biogeochemical, and societal implications of ocean acidification. They produced around 400 peer-reviewed publications, in an area that nowadays counts a scholarship output of more than 2,500 papers since 2011, with 1,085 international collaborations, and 33,000 citations. Ocean acidification continues to be increasingly relevant for research. Some of the most quoted authors in the field, like Jean-Pierre Gattuso or Ulf Riebesell were

involved in EPOCA. 111

N.7.4. Lessons learnt/Areas for improvement

The added-value of R&I to address climate action, environmental, resource efficiency and raw materials issues goes well beyond European borders. The design and implementation of Horizon 2020-SC5 is connected with international initiatives, like the IPCC, GEO or the Belmont Forum, to ensure coordinated responses to societal challenges at global level. The examples of projects financed by Horizon 2020-SC5 are characterised by very large expected impacts, both economic and in terms of resource efficiency (innovation actions), or by bringing together the main European and/or international actors in specific areas, towards a common and trans-national goal – either policy-related or scientific. In all cases, actions could not be carried-out within a single country, because or the scale of the intervention or because of the need of bringing together the main players in a specific field.

Two areas for improvement:

 Large scale demonstrations present ambitious economic and environmental

impacts; but the question remains if they will be able to reach those goals. The Commission services have not yet developed any monitoring system to measure economic impacts (e.g. turnover) and environmental/resource efficiency impacts

(e.g. energy or raw materials savings, reduction of emissions) of its projects.

 Communication: Do citizens know that knowledge or innovation advancements,

which affect their day-to-day life, would had been impossible without European Union’s support? The clearest example is probably climate change, where the IPCC has produced an awareness and behavioural change, but the critical role of

the Framework Programme remains hidden to the general public..

N.8. S UCCESS STORIES FROM FP7

WeSenseIt (2012-2016 112 ) developed a citizen observatory of water that allows citizens

and communities to take on a new role in the information chain: a shift from traditional one-way communication towards a two-way model in which citizens become active

111 Elsevier’s Scival search, on 17/10/2016.

112 www.wesenseit.eu . Based on European Commission (2016) Open innovation, open science, open to the world. A

vision for Europe., op.cit., p.55.

stakeholders in capturing, evaluating and communicating information. For this, WeSenseIt leverages environmental data and knowledge (from both professionals and communities) to manage water resources effectively and efficiently. Citizens (such as civil protection volunteers) help by taking measurements using new apps and sending information and images by phone. They can also help by reading existing sensors and sending authorities the data via mobile apps. New technologies and approaches to water management have tested and validated in three EU countries: the UK, the Netherlands and Italy.

We SenseIt shows also that citizens’ science can create business opportunities, with commercialisation of a tool just ten months after the beginning of the project. Events involving over 600,000 people were monitored with excellent results, demonstrating the power of citizens’ science. In the Veneto region, techniques developed within this project have been adopted by the City Council and Civil Protection in Vicenza and there are plans to extend it to the whole North East of Italy. This platform has also been adopted as a mitigation measure for flood risk in the implementation of the Floods Directive in the Italian Eastern Alps district, currently operational.

SORT-IT (2008-2012) is an example of resource-efficient technology in the paper sector, which produces 100 million tonnes per year on 1,400 paper machines and employs 225,000 people, for a turnover of EUR80 billion. SORT-IT developed a technology that provides more recovered paper, with better quality, thanks to automatic identification and sorting units.

The technologies were implemented into a new green field, full-scale industrial automatic sorting plant in Linz, Austria, with a capacity of 2,000 tonnes per month. The pilot plant demonstrated the soundness of the concepts developed. It showed the viability of the paradigm change in industry, which can now replace manual sorting by fully automatic sorting plants. It also demonstrates that medium sized, tailored sorting plants for the need of municipalities are now economically viable.

Life-cycle assessments demonstrated several environmental, economic and social benefits:

 The environmental and economic sustainability performance of the newsprint

paper (grade 1.11) and the packaging paper (grade 1.04) life cycles were

optimised thanks to the improved quality of recovered paper.

 Savings were achieved by lower consumption of the improved quality recovered

paper, raw materials and energy.

 More efficient logistics were ensured, thanks to transportation of sorted paper

with lower amount of impurities.

As of August 2014, less than two years after the end of the project, SORT-IT partners already obtained sales of EUR1 million, with estimated raw materials savings of EUR200,000 and energy saving of EUR100,000.

C2CA (“Advanced Technologies for the Production of Cement and Clean Aggregates from Construction and Demolition Waste”, 2011-2015) focused on recycling end-of-life of concrete. The production of the cement used in concrete is responsible for at least 5% of worldwide CO2 emissions.

C2CA developed an innovative mobile industrial-scale ADR (Advanced Dry Recovery) machine for wind sifting of EoL (End-of-Life) concrete. The technology is installed in the Strukton plant in Hoorn, the Netherlands, with a capacity of 80-100 tonnes of crushed concrete per hour. Thanks to this technology, clean recycled aggregates can be converted into a new concrete of similar strength and of only slightly lower durability than that made of virgin materials. Fine fractions (0-4 mm, 40% of crushed concrete) can replace more expensive finely-ground materials in brick production. One-fourth of them (calcium–rich) can also substitute limestone in the cement production.

N.9. L ESSONS LEARNT /C ONCLUSIONS

N.9.1. Relevance

 Key findings: In 2016, climate action, environment, resource efficiency and raw

materials issues are even higher in political agendas than in 2011, when the Horizon 2020 Impact Assessment pointed-them amongst the “grand societal challenges”. The Sustainable Development Goals and the COP21 Paris Agreement have established, for

the first time ever, global and compulsory objectives to better protect the planet.

 The strengths are: There is also a growing social awareness and citizens’ concerns about

environmental issues, and therefore a growing market demand for eco-innovative products and services. Moving towards a sustainable, circular and law carbon economy is an opportunity to create growth and jobs in sectors where Europe has competitive advantages. Horizon 2020-SC5 aims at contributing to the first priority of President

Juncker’s agenda.

 The bottlenecks/weaknesses are: The involvement of relevant stakeholders in the

Horizon 2020-SC5 fields, such as NGOs and Civil Society Organisations (CSOs) still appears to be low. The most active stakeholders remain the traditional R&I actors, like academia and industry. This is happening despite the efforts to open Horizon 2020 to new players and to empower citizens, in particular through citizens science/citizens observatories. However, it is worth noting that citizens, local communities and CSOs are

often involved in projects, even if not as beneficiaries.

N.9.2. Effectiveness

 Key findings: It is still too early to assess the effectiveness of Horizon 2020-SC5 based

on results. All projects are still in their initial phases.

 The strengths are: The current approach, largely based on large-scale demonstrations,

should lead to a stronger societal and environmental impact than previous FPs. Also the innovation impact should be higher, thanks to the deployment of new technologies combined with business models and the increased participation of enterprises that should lead to a better uptake of the research results by industry. A stronger policy impact is also expected, thanks to further coordination with other policy Commission services (for instance, through the Environmental Knowledge Community, EKC). Actions on raw materials positively contribute to the objectives of the Raw Materials Policy and the European Innovation Partnership (EIP) on Raw Materials Horizon 2020, and help to

consolidate a growing raw materials R&I community in Europe.

 The bottlenecks/weaknesses are: There will likely be a decrease in “Key Performance

Indicators” like publications and patents – probably mitigated by intervention areas as climate change or earth observation. Some anecdotal evidence about barriers to implement new approaches: resistance to change of traditional (academic) stakeholders, difficulty to attract new kinds of innovators (despite the fact that participation is

widening), or the insufficient maturity of the “markets of the future” that Horizon 2020- SC5 tries to promote.

N.9.3. Efficiency

 Key findings: It is too early to assess the efficiency of Horizon 2020-SC5 on the basis of

its results. Simplification is one of the keywords of Horizon 2020, implemented at the level of procedures and through new management modes. The EC contribution to

projects is higher than under FP7-Environment (net present value).

 The strengths are: EASME is reaching its targets, like time-to-grant, even if those are

ambitious and the procedures demanding.

 The bottlenecks/weaknesses are: There is room for improvement in terms of (formal)

communication flows between the Commission, in charge of policy, and EASME, in charge of managing projects. The monitoring system can be improved. The complexity

of Horizon 2020 is an obvious challenge. The very low success rates (“oversubscription”) is a proof of success, but it also means that a lot of R&I resources are spend to prepare and evaluate unsuccessful proposals.

N.9.4. Coherence

 Key findings: Horizon 2020-SC5 is characterised by a strong coordination with

international strategies – something logical considering the global nature of environmental and climate issues: IPCC, GEO, Belmont Forum, Transatlantic Ocean Research Alliance launched (Galway declaration) or the forthcoming World Forum on

Raw Materials.

 The strengths are: Horizon 2020-SC5 is addressing some historical weaknesses, like the

insufficient knowledge of national issues and policies, the direct exploitation of R&I results in policy-making, the coherence with other EU funding programmes or even the

internal coherence within Horizon 2020.

 The bottlenecks/weaknesses are: Still insufficient synergies with ESI Funds, considered

essential because tackling societal challenges require a strong financial leverage.

N.9.5. EU Added Value

 Key findings: Environmental issues are always mentioned as a clear example of negative

externalities, which, as market failures, provide a rationale for government intervention. To address environmental market failures, R&I activities are a sine qua non condition.

Climate change and its consequences have an obvious global dimension.

 The strengths are: In areas like earth observation, which requires costly infrastructures,

the fragmented action of single countries alone could not get results (effectiveness criterion). The contribution of Horizon 2020-SC5 to the IPCC is another example. The

5 th IPCC report includes around 1,000 quotes to FP6 and FP7 projects’ outputs, which

shows how the Framework Programmes are important for IPCC assessments. Moreover, very high impact is generated by large transnational projects that need big critical mass, such as for the development of very complex Earth System Models and the carrying out of multi-model inter-comparison exercises that require extensive supercomputing runs (synergy criterion). Horizon 2020-SC5 is characterised by a strong coordination of R&I activities at international level (e.g. Belmont Forum, GEO, IPCC, Arctic). Last but not least, evidence demonstrates that collaborative R&I produces greater benefits in terms of

scientific outcomes and innovation results (efficiency criterion).

 The bottlenecks/weaknesses are the difficulty to measure the climate and environmental

impacts of R&I actions, and to communicate them to citizens.

• O. 

  E UROPE IN A CHANGING WORLD - I NCLUSIVE INNOVATIVE AND REFLECTING SOCIETIES

  O.1. I NTRODUCTION

  O.1.1. Context

This report presents the interim evaluation of the specific objective ‘Europe in a changing world – Inclusive, innovative and reflective societies’ (Societal Challenge 6, SC6) of Horizon 2020. This assessment is primarily based on desk study carried out using Corda monitoring data and the analysis carried out by the FP7 project IMPACT EV using e-corda and a survey from a sample of 56 projects under Horizon 2020 that had started in the first wave of 2014- 2015 projects. The report of the study ‘Assessment of the Union Added Value and the

Economic Impact of the EU Framework Programmes (FP7, Horizon 2020) 113 and the Interim

Evaluation of LERU (League of European research Universities, Advice Paper No.21, October 2016 ) were also used, as well as the policy reviews performed by DG RTD-B6 unit. Finally, lessons have also been drawn from the ex-post evaluation of FP7 and the accompanying recommendations.

The research funded under this thematic programme aim to respond to the needs and

challenges identified in the Horizon 2020 Impact Assessment working paper. 114 In this

document, crucial challenges for Europe such as low growth, insufficient innovation, and a diverse set of environmental and social challenges were identified. Science and innovation that bring the development of new products, processes and services are seen as key instruments for promoting growth and tackling pressing social challenges. The Horizon 2020 impact assessment also highlights the importance of public interventions that go beyond the limited possibilities of Member States and have a European added value. In addition, it stresses the need to develop a more result-driven approach. This includes a new system for the evaluation and monitoring of Horizon 2020 that is focused on throughputs, outputs, results and impacts.

The Social Sciences and Humanities (SSH) were funded for the first time as a dedicated programme under the Fourth Framework Programme (FP4), a trend that continued throughout the Seventh Framework Programme (FP7). In contrast, Horizon 2020 represents a new approach whereby SSH research is both part of an SSH-intensive thematic priority (SC6) and a cross-cutting issue integrated across the entire work programme. SC6 has therefore a strong, though not exclusive, SSH component. It represents 4.4% of the budget for Societal Challenges and accounts for 1.84% of the total Horizon 2020 budget. In addition, the SSH are integrated across Horizon 2020 in order to maximise returns on investment from science and technology, foster inclusive and innovative societies, and help find solutions to contemporary

societal problems. 115 The budget going to SSH under this integrated approach largely depends

on the design of calls for proposal, the content of selected projects and the composition of funded consortia.

113 PPMI, Assessment of the Union Added Value and the Economic Impact of the EU Framework Programmes (FP7, Horizon

2020) (2012/S 144-240132)

114 http://ec.europa.eu/research/horizon2020/pdf/proposals/horizon_2020_impact_assessment_report.pdf#view=fit&pagemod

e=none

115 https://ec.europa.eu/programmes/horizon2020/sites/horizon2020/files/Horizon 2020_inBrief_EN_FinalBAT.pdf

While the globalisation of research and innovation is not a new phenomenon, it has become increasingly visible, particularly in terms of collaborative research, international technology production, and the worldwide mobility of researchers.

Europe is confronted with major socio-economic challenges which significantly affect its common future. These include growing economic and cultural interdependencies, ageing and demographic change, social exclusion, inequalities and poverty, the democratic deficit, integration and diversity, migration flows, a growing digital divide, fostering a culture of innovation and creativity in society and enterprises, instability in the EU neighbourhood, violence and radicalisation, and a decreasing sense of trust in institutions and between citizens within and across borders. These challenges are enormous and they call for a common European approach, based upon shared scientific knowledge that social sciences and humanities can provide.

The Impact Assessment accompanying the Communication from the Commission 'Horizon 2020 - The Framework Programme for Research and Innovation' [SEC(2011) 1427)] recognises that the solutions to all of these problems are linked. It is precisely by addressing social challenges that Europe will be able to boost productivity, generate long-term growth and secure its place in the new world order. In this context Societal Challenge 6 is playing a pivotal role by fostering inclusive, innovative and reflective European societies in a context of unprecedented transformations and growing global interdependencies. The capacity of the research programmes to respond rapidly to evolving problems also highlights how research communities can collaborate with all stakeholders in building a more resilient European society.

O.1.2. Objectives and intervention logic

The specific objective of societal challenge 6 (SC6) 'Europe in a changing world - Inclusive, innovative and reflective societies is to contribute to the Horizon 2020 general objectives or priorities by fostering a greater understanding of Europe, by providing solutions to contemporary economic, social and political problems and by supporting inclusive, innovative and reflective European societies in a context of unprecedented transformations

and growing global interdependencies. 116 To address the identified objectives, the thematic

programme has adopted the intervention logic outlined in Figure 259.

The activities of SC6 are focused on developing: 117

• 1) 

  Inclusive societies, including (a) the mechanisms to promote smart, sustainable and inclusive growth; (b) trusted organisations, practices, services and policies that are necessary to build resilient, inclusive, participatory, open and creative societies in Europe, in particular taking into account migration, integration and demographic change; (c) Europe's role as a global actor, notably regarding human rights and global justice; (d) the promotion of sustainable and inclusive environments through innovative spatial and urban planning and design;

116 Regulation (EU) No 1291/2013 i of the European Parliament and of the Council of of 11 December 2013 Establishing

Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No

1982/2006/EC. p. 58 http://ec.europa.eu/research/participants/data/ref/Horizon 2020/legal_basis/fp/Horizon 2020-euestablact_en.pdf

117 http://ec.europa.eu/research/participants/data/ref/Horizon 2020/legal_basis/fp/Horizon 2020-eu-establact_en.pdf

• 2) 

  Innovative societies, (a) strengthen the evidence base and support for the flagship initiative "Innovation Union" (IU) and the European research Area (ERA); (b) explore new forms of innovation, with special emphasis on social innovation and creativity, and understand how all forms of innovation are developed, succeed or fail; (c) make use of the innovative, creative and productive potential of all generations; (d) promote coherent and effective cooperation with third countries;

• 3) 

  Reflective societies: (a) study European heritage, memory, identity, integration and cultural interaction and translation, including its representations in cultural and scientific collections, archives and museums, to better inform and understand the present by richer interpretations of the past; (b) research into European countries' and regions' history, literature, art, philosophy and religions and how these have informed contemporary European diversity; (c) research on Europe's role in the world, on the mutual influence and ties between the regions of the world, and a view from outside on European cultures.

Table 205 - Societal challenge 6 activities in the legal basis

Societal challenge 6 activities in the legal basis 6.1 Inclusive Societies 6.1.1 Smart, sustainable and inclusive growth 6.1.2 Trusted organisations, practices and resilient societies 6.1.3 Europe's role as global actor 6.1.4 Innovative spatial and urban planning and design 6.2 Innovative Societies 6.2.1 Evidence base and support for the IU and ERA 6.2.2 New forms of innovation, social innovation 6.2.3 Making use of the innovative, creative and productive potential for all generations 6.2.4 Cooperation with third countries 6.3 Reflective Societies 6.3.1 European heritage, memory, identity and culture 6.3.2 History, literature, art, philosophy, and religions 6.3.3 Researching Europe's role in the world, the mutual influence and ties between the world regions 6.4 Specific implementation aspects

Source: Horizon 2020 Specific Programme.

The comparison with the objectives of the Specific Programme “Socio-Economic Sciences and Humanities” in the Seventh Framework Programme for Research and Technological

Development (FP7) 118 shows a degree of continuity in the operational objectives related to

economic growth, innovation, demographic and environmental challenges, social inclusion and cohesion, social policy, and the role of Europe in the world as a global actor. The Horizon 2020 programme has gone further than FP 7 in inviting the research community to further develop deeper interdisciplinary methods of analysis oriented towards the resolution of contemporary societal problems, and in particular to devise methods and indicators of impact assessment of projects, that ensure a better analysis of the effectiveness and value added of European funding efforts.

The areas related to research on topics like migration, the economic and social dimensions of the financial crisis of 2007-08, radicalisation, cultural heritage and the evolution of European

118 http://ec.europa.eu/research/participants/data/ref/fp7/90448/fp7ec_en.pdf

identities have emerged with more strength in Horizon 2020. One reason for this evolution has been the recent migratory crisis related to the international context, and the new phenomenon of the violent radicalisation of youth, including young women, which brought the Member States and the European Union to mobilise the research communities on these topics. The policy reviews that are being produced on these areas of research in SC 6 provide tools and analysis to policy makers.

On 15 July 2014, the then candidate for President of the European Commission, Jean-Claude Juncker, presented to the European Parliament a set of political priorities: A New Start for

Europe: My Agenda for Jobs, Growth, Fairness and Democratic Change. 119 These priorities

significantly affect the implementation of Horizon 2020. Starting from the Work Programme 2016-2017, Horizon 2020 actions are targeted at contributing to these objectives. The same applies to policy initiatives launched by any Commission service, including those relating to R&I. More specifically, SC6 aims to contribute to priority 1 (A new Boost for Growth, Jobs and Investment, reducing inequalities), 2 (A connected Digital Single Market), 6 (A Deeper and Fairer Economic and Monetary Union), 7 (An Area of Justice, Fundamental Rights Based on Mutual Trust), 8 (Towards a New Policy on Migration), 9 (A Stronger Global Actor contributing to global justice and stability) and 10 (A Union of Democratic Change - with focus on strengthening citizenship).

Societal Challenge 6 (SC6) is implemented mainly through calls for proposals outlined in multiannual Work Programmes (2014-2015, 2016-2017, and 2018-2020 in preparation). Theses Work Programmes are prepared by the Commission services taking account of independent advice (External Advisory Group), the outcome of stakeholder consultations, and the deliberations in the context of the formal opinion of the Programme Committee (representative of EU and Associated Countries). The Work Programmes establish and communicate the parameters of calls for proposals (objectives, budgets, deadlines, etc.) and other actions. They are adopted through Commission Decision and form the legal basis for implementation of calls for proposals and other actions (i.e. expert contracts, public procurement, grant to identify beneficiary, and prizes).

The development of the Work Programmes spans over a period of about 18 months. The first half is dedicated to identify the main objectives of the Work Programmes, on the basis of the specific programme, while the second sees the development of the content of the Work Programmes on the basis of the identified priorities which are set out in the scoping paper.

In the period 2014-2020 the SC6 is supported by three Work Programmes:

 Work Programme 2014-2015 was implemented through 5 calls of proposals (44

topics) and 36 other actions. The estimated budget for the Work Programme in 2014- 2015 is EUR 310 Million.

 Work Programme 2016-2017 is implemented through 4 calls of proposals (44 topics)

and 31 other actions. The estimated budget for the Work Programme in 2016-2017 is EUR 330 Million.

 Work Programme 2018-2020 is expected to be implemented through 3 calls of

proposals for an estimated budget of EUR 584 Million.

119 https://ec.europa.eu/priorities/sites/beta-political/files/juncker-political-guidelines_en.pdf

Figure 247 - Intervention logic of SC6 ‘Europe in a changing world – Inclusive, innovative and reflecting societies’ in Horizon 2020

Analysis of the following SPECIFIC INPUTS EXPECTED EXPECTED EXPECTED IMPACTS problems OBJECTIVES OUTPUTS RESULTS

Economic and cultural interdependencies

Ageing demographic change, Source: Budget allocated to SC6: €1,309.5 M New knowledge OPEN SCIENCE

Integration and disintegration, Programme Management /understanding of

Inequalities and migration flows, Greater social phenomena

growing digital divide, understanding of and citizen attitudes Strengthened SSH Better and evidence-based

Need to foster a culture of

the societal changes ACTIVITIES capacities/ excellence policies on economy,

innovation and creativity in

society and enterprises, analysis and New or better financing, education, Decreasing sense of trust in develop social, datasets & statistics Enhanced Europe enlargement,

democratic institutions and economic and

Horizon 2020 Work • Research and engagement in innovation, cultural heritage between citizens within and across borders political inclusion Programmes 2014 - innovation actions and others Future-oriented Science 2.0

and positive inter 2015 and 2016-2017

Objective of SC6 cultural dynamics • Coordination and methods for data Better governance

To foster a greater understanding support actions

measurements architecture of the EU

of Europe, provide solutions and 2014-2015 OPEN support inclusive, innovative and • Innovation actions Comparative studies INNOVATION

reflective European societies in a Overcoming the Crisis Innovation in policy Scaling-up of best-practices

context of unprecedented Development of

transformations and growing (EURO 1-6) innovative • ERA-NET Cofund New tools and making

methodologies for Changing/influencing global interdependencies societies and

Young Generation innovation attitudes amongst citizens policies in Europe (YOUNG 1-5) • SME Instrument Increase of innovation for a better Socio-economic

Contribution to Horizon and competitiveness cohesion of Europe • Public procurement Identification of

2020 Objectives Reflective Societies in Europe Explore new forms of innovation (REFLECTIVE 1-11) through a framework

best practices &

contract practical solutions Reduce deprived situation and strengthen the evidence base Engagement in Open of vulnerable groups

research to tackle societal Understanding of

challenges will address social Europe's Europe as a Global exclusion, discriminations and intellectual basis, Actor (INT 1-12)

• Public procurement Innovation Projections and

future scenarios Strengthening EU’s present various forms of inequalities, and future role in the global

which is a prerequisite for a its history and the sustainable European integration. many European New Forms of

• Grant to identified order

and non-European Innovation (INSO 1-10)

beneficiary Policy OPEN TO THE recommendations WORLD

Contribution to EU2020 influences, as an Development of business 2016-2017 • Coordination and International

Strategy and EC inspiration for our support actions Civic participation / positioning model innovation in SMEs

priorities lives today Co-creation for growth

and inclusion (CO- • Individual expert

public engagement Improved services quality

CREATION 1-9) contracts and efficiency

EU2020targets:

Reversing inequalities • Expert contracts

Employment (REV-INEQUAL 1-10) • Prize and Public Other EU Policies Procurement EFSI, ESI funds, Erasmus+, Creative Europe, EU external and R&D Engaging together

globally (ENG- developmental policy, Other EU thematic policies.

Climate change GLOBALLY 1-9) • Framework Partnership

Understanding Europe Agreement External Factors Education

(CULT-COOP 1-11) Socio-economic conditions affecting the availability of funding at EU, national, regional, local level (public and private); MS

Poverty 1374

Junker priorities policy/legislation/regulatory framework; International agreements

O.2. I MPLEMENTATION STATE OF PLAY

O.2.1. Overview of programme inputs and activities

Societal Challenge 6 is implemented mainly by DG RTD (8 Units involved) and by DG CNECT (4 Units involved) with a total budget of EUR 1.2 billion. DG CNECT is in charge of more than 25% of the overall budget (EUR 318 million). The reaming 75% of the budget (DG RTD) is shared between various actions linked to social sciences and humanities, international cooperation, innovation and other actions (EUR 883 million).

Through the Horizon 2020 Work Programmes 2014-2017, each line of activity of SC6 was allocated a share of the overall budget of 640 EUR million. This budget allocation among the activities foreseen on the specific programme is evenly distributed among the three SC6 priorities (see Table 206).

Table 206 - Activities and allocated share of budget for the programming period 2014-2017

Allocated share Percentage

Activities in the SC6 Specific Programme of thematic share on budget (EUR total

million)

6.1.1 Smart, sustainable and inclusive growth 45 7 .03 inclusive growth

6.1.2 Trusted organisations, practices and resilient societies 65 10.16 6.1.3. Europe's role as global actor 40 6.25

6.1.4 Innovative spatial and urban planning and design 15 2.34 6.1 Inclusive Societies 165 25.78

6.2.1 Evidence base and support for the IU and ERA 40 6.25 6.2.2 New forms of innovation, social innovation 80 12.50

6.2.3 Making use of the innovative, creative and productive 15 2.34 potential for all generations

6.2.4 Cooperation with third countries 45 7.03 6.2 Innovative Societies 180 28.13

6.3.1 European heritage, memory, identity and culture 70 10.93 6.3.2 History, literature, art, philosophy, and religions 30 4.69

6.3.3 Researching Europe's role in the world, the mutual 10 1.56 influence and ties between the world regions

6.3 Reflective Societies 110 17.18 6.4 a) Specific implementation aspects 68 10.63 6.4 b) COST 80 12.50 SME instrument 37 5.78

6.4 and SME 185 28.91 Total 2014-2017 640 100 Source: Unit B 6, Open and Inclusive Societies.

Table 207 provides an overview of budget allocation through calls and other actions (i.e. expert contracts, public procurements, grants to identify beneficiary, and prizes) under the SC6 Work Programmes 2014-2015 and 2016-2017.

Table 207 - Calls, other actions and budget in WP 2014-2015 and 2016-2017

Work Budget

Programme Call (million €)

Call EURO-2014/2015: Overcoming the Crisis: New Ideas, 53 Strategies and Governance Structures for Europe

WP 2014- Call YOUNG-2014/2015: Young Generation in an Innovative, 30

2015 Inclusive and Sustainable Europe

Call REFLECTIVE-2014/2015: Reflective Societies: Cultural 51 Heritage and European Identities

Call INT-2014/2015: Europe as a Global Actor 43

Call INSO-2014/2015: New forms of innovation 65

Other Actions 68

Call CO-CREATION-2016-2017: Co-creation for growth and 58 inclusion

WP 2016- Call REV-INEQUAL-2016-2017: Reversing Inequalities and 53

2017 Promoting Fairness

Call ENG-GLOBALLY-2016-2017: Engaging together globally 44

Call CULT-COOP-2016-2017: Understanding Europe – Promoting 93 the European Public and Cultural Space

Other Actions 82

Total 2014-2017 640

Source: Work Programmes 2014-15 and 2016-17.

As of 1 January 2017, the state of play is the following: the EU contribution allocated to the implementation of the calls included in Work Programmes 2014-2016 and which have been closed at the date of 1 Janaury 2017 has been EUR 333.4 million, about 28% of total expected budget allocated to Societal Challenge 6, which is around EUR 1.2 billion for the period 2014-2020. 121 projects have been retained for funding and all of them are ongoing. In addition SC6 has financed 66 projects under SME phase 1 and 17 projects under SME phase 2. The programme has been implemented through a mix of instruments:

 64 Research and Innovation actions (RIA - action primarily consisting of

activities aiming to establish new knowledge and/or to explore the feasibility of a new or improved technology, product, process, service or solution), representing

53 % of the total number of financed projects (excluding SMEs);

 41 Coordination and support Actions (CSA - actions consisting primarily of

accompanying measures such as standardisation, dissemination, awarenessraising, communication and networking), accounting for 34% of the total;

 13 Innovation actions (IA - action primarily consisting of activities directly

aiming at producing plans and arrangements or designs for new, altered or

improved products, processes or services), projects for 11% of the total;

 3 ERA-Net co-fund (actions under Horizon 2020 designed to support publicpublic

 partnerships, including joint programming initiatives between Member

States);

 83 projects under the Small and Medium-sized Enterprises instrument (SME

instrument has three steps: concept and feasibility assessment, R&D demonstration and market replication and commercialisation).

In terms of EU contribution to signed grants per type of action: 54% goes to RIAs, 20% to CSAs, 13% to IAs, 13% to SME projects and to the ERA-NET Co-fund. The average EU contribution to signed grants (excluding the SME instrument) is EUR 2.5 million.

The success rate in terms of proposals and funding is relatively low (Table 208), especially for the RIA (5.2% for the proposals and 5.8% for the funding) and for the

SME Instrument Phase 1 (4.5% proposals and funding) 120 . The average success rate for

SC6 is 6,7% in terms of funding and 5,6% in terms of proposals. These figures are below the overall Horizon 2020 average success rates and are the lowest among the Societal

Challenges. 121 In comparison the success rate for FP7 SSH – Cooperation programme

was around 10% both in terms of proposals and funding (the lowest under the FP7

Cooperation programme). 122

Table 208 - Key data on proposals per type of action for SC6: Nr. of eligible and retained proposals, EU contribution requested and success rates (as % of proposals submitted, and as % of budget available)

Type Of Nr of Nr of EU Contribution EU Success Success Action Eligible Retained requested by Contribution Rate Rate Proposals Proposals Eligible to retained Proposals Funding Proposals (EUR proposals

million) (EUR million)

CSA 293 41 517.1 67.7 14% 13.1%

ERA-NET- 3 3 18.2 18.2 100% 100.0% Cofund

IA 249 13 698.4 42.0 5.2% 6.0%

RIA 1 223 64 3 101.2 178.9 5.2% 5.8%

SME-1 1 480 66 74.0 3.3 4.5% 4.5%

SME-2 405 17 544.2 23.2 4.2% 4.3%

Total 3 653 204 4 953.1 333.4 5.6% 6.7%

Source: CORDA data, 1 January 2017, Success Rates by Type of Action (General).

The budget was allocated through 61 topics included in 27 closed calls for proposals on the date of 1 January 2017. At the time of the interim evaluation, SC6 has signed 204 projects for a total EU contribution of EUR 333.4 million. The minor differences in terms of number of projects and budget with Table 209 are due to the fact that some projects are still in the process of signing the grant agreement.

120 Eligible proposals: Success rate is equal to the number of retained proposals divided by the number of eligible

proposals. EU financial contribution: Success rate is equal to the EU financial contribution going to retained proposals divided by the EU financial contribution requested by eligible proposals.

121 For more detailed analysis please see SWD(2016)376 Horizon 2020 Annual Monitoring report 2015.

122 Horizon 2020 has a challenge based approach and differ from the disciplines approach used in the past. However it

is useful to compare the FP7 – SSH cooperation programme with the SC6 in order to have benchmarks with the previous FP. Data on success rate in FP7 SSH cooperation are available on Corda and FP7 ex-post evaluation.

Table 209 - Key data on signed grants per type of action for SC6: number, EU contribution, time-to-grant, projects' total costs, % of EU contribution in projects

Type of Nr of EU Share of EU Nr of Share of Average Action Signed Contribution Contribution Grants Grants Signed Project EU Grants to Signed to Signed signed within TTG Contribution Grants (EUR Grants (in within 8 Benchmark to signed million) Programme months (in all Signed grants Part) (TTG) Grants) (EUR million)

CSA 39 63.5 15.3% 24 61.5% 1.5

ERA- 3 15.0 4.7% 1 33.3% 5

NET-

Cofund

IA 16 50.9 13.5% 13 81.3% 2.9

RIA 70 191.1 60% 59 84.3% 2.7

SME-1 53 2.7 0.7% 53 100% 0.1

SME-2 13 18.9 5.8% 12 92.3% 1.5

Total 194 342.1 100% 162 83.5% 2.5 (no SME)

Source: CORDA data, 1 January 2017, Selected Projects and Signed Grants by Type of Action.

The time to grant indicator (TTG) shows that around 83% of the grants were signed within the establishment objective of 8 months. The TTG are very close to the benchmark for the SME, RIA and IA. For the CSAs almost 60% of the grants are signed within the 8 months. In contrast, TTG for the ERA-NETs is below the benchmark.

O.2.2. Participation patterns

O.2.2.1. Participation per type of organisation

The selected proposals represent a total of 1436 participations, mobilising 996 distinct

participants 123 (Table 210). In particular the majority of the participants in the signed

grants belong to the realm of publicly funded science and research: 35% of them are affiliated with higher or secondary education establishment (HES) and 18% are research organisations (REC). 24% of the participants come from private for profit entities (PRC), such as for profit organisations, SMEs or consultancies. Public body (PUB) like ministries, regional and local authorities, represent 12%. The shares of the various activity types differ depending on the call and the topic in question.

123 If the proposal is successful and is funded it becomes a project, which is implemented by one or more participants.

And a participant might be involved in other projects, in which case it has a number of participations.

Table 210 - Key data on participation per type of organisation for SC6: number of participants, of project coordinators, of newcomers, of participations, and EU contribution to participations (in million Euros)

Legal Nr of Nr of Nr of Nr of Average EU Entity Participant Projects Newcomer Participation Participati Contribution to

Type s in Signed Coordinators s in Signed s in Signed ons per Participations in Grants in Signed Grants Grants Participan Signed Grants Grants t (EUR million)

HES 345 71 25 589 1.7 156.0

OTH 109 4 76 125 1.2 23.7

PRC 237 81 150 261 1.1 60.3

PUB 124 7 36 175 1.4 32.5

REC 181 31 41 286 1.6 69.5

Total 996 194 328 1 436 1.5 342

Source: CORDA data, 1 January 2017, Participants and Participations by Legal Entity.

The success rate of applicants and funding differs across the various types of organisations. HES, REC and PUB show a higher success rate in comparison with OTH and PRC. This is also true when one considers the success rate of the number of applicants in each category and the number of applications submitted in each category.

O.2.2.2. Attraction of new participants / newcomers

Newcomers are defined as Horizon 2020 beneficiaries that did not participate in FP7. Under SC6 there are 328 newcomers representing 33% of the total number of participants. Most newcomers are coming from private (63%) or others type of organisations (70%).

Table 211 - Newcomers per type of organisation, Societal Challenge 6

Participant type Nr of Participants in Nr of Newcomers in Share of Newcomers in Signed Grants Signed Grants Horizon 2020 Signed Grants

HES 345 25 7%

OTH 109 76 70%

PRC 237 150 63%

PUB 124 36 29%

REC 181 41 23%

Total 996 328 33%

Source: CORDA data, 1 January 2017.

In terms of geographical distribution, the highest proportion of the new participants can be found in the EU candidate countries and associated countries.

Table 212 - Newcomers per country groups, Societal Challenge 6

Country Groups Nr of Participants in Nr of Newcomers in Share of Newcomers in

Signed Grants Signed Grants Horizon 2020 Signed Grants

ASSOCIATED & 79 35 44%

CANDIDATE

EU-13 140 48 34%

EU-15 676 215 32%

THIRD 101 30 30%

COUNTRIES

Total 996 328 33%

Source: CORDA data, 1 October 2016.

O.2.2.3. Geographical participation patterns

A summary of the geographical participation in SC6 is presented in the tables below. A higher percentage of the EU funding goes to EU Member States (92%) and in particular 83% goes to EU-15 and 9% to the EU-13. Associated countries and candidate countries receive 5%, while the third countries 3%.

Table 213 - Key data on participants 124 per group of country EU-28, EU-13, EU-15,

Associated countries, Third Countries for SC6: number of participants, of project coordinators, of newcomers, of participations, and EU contribution to participations

Country Groups Nr of Nr of Projects EU Contribution to

Participants in Coordinators in Signed Participations in Signed Signed Grants Grants Grants (EUR million)

ASSOCIATED 79 8 17.1 COUNTRIES &

CANDIDATE

EU-13 140 15 31.0

EU-15 676 171 284.2

THIRD COUNTRIES 101 0 9.8

Total 996 194 342.1

Source: CORDA data, 1 January 2017, Participants and Participations by Country group.

The success rate of applicants and funding differs across the various groups of countries. One can observe that the success rate of funding for EU-15 is 7.2% while for EU-13 is 4.4%.

124 Participants from a country: Number of distinct (unique) organisations from a country. (ex. Company "X"

participating in two signed grants will only be counted once).

Table 214 - Success rates (as % of applicants and as % of budget available) per group of country for SC6

GROUP Success Rate of Success Rate of Success Rate of Funding Applicants Applications (Applicants)

AC COUNTRIES 9.9% 6.9% 6.0%

EU-13 7.9% 5.3% 4.4%

EU-15 11.7% 7.0% 7.0%

THIRD_PARTY 15.5% 13.0% 9.4%

Total 11.1% 7.0% 6.7%

Source: CORDA data, 1 January 2017, Applicants and Applications by Country groups (General).

Table 215 presents for each EU country the number of participants, coordinators, newcomers and EC contribution in signed grants. It is important to underline a concentration of the funding (68% of the total EU contribution) in the top 7 countries, all belonging to EU-15. In particular UK alone receives almost 18 % of the total EU contribution, followed by Germany, Italy, Spain, the Netherlands, Belgium and France.

Table 215 - Key data on participation per country for SC6: number of participants, of project coordinators, of newcomers, of participations, and EU contribution to participations (in million Euros)

Country Nr of Nr of Projects Nr of Nr of Average EU Contribution Participants Coordinators Newcome Participa Participatio to Participations in Signed in Signed rs in tions in ns per in Signed Grants Grants Grants Signed Signed Participant (EUR million) Grants Grants

Austria 33 5 10 54 1.6 15.3

Belgium 55 10 22 73 1.3 18.5

Bulgaria 8 1 11 1.4 1.4

Croatia 10 3 10 1.1 1.3

Cyprus 5 2 1 11 2.2 2.4

Czech 8 2 14 1.8 2.5 Republic

Denmark 15 3 2 26 1.8 7.0

Estonia 9 3 1 18 2.0 4.2

Finland 19 6 4 25 1.3 7.7

France 42 9 13 64 1.5 17.7

Germany 87 21 29 128 1.5 40.6

Greece 45 12 15 74 1.7 16.4

Hungary 16 2 4 28 1.9 4.2

Ireland 14 2 3 25 1.8 5.4

Italy 94 23 35 130 1.4 34.5

Latvia 6 2 9 1.5 1.2

Lithuania 12 2 7 14 1.2 1.2 Country Nr of Nr of Projects Nr of Nr of Average EU Contribution Participants Coordinators Newcome Participa Participatio to Participations in Signed in Signed rs in tions in ns per in Signed Grants Grants Grants Signed Signed Participant (EUR million) Grants Grants

Luxembour 5 2 1 10 1.8 2.8 g

Malta 2 3 1.3 0.5

Netherland 33 15 9 61 1.9 23.5 s

Poland 27 1 11 36 1.3 5.5

Portugal 22 5 7 37 1.8 6.2

Romania 13 1 5 17 1.3 2.5

Slovakia 12 1 5 16 1.3 1.9

Slovenia 12 3 6 16 1.3 2.2

Spain 81 23 30 105 1.3 21.7

Sweden 25 3 5 38 1.6 10.4

United 106 32 30 156 1.5 56.5 Kingdom

Total 816 186 263 1209 1.5 315.2

Source: CORDA data, 1 January 2017, Participants and Participations by EU-28 Member State.

O.2.2.4. International cooperation

International cooperation in research and innovation is a key cross-cutting priority of Horizon 2020 and SC6 has a significant international dimension. A total of 970 applications coming from third countries were received, incorporated in 731 project proposals, with a proposal success rate of 11.8%. The share of participations of and the share of EU contribution to entities from third-countries in SC6 (data up to October 2016) is the highest among all Horizon 2020 Work Programme parts, with 100 participations in signed projects corresponding to 7.4% of all participations in SC6 and EU contribution of EUR 8.1 Million, corresponding to 2.6% of all EU contribution to participants in SC6 signed projects. The main participating countries are Brazil (10 participations), China (7), USA (7), Australia (6) and South Africa (6).

Furthermore, the Commission has a flagging system in order to inform applicants that certain topics are particularly relevant for international cooperation. The share of budget allocated to projects financed so far under topics flagged as particularly relevant for international cooperation is 19.4% amounting to 46.8 million (WP 2014-2015). In comparison with the other Societal Challenges SC6 is performing fairly well.

O.2.3. Cross-cutting issues

In SC6, 31.7% (EUR 107.8 million) of the budget has been so far allocated to Sustainable development topics (the target for Horizon 2020 is at least 60%), 1.7% (EUR 5.7 million ) to Climate related topics and only one project covers biodiversity issues. 34.5 % (EUR 110 million) of the EU contribution to SC6 is ICT Research and Innovation related.

As regards the integration of Social Sciences and Humanities (SSH) under SC6 in Horizon 2020, in the period 2014-2017 90 topics and other actions have been classified

as being relevant for SSH researchers. 125 It can be observed that within the projects

selected under these topics, 79% of partners indeed have an SSH background, receiving

75% of the EU contribution for these topics. 126

In the 179 SC6 projects 46% of the total workforce are women researchers 127 (5557 out

of 11948), as well as 71% (128 women out of 179 coordinators) of coordinators are women (128 women out of 179 coordinators); 40% (11 women out of 27) of the members of the SC6 Expert Advisory Group are women. Moreover, more than 40% of the signed grants take into account the gender dimension in the research content.

Within the projects of SC6, 13.5% (EUR 42.9 million) of EU contribution is allocated to innovation actions. Within the innovation actions, 100% (EUR 42.9 million) of EU financial contribution focus on demonstration and piloting activities. 64 participants involved in SC6 are SMEs, start-ups or individual entrepreneurs.

O.3. R ELEVANCE

O.3.1. Is SC6 tackling the right issues?

O.3.1.1. The relevance of SC6 given the challenges to address

SC6 aims to tackle relevant needs and problems faced by European Union. It is different from other challenges because it often needs to rapidly design policies to tackle current and contemporary challenges and problems. FP 7 had been designed before the start of the economic and financial crisis of 2007 that had started in the US and subsequently affected Europe and the rest of the world. When Horizon 2020 was being designed in the 2012-2013, Europe had not yet overcome the economic crisis which had led to unemployment rates of 12% in general and 20% among the youth, creating considerable social unrest in several European countries. Europe faces huge challenges in reducing inequality and social exclusion. 80 million people are at risk of poverty and 14 million young people are not in education, employment or training. Subsequently, the migration crisis with more than a million migrants on Europe’s borders became a major new economic, social and geopolitical challenge. The rise of violent radicalism and terrorism in Europe during the first phase of the Horizon 2020 programme needed the research community in several disciplines of SC 6 to propose their analyses and solutions, both for the short and the long term. This challenge driven, problem solving approach was therefore applied to growing economic and cultural interdependencies, radicalisation and violence, ageing and demographic change, social exclusion and poverty, integration and disintegration, inequalities and diversity, migration flows, growing digital divide, the need to foster a culture of innovation and creativity in society and enterprises, a decreasing sense of trust in democratic institutions and between citizens within and across borders. SC6 research priorities are strongly oriented to these needs and EU policy and social goals. Importantly, the SC6 tackles issues concerning the own role and future of the European Union, including its political system, governance and integration. The

125 This information is based on projects already financed under calls closed in 2014 and 2015. For 2016 and 2017,

information will be only available respectively in 2017 and 2018.

126 Data and details on the methodology used can be found in the monitoring reports on SSH projects in 2014 and

2015.

127 Workforce includes people actively participating in and paid by the EU project

way in which Europe advances in these areas will have a strong impact on the development of particular policies and innovations in every field of knowledge. As pointed out above, the SC 6 programme continues to analyse and propose policies for new challenges as they appear while carrying on the necessary work on areas that were decided at the start of the programme and that still remain relevant, as outlined above.

Under the Work Programme 2014-2015, three strategic priorities had been identified:

 The first priority is to gain a greater understanding of the societal changes in

Europe and of their impact on social cohesion, as well as to analyse and develop social, economic, political inclusion and positive inter-cultural dynamics in the

EU and with international partners, through cutting-edge science and interdisciplinarity, technological advances and organisational innovations. The fight against the crisis needed new ideas, strategies and governance structures which will help Europe to surmount its current severe deficiencies (Call EURO). Given the current pessimism in many countries, the political leadership of the European Union, through the Horizon 2020 Strategy deemed it important that Europe defines new visions for its development and these efforts should, as a matter of priority, stress the integration of the young generation in an innovative, inclusive and sustainable Europe (Call YOUNG). The global dimension is also included with a particular focus on the strategic choices Europe should enhance in order to further its research and innovation capacities and strengthen its principles and impact in several important regions of the world (Call INT).

 The second priority is to foster the development of innovative societies and

policies in Europe through the engagement of citizens, civil society organisations, enterprises and users in research and innovation, as well as the promotion of coordinated research and innovation policies. They also involve enterprises, young entrepreneurs, incubators, universities and innovation centres and other relevant actors through support to open innovation, business model innovation,

public sector innovation and social innovation (Call INSO).

 The third priority is to contribute to an understanding of Europe's intellectual

basis, its history and the many European and non-European influences, as an inspiration for our lives today. In challenging times for its internal coherence, Europe should improve the understanding of its cultural heritage and of its identities in order to strengthen cohesion and solidarity and to encourage modern

visions and uses of its past (Call REFLECTIVE).

The Work Programme 2016-2017 tackles four key major challenges currently faced by the European Union, which demonstrates the flexible capacity of the Programme to adopt to evolving challenges as they occur. This work programme was particularly driven by the following challenges:

 Migration - the large influx of refugees and other migrants largely caused by

conflicts, geopolitical shocks and poverty poses short, medium, and long term challenges. The challenges range from immediate hosting of refugees to the lasting integration in the EU of all legally staying migrants (16 Million earmarked

in 2017 for Topics on migration and mobility, spread over several calls, REV- INEQUAL, ENG-GLOBALLY and CULT-COOP).

 Economic recovery and inclusive and sustainable long-term growth. The EU is

facing the need to identify the obstacles to and to find untapped sources of growth and employment, renewing the legitimacy of public policy-making, especially through greater citizens’ involvement, and of delivering better public services for

all. (Call - CO-CREATION).

 The rise in inequalities in Europe. Large disparities in human and social

capacities are counterproductive to a sustainable and creative economy and participatory governance and inclusion. They jeopardise economic growth while threatening, through violent radicalisation, the very foundations of democracy,

the rule of law and respect of human rights in Europe (Call REV-INEQUAL).

 The global environment in which the EU operates is constantly evolving. Recent

developments show just how dynamically the strategic and geopolitical contexts are changing. Given the geographical proximity of many of these developments, they pose severe security threats to the EU and represent intricate challenges, but also opportunities to its risk analysis, anticipative, proactive and reactive capacities. At the same time, it is important to promote the position of Europe on the global scene, attract international partners to European projects, enhance R&I exchanges and dialogues, and strengthen the European R&I presence in strategic

partner countries and regions (ENG-GLOBALLY).

 A better understanding of Europe's cultural and social diversity and of its past

will inform the reflection about present problems and help to find solutions for shaping Europe's future. The resilience and cohesion of European societies are strongly conditioned by beliefs and identities, as well as by collective representations and constructions of past and present realities and expectations

about the future. (CULT-COOP).

With regard to the broader international sustainability agenda, activities funded under Societal Challenge 6 are expected to have an impact on the implementation of the United Nations (UN) Sustainable Development Goals (SDG), in particular 'Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels'. Societal Challenge 6 also addresses two other SDGs: 'Reduce inequality within and among countries' and 'Make cities and human settlements inclusive, safe, resilient and stable'.

O.3.1.2. The relevance of SC6 to address European objectives

The calls and topics within SC6 are clearly aligned with the European priorities as outlined in O.3.1.1. Table 216 shows the strength of the relationships between the different research programmes as outlined in the calls and the objectives outlined in the different political and legal documents of the EU. After the start of the Horizon 2020 programme, European Commission President Juncker outlined a set of 10 priorities within the Europe 2020 Strategy, and the Work Programmes outlined above addressed these priorities over the 2014-2017 period, and continue to do so for preparing the 2018- 2020 Work Programme. It reflects a wide coverage of the EU targets, Flagships and new priorities of the European Commission, as well as a stronger focus in four of the EU2020 targets (Employment, R&D, Education, Poverty) and four of the seven European Flagships (Innovation Union, Youth on the move, New skills and jobs, Poverty), and also the creation of the European Research Area (ERA).

SC6 mainly contributes to the following ‘Juncker’ priorities derived from the Europe 2020 strategy 1 (growth, jobs and investments, reducing inequalities), 2 (A connected digital single market), 7 (Justice, Fundamental Rights and Trust), 8 (Migration Policy), 9 (a stronger global actor contributing to global justice and stability) and 10 (A Union of Democratic Change - with focus on strengthening citizenship). The calls were designed to reflect these priorities.

Table 216 - Correspondence between policy goals and the topics

EURO YOUNG REFLE ENG- CULT- CTIVE INT INSO CO- REV- CREAT INEQ GLO B COOP

EU2020 targets

Employment ++ ++ + + ++

R&D + + + + + + + + + Climate change + + + ++

Education + ++ ++ + ++ + Poverty + ++ +

EU Flagships

Digital agenda + + +

Innovation Union + + + + +

Youth on the move + ++ + +

Resource efficient + +

Industrial policy + +

New skills and jobs + + + +

Poverty + ++ +

Juncker priorities

Jobs, Growth & Investment + + + + + +

Digital Single Market + + + +

Climate Action + + + +

Deeper & Fairer Internal Market + +

Deeper & Fairer Economic &MU +

Free Trade US Agreement +

Justice & Fund. Rights + + + + New Policy on Migration + ++ ++ + Stronger Global Actor + ++ + ++

Democratic Change + + ++

Moedas’3O Strategy

Open Innovation + + + ++ + + +

Open Science + + + ++ + + +

Open to the World + + ++ ++

Source: Analysis carried out by the IMPACT EV team and Unit B6.

In particular, in the 3 O’s strategy proposed by Commissioner Moedas to boost innovation and its funding within the Europe 2020 strategy, the steps taken were the following:

 Open innovation: the calls INSO, CO-CREATION, SME instrument were

introduced to enable greater collaboration between public research and private actors (both for profit and not for profit organizations), in particular to encourage

entrepreneurship and start-ups;

 Open science /access policies of publishing have been reinforced better diffusion

of the results and more and more dissemination is done towards stakeholders;

 Open to the world – the calls INT and ENG-GLOBALLY were strongly oriented

towards international cooperation and the sharing of the results of joint research.

O.3.2. Flexibility to adapt to new scientific and socio-economic developments

A diachronic analysis of the topics in the two published Work Programmes 2014-2015 and 2016-2017, shows that they have evolved and included new social needs as developed in 4.1.1. For instance, WP 2016-2017 and WP 2018-2020 (under preparation) reflects the increasing awareness of the topic of Migration, European Agenda on Migration and further European regulations. There are also emerging needs that the programmes do not fully cover and would need to be taken in consideration in further work programmes (2018-2020) or FP9. These are, among others, a continuance of the Refugee crisis in Europe and the future of the European Union after the Brexit, the potential consequences of the election of President Trump on Europe, and the necessity of reflecting on an European defence strategy. These are among the principal challenges facing the political leadership of the EU. The prioritisation of these areas will take place through the European political decision making process.

O.3.3. Addressing specific stakeholder needs

In preparing the Work programmes, the European Commission is ensuring adequate external advice and societal engagement as established in Article 12 of the regulation concerning the Horizon 2020 programme, which specifies that advice shall include forward-looking activities, targeted public consultations, including, where appropriate, consultations of national and regional authorities or stakeholders, and transparent and interactive processes that ensure that responsible research and innovation is supported.

The SC6 Work Programmes are prepared following a multilevel consultation process and largely draw on four sources. First, they take into account the views from the Expert Advisory Group for SC6. This group (several representatives of academia, research institutions, public authorities, NGOs, industry and business) identifies the major challenges to be addressed in the Work Programmes. Second, stakeholder inputs are

collected at several public events 128 where different research priorities are discussed such

as stakeholder's conference, participatory events and EU 'social platforms'. In total the Work Programmes reflect the views of around 6000 stakeholders coming from the academic world, the research community, public authorities, industry, trade-unions and non-governmental organisations. Third, the Work Programmes incorporate the policy issues highlighted by Member States and Associated Countries during Programme Committees (Member States’ forum to give input to the Commission on the Work Programmes). Fourth, the Work Programmes take stock of foresight studies, policy reviews and analysis of the coverage of the previous Work Programmes of SC6. This process allows taking into consideration different views, positions and interests in order to conceive a truly European Work Programme. The different stakeholders produce reports that are in the public domain, and each phase of the Work Programmes have taken into account the comments and suggestions of the stakeholders’ reports, in particular the synthesis of the discussions of the periodic consultations between the Program Committee for SC6 and the Commission.

For example for the preparation of the SC6 Work Programmes 2014-2015 and 2016- 2017, the European Commission sought advice and societal engagement from a wide range of actors (researchers, public authorities, industry, trade-unions, non-governmental

128 External advice and societal engagement reports: 2016-2017 and 2018-2020

organizations). 129 Most of this advice was from expert bodies and platforms. In view of

the preparation of SC6 Work Programme a gap analysis was notably performed to check to what extent the objectives and activities stated in the legal basis were covered by the previous Work Programmes (2014-2015 and 2016-2017). The analysis shows that topics and projects are in line with the overall objectives of the specific programme and no major gaps have been identified. However, as outlined in 4.2, major new evolutions in the international sphere need to be addressed on a priority basis by the political leadership of the European Union.

At the same time, the direct engagement of citizens (individually or through collective action) is less evident and/or expected to be mediated by the experts, as reports of attendance at meetings and conferences organised for the different projects show. This can represent a limitation for the programme to actually address the specific newly emerged stakeholder needs.

O.3.4. Other issues related to relevance

Based on the preceding analysis, it is possible to claim that Societal Challenge 6 programmes have been both proactive and reactive to outlining the research needs of current and future problems of the EU. This anticipation should be carried further through the foresight activities that are taking place through the Foresight Group in the Commission. The Foresight Group is even reflecting on ‘Unthinkable’ or Catastrophe Scenarios following the Brexit vote.

As far as selected projects are concerned, they are strongly aligned with the defined priorities even if still in the first phase and results are preliminary. The emphasis laid on the measurement of social impact is strongly present in the descriptions of selected projects.

O.3.5. Lessons learnt/Areas for improvement

The SC6 programme has a set of broad objectives closely aligned to the European priorities, as has been developed and explained extensively above. The programme is strongly policy-oriented and seeks to contribute to better governance and evidence-based policies to tackle the societal and political problems of Europe. One limitation could be the extent to which these priorities continue to be relevant for society all throughout or whether they can change, and how the programme is able to accommodate the new ones. So far, the 2016-17 and the 2018-2020 Work Programmes have responded strongly to evolving challenges. As has been pointed out above and in O.3.2, the anticipation of challenges needs to be strengthened through foresight activities.

O.4. E FFECTIVENESS

O.4.1. Short-term outputs from the programme

No project has been completed as of 1 of January and therefore, the 130 projects (excluding the actions under the SME instrument) are on-going. A survey was carried out

129 Societal Challenge 6 External advice and societal engagement reports: Towards the 2016 and 217 Work

Programme and Towards the 2018 and 2020 Work Programme of "Inclusive, Innovative and Reflective Societies" of Horizon 2020.

by the FP7 project Impact EV that analysed a sample of 56 Horizon 2020 SC6 projects funded under the WP 2014-2015 more in depth. The survey points out the following relevant elements:

 As part of efforts to promote Responsible Research & Innovation (RRI) across

Horizon 2020, SC6 fostered the co-creation of scientific agendas and scientific contents, as demonstrated by the projects where citizens, Civil Society Organisations (CSOs) and other societal actors contributed to the co-creation of scientific agendas and scientific contents. This is demonstrated in Table 219 from the rise in the participation of other participants from 7 to 11% between FP7 and

Horizon 2020.

 Regarding the gender dimension in research and innovation content, 71.4% (40

funded projects out of 56) have included a sex and/or gender analysis as part of their research or innovation activities. In most cases, gender is one of the studied dimensions and 12.5 % (7 projects) highlight this in their abstract. None of the

projects has gender/sex as its central focus of the research.

 As the projects are in early-stages, publications in journals are still scarce.

According to the data available as of 1 October 2016, 21 articles have already been published in peer reviewed scientific journals. In FP7, the SC6 projects had

an average of 16 peer reviewed articles published 130 so this initial data may not be

illustrative of the general expected trend. On the other hand, 15 of the already published articles are provided in Open Access. Four of the 56 projects have publications available in the OpenAire platform. Apart of the articles in scientific journals, one can also find in the webpages of the projects a more than 100 working papers, reports and newsletters, some of them likely to serve as the

basis for future publications.

 Around 50% of the sample of projects selected, funded under the several calls of

the SC6, have already developed or expected to develop datasets/ databases, most of them available via open access in the Projects websites. These cover a wide range of fields, from policies to public European speeches (such as the EUSpeech database produced by the EUENGAGE project with more than 18,000 speeches from EU leaders), fiscal indicators, fiscal variables, and others. Some of these datasets include interviews to relevant stakeholders and some of them enable the co-creation and participation of the general public, such as the wikisite

from the TransSol Project, about forms and structures of transnational solidarity.

 A number of projects are expected to produce simulation models and softwares.

These can be related to financial and growth strategies, such as the user-friendly fiscal simulation framework that will be produced by FIRSTRUN, which will “compute the minimum fiscal effort that the various corrective rules in the reinforced Stability and Growth Pact imply in different macroeconomic

conditions.”

 None of the projects that have filled out the survey has responded that there are

potential patents resulting from the project, while 8.9% of the respondents anticipate or have achieved to produce new IPRs as a result of the project. Particularly, 14.3% of the respondents foresee creating and commercializing new

products, processes or services as a result of the project.

 With regards to human capital development, several projects do include training

services and products, ranging from summer schools to Master Modules (EL_CSI, on Diplomacy in the Global Area), e-modules, presentations and

lectures.

130 Ex-post evaluation of the FP7 Cooperation in the Socioeconomic Sciences and Humanities

O.4.2. Expected longer-term results from the programme

Based on the survey (23 respondents), SC6’s activities are expected to lead to:

 Creation of spin-offs: 30.4% of the projects foresee to set up spin-offs as a result

of the project.

 17.4% of the projects have SMEs which have introduced or are foreseen to

introduce innovations to the company or the market as a result of the project.

 52.2% of the projects incorporate training activities or other non-commercial

transference plans as a result of the project.

 Regarding the expected political impact at the international, European, national,

regional and/or local level, 91.3% are aimed at making political recommendations based on scientific evidence obtained, and 65.2% work in order to have an impact

on the formulation of new policies.

Difficulties of prediction: As far as Societal Challenge 6 is concerned, the projected social and economic impacts, for example on the creation of spin offs, on employment or the development of new innovation, are difficult to measure (in terms of causality with the projects financed), in particular because they might happen at a point beyond the lifetime of the project. This needs to be taken into account in future ex post impact evaluation exercises. It is also difficult to predict if stakeholder collaboration across different types of organisations will last beyond the duration of the projects.

O.4.3. Progress towards attaining the specific objectives

The outputs and expected results of the funded projects are contributing to attain the specific objectives of the SC6 programme:

 “To gain a greater understanding of the societal changes in Europe and of their

impact on social cohesion and to analyse and develop social, economic and political inclusion and positive inter-cultural dynamics in the EU and with international partners”: The projects are creating new knowledge about the main dynamics and challenges for the future of Europe and its citizens. The approach of the projects integrates emergent developments such as the effects of the Brexit, as is done by at least two projects under the EURO-4 Topic (EUENGAGE and

ENLIGHTEN).

 “To foster the development of innovative societies and policies in Europe through

the engagement of citizens, civil society organisations, enterprises and users in research and innovation and the promotion of coordinated research and innovation policies”: The funded projects are engaging a wide range of public

and private institutions, non-profit associations, as well as the engagement of endusers during the projects. In particular, 73.9% of the projects responding to the questionnaire state that they collaborate with public institutions, 78.3% with private, non-profit associations and 39.1% with companies. 65.2% engage with end-users during the project, including groups that traditionally have not fully participated in the co-creation of scientific knowledge and agendas, such as the youth. The piloting of services and other results of the projects is lower among the analysed selection, being used by 26.1% of the projects.

 “To contribute to an understanding of Europe's intellectual basis, its history and

the many European and non-European influences, as an inspiration for our lives today”: The projects provide open access to new digital tools that enable the exploration and reflections about heritage, values and conflicts that have shaped

our identities and the present.

O.4.4. Progress towards the overall Horizon 2020 objectives

O.4.4.1. Fostering excellent science in scientific and technological research

The success rates on the number of eligible projects as compared to the number of funded projects (in Corda data), show that excellence in research and innovation remained a driving force and continued the trend towards continuously higher standards of excellence of the previous FPs. In the consortia and disciplines present in SC6, it has been observed that there is a large number of perfect or near-perfect score, of 15 or just below, which cannot all be selected, and many projects with high scores did not receive financing after the final rankings. Hence there is a need to increase the number of projects financed in the calls, which has become an objective for 2018-2020. This striving for excellence creates best practices in the European research community and is a source of motivation. Table 219 shows the rise in the presence of stakeholders who are not academic or research institution in a systematic way, thus showing that the extension of stakeholder involvement is taking place, even though it is too early to measure its impact under Horizon 2020.

It also needs to be remarked that CSA (Coordination and Support action) projects have a strong relationship with the ERA promotion as they promote exchanges and foster the scientific community rather than producing specific new knowledge. The mobility and exchanges generated by these projects promote both excellence, cooperation and visibility. We do not yet have a project based empirical analysis of human mobility, what one may observe from the projects monitored is that the projects induce conferences and movement in the lifetime of the projects, thus increasing European exchanges.

Box 24 - Contribution to the achievement and functioning of the ERA

 More effective national research systems: The European dimension induces the national systems towards best practices and excellence at European and international levels, by increasing the number of publications in internationally ranked journals (an evolution based on data of past programmes)

 Optimal transnational co-operation and competition: the stakeholders are obliged to seek cooperation with institutions in countries with whom they did not have contacts before, thus greatly enhancing European cooperation.

 An open labour market for researchers: All European programmes greatly contribute to increasing the mobility of researchers and students at advanced levels preparing for research in disciplines related to SC6

 Gender equality and gender mainstreaming in research: gender requirements increase the participation of female scholars and students in European programmes, as has been pointed out earlier.

 Optimal circulation and transfer of scientific knowledge: there is an increasing sharing of knowledge, information and data between research institutions as the history of projects over several FPs demonstrate

O.4.4.2. Boosting innovation, industrial leadership, growth, competitiveness and job creation

Innovation and Growth are recurrent issues in the topics and projects covered by SC6. Both RIAs and CSAs are explicitly contributing to induce innovation and fruitful relationships between research and industry. This can be through co-location of industry laboratories in universities (i.e. Science2Society, Horizon 2020 project that will assess the mechanisms through which universities, research organisations, society and industry collaborate to create value) and identifying and promoting examples for the specific incentives for European SMEs. Based on the programme focus, the early outputs and expected results from projects and the participants selected so far, the programme appears to be on track to contribute addressing this Horizon 2020 specific objective, but as pointed out previously, concrete results are not yet available. SC 6 will also incorporate projects to evaluate the impact of the Digital Single market on economic and social value added in the cultural sphere.

O.4.4.3. Addressing the major societal challenges

The analysed projects respond to the Societal Challenge as expected as has been explained extensively in the relevance section. SC 6 projects provide a considerable body of informed theoretical and evidence based analysis of Europe’s major problems and challenges, even though results are in an early stage. The integration of the social sciences and humanities disciplines as a cross cutting issue reinforces the role of SC 6 in the overall structure of Horizon 2020.

O.4.4.4. Spreading excellence and widening participation

From the sample of SC6 projects analysed more in-depth through a survey of coordinators the average number of institutions within each project consortium is 11, and the average number of researchers is 32.5 per project. As shown in section O.1, these teams are usually formed fully covering geographic diversity. Moreover, the teams are cross-disciplinary and integrate relevant institutions and researchers, offering relevant opportunities for junior researchers’ careers. As Corda and Table 215 data shows, even though there is a rise in the number of newcomers and the EU-13 are entering the ERA, the dominance of a group of 7 West European countries and their key institutions is a constant over several FPs.

O.4.4.5. Science with and for society

Projects make a considerable effort to reach the specific stakeholders and the general audience with web-based platforms, social media and communication resources. Even in their early stage, from the sample of SC6 projects analysed, 45% of the projects have launched a Twitter account, and several projects do have or have foreseen to produce films or other media content. For example the project DANDELION (Promoting EU funded projects of inclusive, innovative and reflective societies) aims to support the uptake and valorisation of Inclusive, Innovative and Reflective Societies (IIRS) research and improve its dissemination towards citizens, policy makers, academia and media. This will be achieved through a series of innovative and creative communication activities targeted at a range of audiences.

O.4.4.6. Science for policy

All the reviewed projects do consider the relevance of their outcomes to provide a basis for evidence-based policies in the diverse fields related to SC6. All the projects responding to the questionnaire have stated that they will produce policy recommendations, and seven of the on-going projects have already Policy briefs in their webpages. Collaborations with policy makers at both national and European level are described in most of the approved projects. For example the Action Plan on the

integration of third country nationals 131 takes into account recommendations arrived from

the migration policy review of projects under FP7 and SC6.

O.4.5. Early success stories

Project title: ‘Quality of jobs and Innovation generated Employment outcomes’

Project duration: April 2015 – March 2018

Project budget (EU contribution): EUR2,498,869

Type of action: Research and innovation actions

QUINNE project also address the topic EURO-2-2014: The European Growth Agenda. The project investigates how job quality and innovation mutually impact each other at the organization level, and what employment outcomes result from this interaction i.e. how more and better jobs are created. The employment outcomes are then tracked in terms of their impact on social inclusion and inequality. QuInnE will produce evidence-based advice on how to boost innovation and economic and employment growth in the EU,

along with an awareness of ensuing impacts on social inclusion and inequality. 132

Project title: SIMplifying the interaction with Public Administration Through

Information technology for Citizens and cOmpanies

Project duration: March 2016 – February 2019

Project budget (EU contribution): EUR 3 628 718

Type of action: Research and innovation actions

SIMPATICO addresses the topic “EURO-6-2015: Meeting new societal needs by using emerging technologies in public sector”. A seamless interaction with the public administration (PA) is crucial to make the daily activities of companies and citizens more effective and efficient, saving time and money in the management of administrative processes. In particular, online public services have an enormous potential for reducing the administrative burden of companies and citizens, as well as for creating saving opportunities for the public administration. This potential is however far from being fully exploited. Online services made available by the PA typically rely on standardized processes, copied from their offline counterparts and designed only from the public sector organizations’ own perspective. This results in online services that fail to adapt to the specific needs of citizens and companies. SIMPATICO addresses the issues above by proposing a novel approach for the delivery of personalized online services that, combining emerging technologies for language processing and machine learning with the wisdom of the crowd, makes interactions with the public administration easier, more

efficient and more effective. 133

131 COM(2016) 377 final i

132 Website QUINNE: http://bryder.nu/

133 Website SIMPATICO: http://www.simpatico-project.eu/

Project title: Reconsidering European Contributions to Global Justice (GLOBUS)

Project duration: June 2016 – May 2020

Project budget (EU contribution): EUR 2 400 000

Type of action: Research and innovation actions

GLOBUS is an international project (participants from China, Brazil, and South Africa) that aims to critically assess the EU’s impact on justice in a global system characterised by uncertainty, risk and ambiguity. Its research agenda directs attention to underlying political and structural challenges to global justice that are prior to the distributive problem, as well as to the fact that what is just is contested both by theorists and policy makers. It provides in depth knowledge of how the EU proceeds to promote justice within the specific fields of climate change, trade, development, asylum/migration and security while also speaking to the key horizontal issues of gender and human rights

within each of these fields. 134

O.4.6. Lessons learnt/Areas for improvement

It is too early to get a good overview of the potential impacts of successful projects in SC6. There are few available data on outputs and results from the projects as they are in their initial phase. However SC6 expected outputs in Horizon 2020 have evolved compared to FP7, towards a higher share of databases, specific tools and simulation models. This can make the SC6 more likely to lead to product innovations than in FP7. One encouraging change is the rise in the participation of stakeholder other than academic and research bodies. The projects submitted and selected go further than past FPs in proposing the measurement of social and economic impact, beyond proposing just publications or dissemination. Progress is slow in the extension of participation beyond the 7 leading countries and their institutions, and in the integration of more actors from Eastern Europe.

O.5. E FFICIENCY

O.5.1. Budgetary resources

The total budget allocation for 2014-2017 for SC6 amounts to EUR 640 million. This represents more than half of the total budget earmarked for the SC6 for the seven years (EUR 1.2 billion). Grant agreements were signed for 178 projects as of 1 October 2016 (including SME instrument) representing an EU contribution of EUR 317.6 million. The SC6 is the smallest theme under the overall Horizon 2020 Societal Challenges in terms of budget allocation. In the first Calls of the SC6 Work Programme there have been no relevant deviations from the budget expectations for each topic. The distribution of the allocated budget among the topics as shown in Table 207 reflects the close alignment with the European priorities. The budget appropriation is balanced among calls and topics addressing different challenges. However further flexibility should be allowed in order to align the research funding to evolving challenges as argued above.

The ratio between Calls for proposals and other actions (i.e. expert contracts, public procurements, grants to identify beneficiary, prizes) is reasonable because there is no concentration of excessive funds in any particular area. In terms of budget distribution per type of action: 60% goes to RIAs, 15% to CSAs, 13% to IAs, and the rest to ERA-

134 Website GLOBUS: https://www.globus.uio.no/

NET and SME instrument. Between 20% - 25% of the total budget was dedicated to other actions (i.e. Horizon 2020 prizes, Experts, Public Procurements, Grant to identify beneficiary, etc.). These other actions complement the calls by a set of specific smaller activities supporting in particular the implementation of the Innovation Union, the European Research Area and the strategy for international cooperation in R&I. The Work

Programmes also support COST 135 , a European cooperation in Science and Technology

network, through a grant to an identified beneficiary.

This Programme part is implemented mainly by DG RTD and by DG CONNECT. The implementation of the Research and Innovation Actions has been delegated to the Research Executive Agency (REA), while the ERA-NET and the Coordination and Support Actions (CSA) were kept in the parent DGs (DG RTD and DG CONNECT).

The time-to-grant indicator for the Societal Challenge 6 is 83.9% (Horizon 2020 average: 92.4% excluding ERC projects).

The Key Performance Indicators which are particularly relevant for the Societal Challenges are:

 Number of publications in peer-reviewed high impact journals

 Number of patent applications and patents awarded

 Number of prototypes and testing activities

 Number of joint public-private publications

 New products, processes, and methods launched into the market

The KPIs are reported by Horizon 2020 beneficiaries during and after the project. Though still early, several publications have been attributed to Societal Challenge 6. Further analysis is needed in terms of assessing the performance of the publications in high impact journals and share of joint public-private. For the last three KPI's data is not yet available.

The budget allocation is in line with the objective specified in the Specific Programme and has been calibrated in a coherent manner among the different calls, without major imbalances. The resources were also re-allocated in order to provide more impact on the current issue of migration in the WP 2016-2017 by adding 5 new topics on migration (EUR 11 million) in addition to the exiting one (EUR 5 million) coming from a reallocation of budgetary resources.

O.5.2. Programme's attractiveness

O.5.2.1. Mobilisation of stakeholders

Simplification is one of the major features of Horizon 2020, which brings a new, userfocused approach to the EU's research and innovation funding policies. The programme's simpler design and rules have been supplemented by improved implementing procedures, in an effort to make the funding programme more attractive and easier to navigate.

The success rate of applicants and funding differs across the various types of organisations. HES (19.5% for applicants and 6.3% for funding), REC (11.1% for applicants and 10% for funding), PUB (16.1% for applicants and 14.2% for funding),

135 COST actions are financed by SC6 and SWAF Horizon 2020 parts.

PRC (5.5% for applicants and 4.7% for funding), and OTH (10.4% for applicants and 7.6% for funding). This indicates the high attractiveness for EU based organisations to participate in a cooperative programme specifically related to the SC6 topics, as well as their commitment to submitting proposals despite the low expectations of success.

Around 83% of the grants were signed within the establishment objective of 8 months. The performance is better for SME instrument, RIA and IA than for the CSAs where almost 60% of the grants were signed within the 8 months. In contrast, TTG for the ERA-NETs is below the benchmark.

Table 217 shows in more in detail the Horizon 2020 SC6 top 5 beneficiaries with higher EU requested contributions per type of organisation. It is possible to observe that among the higher or secondary education establishment: University of Utrecht, Univ. of Oslo, and Univ. of Oxford are very well represented. This concentration of funding is a normal result of a system of excellence combined with the existence of well-established networks in academic institutions, however there is a need according to all stakeholders to increase the number of the newcomers.

Table 217 - Top 5 SC6 entities by EU requested contribution within projects

HES - higher education establishment

Rank Legal Name of the Participant

Orga 1 UNIVERSITEIT UTRECHT

2 UNIVERSITETET I OSLO

3 THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF

OXFORD 4 TECHNISCHE UNIVERSITEIT DELFT

5 ALMA MATER STUDIORUM - UNIVERSITA DI BOLOGNA

OTH - other

Rank Legal Name of the Participant

Orga 1 All European Academies

2 DEUTSCHE AKADEMIE DER TECHNIKWISSENSCHAFTEN

3 STRAVV.DE - STRATEGISCHE VERWAWALTUNGSVERNETZUNG

DEUTSCHLAND 4 European Business and Innovation Centre Network

5 CONSEIL EUROPEEN DES APPLICATIONS DE LA SCIENCE ET DE

L'INGENIERIE (EURO-CASE) PRC - private

Rank Legal Name of the Participant

Orga 1 DUEDIL LIMITED

2 WITHLOCALS BV

3 VALUECHAIN.COM ENTERPRISES LTD

4 DATAWIZARD SRL

5 TRAFFIC ENFORCEMENT SYSTEMS LTD

PUB - public

Rank Legal Name of the Participant

Orga 1 ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT

2 OESTERREICHISCHE FORSCHUNGSFOERDERUNGSGESELLSCHAFT MBH

3 ECONOMIC AND SOCIAL RESEARCH COUNCIL

4 NORGES FORSKNINGSRAD

5 DEUTSCHE FORSCHUNGSGEMEINSCHAFT

REC - research organisations

Rank Legal Name of the Participant

Orga 1 DEUTSCHES ZENTRUM FUER LUFT - UND RAUMFAHRT EV

2 FRAUNHOFER GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN

FORSCHUNG E.V 3 JRC -JOINT RESEARCH CENTRE- EUROPEAN COMMISSION

4 ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS ANAPTYXIS

5 ZENTRUM FUR SOZIALE INNOVATION GMBH

Source: CORDA data, 1 January 2017.

The following table presents the major SC6 events in the last years as well as projects that promote active SC6 stakeholder participations.

Table 218 - Major SC6 Events in the past years and projects that promote active stakeholder participation

Major SC6 Events in the past years

SC6 Info-days in Brussels and at national level (Brussels and EU Member States capitals)

A Great start in life – Conference on early child education 30 November 2016

Addressing Radical Ideologies and Violent Extremism: The Role of Research" to be held on 26 September in Brussels.

Towards a new dynamic e-Government Action Plan 2016-2020 (Multi stakeholder event), 4 March 2016 in Brussels

Welfare, Wealth and Work for Europe, Brussels, 25 February 2016

Understanding and Tackling the Migration Challenge: The Role of Research, Brussels, 4-5 February 2016

Meetings of the Strategic Forum for International Science and Technology Cooperation SFIC plenary meetings: 2 December 2015, 2 March 2016 and 15 June 2016

Simple, secure and transparent public services, Luxembourg , 1-2 December 2015

Horizon 2020 and culture, Brussels, 25 November 2015

Social Innovation 2015: Pathways to Social Change - Vienna, 18-19 November 2015

Trust: European Research Co-creating Resilient Societies (Info Day) - Brussels, 29-30 October 2015

New Horizons for Cultural Heritage – Recalibrating relationships: bringing cultural heritage and people together in a changing Europe, 19 October 2015

Opening up to a new ERA of innovation – Brussels 22-23 June 2015

Digital Single Market Strategy - Bringing down barriers to unlock online opportunities, 8 May 2015 Brussels

Projects that promote active SC6 stakeholder participations

NET4SOCIETY - International network of National Contact Points (NCPs) for the Societal Challenge 6 in Horizon 2020

ACCOMPLISH - ACcelerate CO-creation by setting up a Multi-actor PLatform

for Impact from Social Sciences and Humanities

DANDELION - Promoting EU funded projects of inclusive, innovative and reflective societies

STEP- Societal and political engagement of young people in environmental issues

I-LINC- Empowering youth for employability

Source: RTD website and projects website.

Several of these events covered core areas of Europe 2020, Juncker's Priorities and the 3 O's of Commissioner Moedas. For example, the Trust Conference on European Research Co-creating Resilient Societies took place in Brussels, October 2015. The two-day conference has offered a unique forum to both discuss the different perceptions of trust and how research can contribute to fostering trust in societies. The conference, which was a key event on SC 6 has not only highlighted research within the social sciences and humanities but has also connected researchers with policy-makers and stakeholders willing to co-create resilient European societies.

Among the SC6 participants in the closed calls of SC6, the programme has attracted a more balanced share of types of actors than in FP7 (see Table 219). In FP7, the Higher or Secondary Education Establishments represented 47% of the participants and Research Organisations 29%, while in Horizon 2020, these are respectively 35% (HES) and 18% (REC). However, the Private for-profit entities have increased their presence from 9% in FP7 to 24% in Horizon 2020, Public bodies from 8% to 12% and Other organisations, from 7% to 11%. In other organisations, there are civil society organisations, whose participation is relevant due to the nature of the issues addressed by the programme, which are closely related to the social problems faced by citizens. Overall, these data show a clear diversification of the types of actors participating in the programme, and stronger synergies between organisations in the projects.

Table 219 - Participation by type of actor in FP7 (SSH) and Horizon 2020 (SC6)

Type of participant (Type of activity) – All participations Total

PRC REC PUB HES OTH

Horizon 24% 18% 12% 35% 11% 100% 2020 – SC6

FP7 – 9% 29% 8% 47% 7% 100%

SSH

Source: CORDA data, 1 January 2017, Applicants and Applications by EU-28 Member States (General).

O.5.2.2. Geographical dimension

In terms of geographical distribution the SC6 has mobilised participants from all EU 28 Member States, and it is worth highlighting that SC6 has been able to include more

diversity across the European geography than other challenges. 136 Table 220 gives an

overview of the distribution of funding and participations in signed grants for Member States and overall numbers for Associated and Third Countries in FP7 and Horizon 2020. Overall 92% of the EU funding goes to EU Member States (83% to EU-15 and 9% to the EU-13) - which is higher than in FP7-SSH (90%). Associated countries and candidate countries receiving 5% and third countries 3% of the total EU contribution. The funding is concentrated (68% of the total EU contribution) in seven countries, all belonging to EU-15 (UK alone receives almost 18% of the total EU contribution). However compared with other societal challenges the representation of Eastern European countries is slightly higher.

The countries and institutions that obtained substantial funds due to the following factors:

• 1. 

  Provide incentives to teams that apply and obtain EU projects that promotes the emergence of excellence and increase attractivity

• 2. 

  Put in place administrative and support structures to help the management and application process

• 3. 

  Facilitate networking at the EU and international level 137

While existing networks are being reinforced and extended, new networks are also being created.

In terms of success rate it decreased in Horizon 2020 compared with the FP7-SSH programme. However the Horizon 2020 success rate per country groups has a similar pattern than in the past programme. The success rate ranges from 3.1% for Bulgaria to

136 Horizon 2020 Annual monitoring report 2015 (Horizon 2020 average for EU-13 receives 4.7% of the total EU

funding).

137 This is based on expert groups report on SC6 stakeholders and FP-7 ex post evaluation.

27.3% for Luxembourg. As regards Romania has the lowest success rate of funding with

1.9% and Luxembourg has the highest with 16.3%.

As for the geographical distribution of the project coordinators the vast majority are coming from the EU-15 (171 project coordinators), 15 project coordinators are EU-13, and 8 from Associated countries. In comparison with FP7-SSH cooperation programme, there is a similar pattern in terms of geographic representation of project coordinators.

Table 220 - Participants by group of countries in FP7 SSH – Cooperation vs.

Horizon 2020-SC6

Country Nr of Nr of Nr of Nr of EC EC Success Success Groups Participa Participa Coordinato Coordinato Contribution Contribution rate of rate of nts in nts in rs in rs in FP7 to participants to participants funding funding

Horizon FP7 Horizon Signed in Horizon in FP7 Grants Horizon FP7

2020 Signed 2020 Grants 2020 Grants (EUR 2020

Signed Grants Signed (EUR million)

Grants Grants million)

AC 79 112 8 13 17.1 35 6.0% 7.2%

Countries

EU-15 138 676 760 171 162 284.2 476 7.0% 10.6

% EU-13 140 203 15 5 31 46 4.4% 5.8%

Third 101 177 0 0 9.8 27 9.4% 8.2%

Party

Countries

Total 996 1252 194 180 342.1 583 6.7% 9.59%

Source: CORDA data, 1 Janaury 2017.

Table 221 presents the seven countries receiving the highest EU contribution in FP7 and

Horizon 2020. The top 7 countries are the same in FP7-SSH and in SC6 but in a different order.

Table 221 - Top 7 countries in FP7 SSH – Cooperation vs. SC6 Horizon 2020

Participant Nr of Nr of Nr of Nr of EU EU

Country Participants Participants Coordinators Coordinators Contribution Contribution

Code in Horizon in FP7 in Horizon in FP7 to to

2020 Signed Signed 2020 Signed Signed participants participants

Grants Grants Grants Grants in Horizon in FP7

2020 Grants Grants

(EUR (EUR million) million)

BE 55 65 10 6 18.5 40

DE 87 121 21 27 40.6 (II) 67 (II)

ES 81 65 23 10 21.7 29

FR 42 85 9 14 17.7 37

IT 94 84 20 23 34.5 (III) 49

NL 33 54 15 15 23.5 54 (III)

UK 106 116 32 35 56.5 (I) 96 (I)

Source: CORDA data, 1 Janaury 2017.

138 In particular 7 EU-15 countries receive 68% of the total EU contribution

O.5.2.3. Cross-cutting issues

The Horizon 2020 regulation clearly flags SC6 as the Societal Challenge that will support social sciences and humanities research by focusing on inclusive, innovative and reflective societies. As regards the integration of social sciences and humanities (SSH) under SC6 in Horizon 2020, in the period 2014-2017 90 topics and other actions have been classified as being relevant for SSH researchers, and the SSH partner received EUR 132 million. According to the SSH monitoring report SC6 is performing very well both in terms of quantity (number of SSH partners and budget) and quality of the SSH integration. There are still room for improvements when it comes to the integration of the

Humanities, 139 in particular:

 Appropriate wording needs to be introduced in order to make sure that the SSH

dimensions constitute an integral part of the topic description and are recognised by proponents.

 Ensure a fair and consistent evaluation of SSH-flagged topics, the participation of

experts with SSH expertise in the evaluation panels is key element.

SC6 is also attracting several participants from Third Countries. The share of participations of and the share of EU contribution to entities from third-countries in SC6 (data up to October 2016) is the highest among all Horizon 2020 Work Programme parts.

O.5.3. Cost-benefit analysis

Societal Challenge 6 is managed by DG RTD (8 Units involved) and by DG CNECT (4 units involved, more than 25% of the overall budget (EUR 318 million). The Work Programme preparation and drafting falls under the responsibility of the Commission, whereas the vast majority of the projects are implemented (evaluation, project management and follow-up) by the Research Executive Agency (REA) and Executive Agency for Small and Medium-sized Enterprises (EASME) for projects financed through the SME instrument. In particular, the implementation of the Research and Innovation Actions has been delegated to the REA, while the ERA-NET Co-fund and the Coordination and Support Actions (CSA) were kept in the parent DGs (DG RTD and DG CNECT). With the process of delegation of project management to the Agencies, the Commission concentrates on its core competences of proposing policies. The CSAs, which are still managed within the Commission, enable the tracking of the policy developments. The Commission services are cooperating with REA in the evaluation process, management of the projects and for the exploitation of project results. Internally in the Commission the Common Support Centre is in charge of the IT resources to collect and handle data. These data are relevant in order to develop and implement an evidence-based policy and for its relationship with the stakeholders. The involvement of different Units and Executive Agencies represents an opportunity to bring different and valuable inputs to the Work Programmes but also a challenge in terms of coordination and overall coherence of the programme. In particular DG CNECT specialises on topics related principally to the Digital Agenda, whereas DG RTD focuses on the larger scope of current socioeconomic and technological issues.

The process of delegation of the majority of projects to the executive agencies comes with challenges and opportunities. In terms of implementation is possible to observe a

139 SSH report 2014 and 2015

positive trend. The high percentage of grants signed before the target (8 months) indicates that REA is managing the evaluations and grants in an efficient manner. These procedures are quite cumbersome and it REA manages in an efficient way to distribute the workload over the year. As regards the information flow between REA and the Commission while the EC services need information from projects to elaborate its policies, the REA needs to be aware about policy developments in order to provide policy inputs to the projects. This two ways relationship should be reinforced in order to have a more intense exchange on the projects and their outputs between Policy Officers of the Commission and the REA. In this context there are good examples of cooperation like joint kick-off meetings under different calls and joint call planning.

It is also necessary to increase the flexibility of projects work plans of the and deliverables in order to assure alignment with evolving policy needs.

Findings from survey with project coordinators and participants carried out by DG CNECT, suggest that having a clear and transparent call and topic text is vital as it helps to ensure that both applicants and evaluators have a common understanding, and

that proposals are assessed according to clear and transparent criteria. 140

Simplification and efficiency are inextricably linked concepts. However it is too early to evaluate the actual outputs, outcomes and impacts of SC6. All projects are on-going and in their initial phases and few data is available on concrete outputs and results.

SC6 is the most SSH intensive Societal Challenge (SC) and is the smallest funded SC of Horizon 2020. However taking into consideration that Social Sciences and Humanities represent one of the largest research communities and student bodies in higher education

in Europe, 141 the programme produces substantial added value at a reasonable cost.

Under FP7 the Socio-Economic Sciences and Humanities sub-programme of Cooperation accounted for the smallest budget share and smallest total number of projects across all themes of FP7- Cooperation with 253 funded projects and EUR 580 million of funding (1.3% of the total FP7 budget). The situation is the same under Horizon 2020 where SC6 is the smallest funded theme under the overall Horizon 2020 Societal Challenges. From this perspective the cost benefit analysis can be considered to be good given the resources allocated to SC6 even after one takes into account both direct and indirect costs of project management. According to the stakeholders a major issue of the SC6 programme is the high level of oversubscription and in order to address this the Commission is experimenting with a two-stage evaluation process in 2017 and increasing the number of projects per topic.

One of the recommendations of the ex ‐post evaluation of FP7 recommendations clearly addresses the issue of efficiency: Integrate the key components of the Framework Programmes more effectively (fragmentation and the emergence of ‘silos’ have tended to threaten efficiency and coherence of the Framework Programmes in terms of compartmentalization and duplication of themes). In this context Horizon 2020 and SC6 have addressed these concerns by exploiting synergies and avoiding duplications between the different specific programmes and sub-programmes.

140 DG CNECT – Support study for the Interim Evaluation of Horizon 2020 – DG Connect Activities

141 Eurostat - Tertiary education statistics

O.5.4. Lessons learnt/Areas for improvement

The high number of applications shows that the SC6 is very attractive. SC6 has been able to become of high interest for a wide range of organisations from all over Europe and non-European countries. In fact, due to the high number of submissions, SC6 has the lowest rate of success compared to the other societal challenges, which indicates the willingness of European researchers in the field of SSH to be involved in transnational consortia dedicated to the common social problems of our societies.

At the level of programme implementation evaluation procedures play an important role in determining success rates. When using two-stage evaluation procedures it is possible to coordinate the number of proposals in the second stage, while reducing the time and resources put into proposals with limited chance of receiving a fund. Two stage calls were launched in the case of several Topics open in 2017 under call Understanding Europe – Promoting the European Public and Cultural Space to see whether it would help in increasing the success rate.

The majority of organisations undertaking SSH research were universities – more than half of participating organisations in SSH were universities (43% in whole FP7, 35% in FP7-Cooperation, 63% in SSH). However, the involvement of the business sector in SSH was extremely low (25% in FP7, 8% in FP7-Cooperation, 4% in SSH). From FP7 to Horizon 2020, the programme is attracting more diverse types of organisations, with a decrease of the representation of Higher and Secondary Education Establishments and an increase of private for-profit centres, public bodies and other organisations (including civil society organisations). The stronger orientation to products and services production of the Calls for proposals may have influenced this increasing diversity of actors. In addition, SC6 has the potential to attract more participants from the less represented countries such as some European Eastern countries and third countries.

O.6. C OHERENCE

O.6.1. Internal coherence

O.6.1.1. Internal coherence of the actions implemented for SC6

In the period from 2014 to 2017 Societal Challenge 6 has supported several objectives in the three intertwined areas of inclusive, innovative and reflective societies. The 2014-15 and 2016-17 Work Programmes focused on overcoming the economic crisis and mitigating its effects (Call EURO), reducing inequalities and promoting social fairness (Call REV-INEQUAL) and integrating the young generation in a more innovative, inclusive and sustainable Europe (Call YOUNG). They also addressed new forms of innovation and untapped sources of growth (Calls INSO and CO-CREATION) and Europe’s cultural heritage and identities (Calls REFLECTIVE and CULT-COOP). The strengthening of EU's capacities for developing and improving its external action and international cooperation were also covered (Calls INT and ENG-GLOBALLY).

SC6 has supported up to now 15 Innovation actions, 70 RIA, and 33 CSA, plus 57 SME instrument and 3 ERA-NETs (see table 5). The IA, RIA and CSA have been coherently distributed among the different Calls in order to achieve the SC6 objectives. In line with the Specific Programme these actions were supplemented by a set of specific smaller activities (other actions) supporting in particular the implementation of the Innovation Union, the European Research Area and the strategy for international cooperation in R&I. SC6 also finances COST (European cooperation in Science and Technology network actions) through a grant to an identified beneficiary.

The analysis 142 shows both complementarities, synergies and a certain overlapping

among projects funded under every Call and/or topic of research. This can be due to the description of the scope in each topic. For example, the topic “EURO 2-2014: The European growth agenda” indicated four different dimensions that projects could cover separately (not all of them). As a result, projects funded under this topic are focused on different issues such as Migration, Industrial Innovation or Financial reforms. This issue of different dimensions was also raised during several info-days.

Conversely, the description of the topic “YOUNG 3: Lifelong learning for young adults” does not highlight different dimensions and specifies that projects should develop an Intelligent Decision Support System for supporting access to information and policy making. As a result, the three funded projects under this topic are working in the same direction to provide each one a different intelligent decision device. This aspect however increases the risks of duplications.

Other examples of projects with clearly similar objectives and even deliverables are found among the projects financed. It is relevant to highlight that on-going projects are making efforts to work together and to build synergies, for instance in joint conferences. Joint kick-off meetings for each call have been organised in order to create synergies and avoid duplications from the outset of the implementation of projects. Mid-term review meetings of 2014 Topics provided clear recommendations on how to create synergies and avoid duplications between the projects selected under the same call.

Internal coherence will be also supported by Coordination and Support Actions. For example the ACCOMPLISSH project (Accelerate co-creation by setting up a multi-actor platform for impact from Social Sciences and Humanities) will create a platform for dialogue where academia, industry, governments and societal partners equally contribute in identifying barriers and enablers of co-creation. The results from both practice and the theory of co-creation form the basis of the valorisation concept and will be tested in the project in a quadruple helix setting. This concept will be tested and developed in such a way that it is transferable, scalable and customized for academia, industry, governments and societal partners in the whole of Europe. The project will identify all barriers and enablers of co-creation in order to develop an innovative valorisation concept, which will foster knowledge exchange within the quadruple helix and strengthens the position of SSH research.

Topics related to ICT-enabled public sector innovation in 2014-2015 have focused on piloting the concept of open government. Innovation actions were supported, re-using open data and services for the creation of new, personalised services or for increased transparency (INSO-1 and CULT-COOP-11 as follow-up to understanding the transformation of European public administrations), while another set of projects focused on opening data and processes in public administrations, in order to facilitate the engagement of the youth in policy-making using digital means (YOUNG-5b). In addition, the SME instrument allowed supporting small companies aiming to set up a business on mobile e-Government applications using open data or services (INSO-9). While open government remained relevant for the period of 2016-2017, ICT-enabled public sector modernisation was highlighted as an area where co-creation could bring

142 Project IMPACT EV and in house desk analysis of a sample of projects

significant benefits to public administrations and stakeholders alike. Co-creation and collaboration between administrations can improve their efficiency and effectiveness and reduce the administrative burden on businesses and citizens. An innovation project is aiming to pilot this so-called 'once-only' principle for businesses and the related Coordination and Support Action will explore the feasibility of applying the principle for citizens (CO-CREATION-05).

Coordination and support actions and a service facility (INT-1, INT-2, Service Facility in support of international cooperation) provide services in support of policy development, priority setting, follow-up and implementation of the strategy for international cooperation in R&I. They include activities such as awareness raising and training activities to enhance international cooperation activities in Horizon 2020, support to National Contact Points and other multipliers, organisation of workshops in support of policy dialogues, brokerage, networking and twinning events, and analysis and monitoring activities.

Finally, SC6 participates in the Pilot on Open Research Data which aims to improve and maximise access to and re-use of research data generated by projects.

O.6.1.2. Internal coherence with other Horizon 2020 intervention areas

The Work Programmes for SC6 establish complementarities with other issues, especially with Societal Challenge 7 - Secure societies "Protecting freedom and security of Europe and its citizens", Societal Challenge 5 "Climate action, environment, resource efficiency and raw materials", and the Industrial Leadership part (in particular ICT). Other complementarities exist (for instance research on cultural heritage of European coastal and maritime regions, and with the marine research in SC2) but with a smaller scope.

SC6 Work Programmes (in particular WP 2016-2017) addresses radicalisation and migration that appear directly and in a crosscutting way in several Topics throughout the Work Programme and complementarily with Societal Challenge 7. Specific links also with the Societal Challenge 5 have been established on the Cultural Heritage topics. While there are few explicit references to LEIT-ICT topic (only in the Work Programme 2016-2017) ICT-related issues appear in several Calls and funded projects. Cross-cutting actions targeting the field of interaction between humans and technology were implemented. In particular the Topic Boosting inclusiveness of "ICT-enabled research and innovation" requires synergies with the LEIT-ICT topics "ICT35-2016: Enabling responsible ICT-related innovation". As take-up of LEIT-ICT result, projects were funded to demonstrate how emerging technologies can be applied in the public sector (EURO-6) in order to highlight the role of public administrations in bringing innovation to the market and thereby contributing to growth.

The Call – The Young Generation in an Innovative, Inclusive and Sustainable Europe (WP 2014-2015) clearly links the research activities with actions aimed at making science education and careers attractive for young people, supported by the Science with and For Society programme.

The project NET4SOCIETY foresees targeted activities to support applicants in SC6. The project will facilitate interdisciplinary and international consortium building through the organisation of brokerage events, through a dedicated partner search service. To support the successful implementation of integration Socio-economic Sciences and Humanities (SSH) in all parts of Horizon 2020, NET4SOCIETY will carry out surveys on the integration of SSH in Horizon 2020. The project will publish success stories and factsheets on integration, as well as a document listing funding opportunities for SSH in all of Horizon 2020.

The projects ERA-NET co-fund Smart Urban Futures and Towards We-Government: Collective and participative approaches for addressing local policy challenges will provide inputs for the “Smart and Sustainable Cities” cross-cutting focus area that has the aim of bringing together cities, industry and citizens. The SC6 also supports the coordination of international networks for excellent researchers and innovators such as COST, and therefore also contribute to the ERA.

Finally SC6 contributes to the majority of the cross-cutting issues listed in Art.14 of the Horizon 2020 Regulation, thus increasing the synergies between the specific objectives of the three priorities of Horizon 2020, in particular by translating knowledge into economic and societal value.

The graphic below, based on a EC internal questionnaire on internal coherence of SC6 with other Horizon 2020 objectives gives an idea of the internal coherence between SC6 and the other societal challenges. The graphic show that there is internal coherence, but at a relatively weak level because the polygon does not spread outwards very far. The thematic links with other Societal Challenges exist but are not very strong as the diagram below shows.

Figure 248 - Internal coherence of SC6 with other Horizon 2020 specific objectives

Source:Internal Questionniare

O.6.1.3. Ensuring that every euro spent counts twice

The results and policy proposals from Horizon 2020 projects (such as under EURO, YOUNG or REV-INEQUAL) systematically address multiple objectives of the European Commission current priorities. These projects are expected to enable the achievement of multiple goals. Calls like CO-CREATION and INSO enable innovation while at the same time responding to societal challenges like inclusion and growth.

O.6.2. External coherence

O.6.2.1. Coherence with other EU funding programmes

From a general perspective several topics and calls include recommendations for taking into account other EU and national initiatives in their implementation.

In terms of policy priorities, the SC6 priorities are fully linked to the main EU policies dealing with Migration; Jobs, growth and investment; the Digital single market; Justice and fundamental rights based on mutual trust; Making the EU a stronger global actor; and Fostering a Union of democratic change. In particular DG RTD is involved in different inter-services group meetings where the results of the projects contribute to the policy debates (e.g. migration polices and projects are extensively discussed with the relevant services in the Commission).

The EU Agenda on Migration, the European Agenda for the collaborative economy, the EU Global Strategy and the Social Investment Package are also particularly relevant for this SC6 work programme.

The migration crisis has recently challenged Europe's capacity to act in coherent and unified way. The large influx of refugees and other migrants largely caused by conflicts, geopolitical shocks and poverty poses short, medium, and long term challenges. It goes from immediate hosting of refugees to the lasting integration in the EU of all legally staying migrants. The SC6 2016-2017 has earmarked EUR 16 million for research on migration and mobility taking into account the political priorities of the Services in the Commission directly responsible for theses dossiers (European Agenda on migration).

Even though there are no specific provisions to foster complementarity between European structural and investment funds ( ESIF) and the SC6 in Horizon 2020 (these provisions are more likely to be provided at Member State level), thematically there is a strong link between the research on cohesion/inclusion/inequalities (territorial and social cohesion) and the EU structural funds. In the same vein, synergies and coordination between Horizon 2020-SC6 and European Fund for Strategic Investments (EFSI) are not specifically mentioned in the programme documents but research under the Call – Overcoming the Crisis: New ideas, Strategies and Governance Structures for Europe will critically asses the economic added value of EFSI (EURO-2-2014 - The European growth agenda).

Within the Co-Creation for growth and inclusion call the topics dealing with education invite the projects to coordinate their activities with the Erasmus+ and Knowledge alliances programmes.

Coherence between the EU external and internal funding programmes features as a key research question in coherence and consistency of trade policy, enlargement and neighbourhood policy, development policy, with other EU external polices such as economic, development, environmental and social polices, labour and human rights. For example the topic ENG-GLOBALLY-05-2017: The strategic potential of EU external trade policy will propose recommendation on how the coherence between the EU's and Member States' trade policy should be ensured, as well as coherence between trade and other (external) policies.

On the same line, the project The Impact of Cohesion Policy on EU Identification (COHESIFY - REFLECTIVE-3-2015), deals with the impact of cohesion policy especially towards the citizens' perception of citizens. The results of the project will be useful for other EU policies and programmes such as development programmes, communication and dissemination activities to promote the EU and to regain trust in the European project amongst its citizens. Thematically it is coherent with the European Structural and Investment Funds and sustainable development goals.

The international cooperation dimension of SC6 also supports exploiting synergies with actions and activities of other EU programmes and policies. For instance, synergies are exploited with the 'EU Macro-regional Strategy for the Danube Region' and the Instrument for Pre-Accession Assistance, aiming to increase the effectiveness of investments into R&I, enhance the regional research and education capacity and develop smart specialisation strategies for R&I. Complementarities have also been developed with the programme for S&T Innovation and capacity building in African, Caribbean and Pacific countries of the European Development Fund, the European Neighbourhood Instrument, the R&I pillar of the Cross Border Cooperation Programme for the southern

Mediterranean countries, and the African Union Research Grants. 143

The Figure 249 is based on an internal EC questionnaire on external coherence of SC6 with other European policies and programmes. It shows that that the coherence between SC6 and some major policy areas is limited to certain areas (such as culture, development, education, industry and competitiveness) and very weak in areas such as internal market policy, energy or consumer protection policies. Since SC6 is dealing with economic social and human dimension of current challenges it is complementary rather than overlapping.

Figure 249 - External coherence of SC6 with other European policies and programmes

Source:Internal questionnaire.

143 2nd Report on the Implementation of the Strategy for Int. Cooperation and roadmaps (2016)

O.6.2.2. Coherence with other public support initiatives at regional, national and international level

Article 13 of the Horizon 2020 regulation requires coherence between Horizon 2020 and national programmes. Article 13 also refers to Joint Programming Initiatives, with instruments defined in article 26 (“public-public partnerships”): ERA-NETs and Article 185 TFEU. ERA-NET Co-fund actions under Horizon 2020 are designed to support public-public partnerships, including joint programming initiatives between Member States, in their preparation, establishment of networking structures, design, implementation and coordination of joint activities as well as EU topping-up of a transnational call for proposals. The main and compulsory activity of ERA-NET Co-fund actions under Horizon 2020 is the implementation of the co-funded joint call for proposals that leads to the funding of trans-national research and/or innovation projects. Under the SC6 Work Programmes 2014-2017 there are four ERA-NETS: ERA-NET on Smart Urban Futures, ERA-NET on Uses on the past, ERA-NET Dynamics of inequalities across the life-course and ERA-NET on Culture, integration and European public space (the last one will be selected in 2017).

The official channels of information between Member States and the Commission services are the Programme Committee and the National Contact Points (NCPs). Ensuring coherence of SC6 with Member States priorities requires a deep knowledge of environmental and R&I situations, issues and policies at national level. For this purpose a specific action has been supported to strengthen the administrative and operation capacity of transnational networks of National Contact Points while improving the operational of NCPs and the flow of information between them and the Horizon 2020 implementing bodies (NET4SOCIETY Project). The NCP system enables the construction of networks more easily, provides information and clarification on the calls and topics by administrative staff experienced in EU projects, and facilitates the flow of information from the Commission to potential participants with greater fluidity.

In this context the following actions contribute to increasing consistency between EU funding and other initiatives at regional, national and international level:

 The pilot Synchronised Call initiatives (INSO-8-2014) aims to promote cooperation

between national/regional funding bodies and contribute to raising the quality of research in Europe, which in turn could enable excellent researchers to enter Horizon 2020 if they are not able to access via ERC actions. Strategic programming activities provide support for the setting of R&I priorities in Horizon 2020 and bring forward common EU and Member State orientations in the field. To this end, national/regional funding will inform consortia to launch synchronised calls at European level, addressing a pre-determined scientific field

with one identical call deadline and using joint international peer review.

 The project Bridging the gap between public opinion and European leadership:

Engaging a dialogue on the future path of Europe (EUENGAGE project - EURO- 4-2014) deals with the tension between supranational EU governance and popular mobilisation at the national level questioning EU legitimacy. The results of the project will certainly be of added value for both regional, national and European policy makers. Findings from the project may well be useful for authorities in many EU Member States in order to assess the current situation and consider a change in national policies to better align with the direction the EU is taking.

 The project Dandelion (Promoting EU-funded projects of inclusive, innovative

and reflective societies) Dandelion will promote the work done by inclusive, innovative and reflective societies’ projects in SC6 on a local, regional and European level by developing and implementing a series of innovative and exciting communications activities aiming to inform, educate and entertain a wide cross section of the European population, policy makers, academics and media. By giving tools and guidelines to the dissemination managers towards the general public, policy makers, academia and media Dandelion will guarantee an

improved access to research projects’ data in the future.

 Centres of European research and innovation (ENG-GLOBALLY-09) – a first

wave established in Brazil, China and the USA - will connect and support European researchers and entrepreneurs globally, in order to strengthen the position of Europe as a world leader in science, technology and innovation. They will build on existing European science, technology and innovation structures located in international partner countries and regions and engage in activities such as networking services, advice and support, advocacy, awareness raising, training, and infrastructure provision. The centres should lead to reinforced international cooperation, higher visibility and prestige for European R&I, and stronger presence of European organisations in the science and innovation environment of

the partner country/region.

 A platform for EU-China cooperation on sustainable urbanisation (ENG-

GLOBALLY-08) will bring together a wide-ranging partnership of stakeholders in Europe and China for developing and piloting innovative solutions in sustainable urbanisation. The platform will develop joint strategies, be the 'nursery' of joint projects and a broker of science-industry partnerships between

Europe and China.

O.6.3. Lessons learnt/Areas for improvement

The facts that topics contain different dimension for the same specific challenge could create confusion for the applicants and for the evaluators.

SC6 has strong potential links with all parts of Horizon 2020 and other EU funding programmes. These links should be better framed. There are some explicit links to other parts of Horizon 2020 and other programmes but there is still space for improvement in the synergies and acknowledgement of other initiatives that can be complementary. In particular, more efforts are necessary to ensure synergies with ESI Funds, considered essential because tackling societal challenges require a strong financial leverage.

The RIAs and CSAs in SC6 are balanced and tend to be complementary. Among the research projects (RIAs) there are consortiums that are tackling the same problem with complementary methods and approaches, or that are already establishing synergies to collaborate in the search for new solutions. In other cases, the description of the scope and expected impacts in the Calls for proposals are very detailed and different projects respond with similar proposals creating a risk of duplicating efforts and tasks.

Projects’ findings should be translated into policy advice that is tailored for end-user policymakers.

There is scope for increasing synergies with what Member States are doing at national and regional level in order to better ensure coherence with the national funding.

There is a possibility of confusion within the research community and stakeholders concerning the research funds in culture and education. These funds are spread out through SC5, SC6, LEIT and a Joint Programming Initiatives on Cultural Heritage. Such a fragmented research policy approach is ill-fitted to theoretical and policy developments in the field but also to institutional policy realities at international/UN (UNESCO), EU (DG EAC) and national (cultural ministries) levels.

O.7. EU A DDED V ALUE

O.7.1. Horizon 2020 projects demonstrating EU Added Value

Project title: EUENGAGE: Bridging the gap between public opinion and European leadership: Engaging a dialogue on the future path of Europe.

Project duration: April 2015 – March 2018

Project budget (EU contribution): EUR2 500 000

Type of action: Research and innovation actions

The goal of the EUENGAGE Project is twofold: first, to inquire into the current tensions between supranational EU governance and popular mobilisation at the national level, critically questioning EU-driven policies and EU legitimacy; and second, to propose remedial actions based on sound empirical research on the relationship between public opinion, national and supranational political elites. The medium to long-term evolutionary trend of the EU system of supranational governance has already in the past given rise to a manifestation of problems.

Project title: ENLIGHTEN: European Legitimacy in Governing through Hard

Times: the role of European Networks.

Project duration: April 2015 – March 2018

Project budget (EU contribution): EUR2 500 000

Type of action: Research and innovation actions

ENLIGHTEN responds to the first part of the EURO-4 call on “The future of European integration - 'More Europe – less Europe?'” by bringing together an interdisciplinary ‘next generation’ research team that integrates insights from Comparative Political Economy, European Studies, International Political Economy, and Sociology. ENLIGHTEN focuses on how European modes of governance respond to ‘fast-burning’ and ‘slow-burning’ crises. These types of crises differ in how they affect the legitimacy of European input, output, and throughput processes in established and emergent modes of governance. In fast-burning crises interests are quickly formed and ideational and resource battles ensue over how to coordinate policy ideas, what institutions should be engaged, and communicating these changes to the public.

O.7.2. Other issues related to EU Added Value

Research policy was initially focused mainly on competitiveness and industrial policy. The Targeted Socio-Economic Research (TSER) programme, under the Fourth Framework was the first major landmark in the evolution of EU support for social sciences and humanities research. With the rise of new technologies and high-tech industries, and the development of new services, there was a need to analyse the economic and social impacts of the changes that were taking place.

Projects funded under SC6 tackle social problems that go beyond the Member State level and for which solutions cannot be sought at a purely national scale. They thus require new knowledge based on international cooperation and perspectives:

 Effectiveness: the themes of the projects have a panEuropean nature and deal

with the growing interdependency of our societies: recovery from the economic crisis, European identity and culture, generational challenges. A large number of projects create or improve the availability of European-level information, for instance with new databases or datasets that gather and allow comparison from

national policies and data.

 Efficiency: research projects include in many cases a comparative study among

different European countries. Having transnational consortiums is an added value to achieve these comparative studies. The consortiums also enable cooperation

amongst different actors that are collecting data on the same issues.

 Synergy: EU intervention enables to stimulate action in fields such as Social

innovation, which need high standards of creativity and interaction. Also, it helps to build synergies and new collaborations between high R&I performing States

and lower performing States.

The European nature of the consortia allow allows the SC6 to achieve a better understanding of some of the most pressing problems of our societies, which are defined by their interdependency. This deduction is based on a large sample of projects analysed by the Impact EV team and in house.

As shown in the PPMI survey conducted on a sample of 32 SC6 projects 144 a key aspect

of European added value concerns the concept of project additionality -i.e. the capacity of the project beneficiaries to carry out the same or very similar projects without EU funding. On the other hand, full project additionality and European added value is achieved in cases where Horizon 2020 projects would not have gone ahead at all without EU funding. In the PPMI one clearly sees the importance of EU funding for SC 6, a large majority of projects critically relied on EU funding to come into existence. The PPMI report shows that among all the SCs, SC6 projects are the ones that rely most on EU funding.

Table 222 presents the findings of the Horizon 2020 survey on the additionality of Horizon 2020 projects. For SC6, almost 2 out of 3 projects (64%) would not have gone ahead without EU funding. About a third of the projects (30%) would have gone ahead with significant modifications. As a result, it is estimated that total project additionality amounted to 93% for SC6. On average, only 6,7% of the projects would have gone ahead without EU funding with none or minor modifications. Hence for SC6, the additionality of projects in terms of EU value added appears substantial.

144 PPMI report

Table 222 - Continuity of Horizon 2020 SC6 projects had they not received EU funding

Horizon Answer categories

2020 specific No of The project would have The project would have The project would objective valid gone ahead with none or gone ahead with not have gone

answers minor modifications significant modifications ahead

SC1 105 12,3% 39,6% 48,0%

SC2 44 25,6% 32,9% 41,5%

SC3 134 14,9% 30,6% 54,5%

SC4 95 7,3% 43,7% 49,0%

SC5 72 17,7% 39,5% 42,8%

SC6 32 6,7% 29,2% 64,1%

SC7 31 11,2% 33,4% 55,3%

Note: Responses to question ‘a1- What do you think would have happened to your project had it not been funded by Horizon 2020’. Proportions/percentages show answers of weighted responses.

Source: PPMI report.

O.7.3. Lessons learnt/Areas for improvement

The SC6 addresses Pan-European challenges like migration, economic crisis, social welfare, inequalities, etc. EU financing on social, economic, political, and human dimensions of current problems remains indispensable as the preceding analysis shows.

The SC6 projects can rely on critical mass in different domains of specific research where EU-level intervention is able to make a difference by bringing together different knowledge and reducing costs and bringing together stakeholders who are not traditionally cooperating.

The examples of projects financed by Horizon 2020-SC6 are characterised by very large expected impacts, both economic and in terms of resource efficiency (innovation actions), or by bringing together the main European and/or international actors in specific areas, towards a common and trans-national goal – either policy-related or scientific.

Additional opportunities for researchers become available at a scale not reachable at national and local level: a large majority of SC6 would not have been implemented without the level of support provided by EU funding.

Broad and well-funded research activities in the SSH are essential for a better understanding of the current developments in our Union and are instrumental in solving some of the problems associated with these developments. Similarly, there are many areas of SSH research that could significantly contribute to furthering the “European idea”.

Communication and dissemination of the project results has the objective of raising awareness among the stakeholders and citizens on the knowledge or innovation advancements thanks to the European Union’s support.

O.8. S UCCESS STORIES FROM PREVIOUS F RAMEWORK P ROGRAMMES

Among the many success stories from FP7 finished projects, the following have been selected as having been particularly successful in tackling the challenges of previous Work Programmes in inducing socio-economic impacts and in providing EU Added Value:

CARE Project - Curriculum and Quality Analysis and Impact Review of European Early Childhood Education and Care

In line with EU strategies for 2020 and the need for a systemic and integrated approach to Early Childhood Education and Care (ECEC), the project identifies eight key issues for which effective policy measures and instruments should be developed. The issues concern assessing the impact of ECEC, optimising quality and curricula for ECEC to increase effectiveness, raising the professional competencies of staff, monitoring and assuring quality of ECEC, increasing the inclusiveness of ECEC, and optimal funding of ECEC. The project addresses these issues by combining state-of-the-art knowledge of factors determining personal, social and economic benefits of ECEC with knowledge of the mechanisms determining access to ECEC. Recent and ongoing large-scale studies from several European countries are used to identify the factors that determine quality and child outcomes. In addition, the project aims to integrate the cultural beliefs and values of stakeholders in developing a culture-sensitive framework. A comparative overview of early childhood curricula across Europe and an evaluation of their effectiveness, showing a high degree of agreement across countries, an emphasis on academic skills and still limited articulation of new (21st century) skills like selfregulation, creativity and collaboration.

MAFE project - Migration between Africa and Europe

Despite the attention it raises in the media, the scope, nature and likely development of Sub- Saharan African migration to Europe remains poorly understood, and, as a result, European polices may be ineffective. A major cause of this lack of understanding is the absence of comprehensive data on the causes of migration and circulation between Africa and Europe. MAFE collected unique data on the characteristics and behaviour of migrants from Sub-Saharan countries to Europe. Underpinning this project was the recognition that migration is not simply a one-way flow from Africa to Europe. Rather, return migration, circulation, and transnational actions are significant and need to be recognised in policy design. The project sought to address four key areas: i) patterns of migration: trends, migrants characteristics, migratory routes; ii) determinants of migration: poverty, education, gender, policies; iii) migration and economic integration: remittances, investments, integration and reintegration of migrants; iv) migrations and families: family construction, structure and formation, families over time and space. Comparable data on African migration was collected in both sending and receiving countries. It included background information on individuals as well as data linking their histories to other details in both the origin and destination countries. The workable data sets were released to general users in January 2015. The project identified very clear changes in migration trends and strong differences across countries, in particular with regard to female migration. The MAFE team also studied the family arrangements of migrants by comparing three groups of families: current migrants, non-migrants and return-migrants. In each case, the results indicated significant differences between the three family groups. The MAFE project informed the continuing debate on migration and development, and created sustained and mutual interest in EU–Africa cooperation on migration.

RELIGARE project - Religious Diversity and Secular Models in Europe—Innovative Approaches to Law and Policy

The project examines the legal rules protecting or constraining the experiences of religious or other faith-based communities in Europe at local, national and European levels. The project gives precise scientific results that thwart stereotypes about a supposedly homogeneous Muslim community being at odds with European values. It stresses that explicit or implicit discrimination against Muslim communities should be dealt with at different legislative and administrative levels, from schools and local municipalities all the way to EU legal and institutional frameworks. The findings of RELIGARE highlight that although the number of Islamic followers is growing and is already reaching 3% of the European population, they are mostly excluded from various forms of support reserved by the Member States for traditional «majority» religions even in countries where the Church- State separation is enshrined in the Constitution. In other words, despite much political rhetoric about «unity», «equality» and «citizenship», migrants of Islamic origin or confession remain “not so equal” in many European countries, thus adding to the social and political frustration of these citizens. In its final summary report, RELIGARE advocates a balanced approach to multiculturalism in Europe, arguing that “the EU institutions should include religion not only in its individual but also in its collective dimension thanks to the promotion of two yardsticks — ‘inclusive neutrality’ and ‘justice as even-handedness".

IMPROVE project - Poverty Reduction in Europe: Social Policy and Innovation

The project started from two major observations about the political and socio-economic context of the European welfare states. First of all, before the crisis, despite higher employment rates and economic growth, the EU social indicators show that nowhere any substantial progress has been made in combating relative financial poverty for the population an active age. Second, new (often small-scale) social policies and actions have emerged in the spatial and institutional margins of national welfare states. In this context, the two central questions of the ImPRovE project were:

How can social cohesion be achieved in Europe? How can social innovation complement, reinforce and modify macro-level policies and vice versa?

For answering these questions, the project evaluated the Lisbon decade and the impact of the economic crisis and austerity measures on poverty and inequality. In addition, it studied the links between institutionalised macro level social policies and local initiatives of social innovation. By developing and evaluating policy scenarios that contribute to meeting the EU 2020 social inclusion target, it also looked at the future. Finally, it contributed to the development of new cross-national indicators in the areas of minimum income protection and poverty measurement.

BEUCITIZEN Project - Barriers to EUropean Citizenship :

Twenty years after the EU introduced the concept of ‘European Citizenship’ in the Treaty of Maastricht, the European Commission proclaimed 2013 the ‘Year of European Citizenship’. This was done to draw additional attention to a perceived problem: why don’t Europeans realise their rights as European citizens? With the term ‘realise’ here being used to mean both being aware of these rights and demanding, using and thereby materialising them. This year, the European Commission also awarded a consortium of 26 institutes from 19 countries in and outside Europe, coordinated by Utrecht University, a major research grant to carry out a 4-year research project to study this problem. This multinational and multidisciplinary project entitled bEUcitizen, identifies and analyses which impediments hinder European citizens from realising these rights and why.

Important Policy reviews:

Great Start in Life! - The Best Possible Education in the Early Years

Early childhood education and care is key for preventing the transmission of disadvantage across generations, for addressing child poverty and social exclusion, for providing Europe with skilled citizens, able to contribute to growth, innovation, justice, democracy, in line with the Europe 2020 Strategy for smart, sustainable and inclusive growth. However, European Union education and child care systems are challenged more and more by reductions in public expenditure and the increasing need to accommodate growing ethnic, cultural and linguistic diversity as well as socio-economic inequalities.

The EU research Framework Programmes for Research and Innovation (Horizon 2020 and FP7) have funded a substantial body of research on issues related to early childhood education and care. Their results support policy makers in developing more effective policies. These education policies are also reinforced by EU funding programmes like Erasmus+, which support early childhood staff and teachers in introducing effective educational practices. A selection of projects and partnerships supported by these programmes is also presented in this publication.

Fighting Poverty & Exclusion through social investment - A European research perspective

The fight against poverty and social exclusion is at the heart of the Europe 2020 strategy for smart, sustainable and inclusive growth. With more than 120 million people in the EU at risk of poverty or social exclusion, EU leaders have pledged to bring at least 20 million people out of poverty and social exclusion by 2020. In the aftermath of the crisis welfare states are called to address multi-level social risks while securing their financial sustainability.

This Review presents evidence from Framework Programme research projects with a view to addressing the challenges of poverty and social exclusion. It puts forward policy recommendations that put the emphasis on social investment and protection and pave the way for upward convergence in employment and social issues.

Addressing Terrorism - European Research in social sciences and the humanities in support to policies for Inclusion and Security

European societies, national governments and institutions of the European Union are currently facing an important challenge. Terrorist attacks hit France, Denmark and Belgium between 2014 and 2016, after several other deadly terrorist attacks in the United Kingdom, Spain, Belgium, the Netherlands and France in previous years. This specific terrorist phenomenon is new. Europeans need to understand what has happened and to be better prepared for anticipating, preventing and combating terrorism. The quality of the diagnosis is key to the efficiency of adequate policies. This Review thus aims to take stock of the available scientific knowledge on this new form of terrorism and suggest briefly what more should be done to increase this knowledge. Chapter 1 of the Review presents an overview of the approach that has dominated research over the last ten years: namely the notion of “radicalisation”. It also analyses the most important research projects funded by the EU under Framework Programme 7 (FP7) in this area in order to assess their contributions to the current inclusion and security challenges in Europe. Chapter 2 outlines why and how research lines could be broadened in order to understand the current terrorist phenomenon of jihadism. It also presents the most promising research trends. A brief conclusion sums up the main findings of the report and presents a series of recommendations in order to steer and support lines of research to better equip the EU with inclusion and security policies to address contemporary terrorism.

Understanding and Tackling the Migration Challenge - The Role of Research: Conference Report

The Directorate-General for Research and Innovation of the European Commission hosted a two-day international conference, ‘Understanding and Tackling the Migration Challenge: The Role of Research’, on 4-5 February 2016.

The conference brought together leading researchers and policy-makers from national, EU and international bodies to explore how European research can support effective and sustainable migration policies.

Findings from social sciences and economic research, including on integration, circular migration, migration and development, as well as data and statistical modelling, featured alongside short- to long-term health care needs of migrants and the link between climate change and current and future migratory processes. The Science4Refugees initiative, designed to identify and provide opportunities for refugees with scientific qualifications within the European Research Area was also presented alongside similar initiatives in the EU Member States. The identification of future research needs, both immediate and longterm, was a cross cutting theme at the conference.

Furthermore, a number of areas were highlighted in which EU funded research has produced highly policy-relevant recommendations. These recommendations and the identified future research needs are presented in this report.

Migration - Facing Realities and Maximising Opportunities

Migration has become a crucial issue for Europe, one that is likely to dominate policy and political agendas for many years to come. Migration is also increasingly presented, both in public and expert discourse, as a challenge requiring coordinated European responses, involving both Member States and the European institutions.

FP7 research projects studied different aspects of the migration phenomenon such as integration and diversity, trans-nationalism, temporary/circular migration, migration and development, migration flows, data modelling, to mention just a few of the areas covered. The European comparative perspective brought in by most of this research is an important added value of working with multi-country research teams in the study of migration.

O.9. L ESSONS LEARNT /C ONCLUSIONS

O.9.1. Relevance

Key findings:

 The calls and topics within SC6 are clearly aligned with the EU targets, Flagships

and Junker priorities. There is a stronger focus in four of the EU2020 targets (Employment, R&D, Education, Poverty) and four of the seven European

Flagships (Innovation Union, Youth on the move, New skills and jobs, Poverty).

 The work programmes do reflect to a greater extent European strategies and

polices than social needs identified by social actors.

The strengths are:

 The policy orientation of the SC6.  The broad scope of the problems and their relevance for the further development

of policies and innovations in other fields.

 The SC6 programme has a set of broad objectives closely aligned to the European

priorities. The programme is strongly policy-oriented and seeks to contribute to better governance and evidence-based policies to tackle the societal and political

problems of Europe.

The bottlenecks/weaknesses are:

 One limitation could be the extent to which these priorities continue to be relevant

for society all throughout or whether they can change, and how the program is able to accommodate the new ones. So far, the 2016-17 and the 2018-2020 Work

Programmes have responded strongly to evolving challenges.

O.9.2. Effectiveness

Key findings:

 Despite being in a very early stage, there are already early publications in highly

ranked scientific journals.

 There is a clear commitment with Open Access policies, as shown in the

questionnaire and webpages reviewed.

 Around 50% of the projects, funded under the several calls of SC6, have already

developed or expected to develop datasets/ databases. Others will produce simulation tools and other technological devices aimed to foster access to ground

information and provide evidences for better policy decision making.

The strengths are:

 The projects are clearly oriented to evidence-based policies.  There is a relevant involvement of stakeholders, including policy-makers and

CSOs. In particular the rise in the participation of stakeholders other than

academic and research bodies.

 Some of the projects introduce non expected outcomes and issues such as the

refugee crisis or the consequences of the Brexit in the development of the research, increasing the relevance of the SC6.

The bottlenecks/weaknesses are:

 Social impacts beyond the traditional publication and dissemination dimension is

an important objective of SC6. This impact is difficult to measure especially in the lifetime of a project. Instruments need to be developed in order to measure

impact beyond the lifetime of the projects.

O.9.3. Efficiency

Key findings:

 The high number of applications shows that the SC6 is very attractive. In general,

the high number of applications relative to the funding available demonstrates the need for such funding for research and innovation.

The strengths are:

 SC6 is the smallest funded theme under the overall Horizon 2020 Societal

Challenges, however it has been able to address very large of a number social problems through a variety of projects in many different areas and disciplines.

The bottlenecks/weaknesses are

 Success rate is the lowest among the Societal Challenges.  SC6 has the potential to attract more participants from countries with a lower

level of participation.

O.9.4. Coherence

Key findings:

 SC6 topics and calls include recommendations for taking into account other EU

and national initiatives in their implementation.

The strengths are:

 The different types of actions are shown to be complementary and are internally

balanced.

 Topics in the work programmes are well oriented to respond to political priorities.

The bottlenecks/weaknesses are:

 More than one dimension within the same topic makes a globally coherent reply

to the calls more difficult.

 Some areas are repeated in different calls defined which may pose some risk for

possible duplications..

O.9.5. EU Added Value

Key findings:

 Collaborative projects under SC6 areas are incorporating complex social,

economic, political and technological problems. Reaching out to potential partners from 28 Member States and beyond has been a substantial achievement –

in particular strengthening the European Research Area.

 In SC6 the participation patterns shows an increase as compared to the previous

programme in the participation of private, public and civil society stakeholders, which provide wider research based evidence for policy makers.

The strengths are:

 SC6 project results will provide solutions to societal problems that are pan

European.

The bottlenecks/weaknesses are:

 One can observe that the major part of SC6 funding is concentrated in small

group of countries that are dominant. There is lack of upward mobility based on excellence and more rapid European collaboration.

• P. 

  S ECURE SOCIETIES - P ROTECTING FREEDOM AND SECURITY OF E UROPE AND ITS

  CITIZENS

  P.1. I NTRODUCTION

  P.1.1. Context

Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens constitutes the seventh societal challenge under Horizon 2020 EU Research and Innovation programme. It has a funding of EUR 1.694 billion and it focuses on enhancing research and innovation activities needed to protect EU society, infrastructure and services as well as the prosperity, political stability and wellbeing.

Research and innovation activities under Secure Societies Challenge aim at understanding, detecting, preventing, deterring, preparing and protecting against security threats. This is especially important in a context of ever- increasing globalisation in which societies are facing security threats and challenges that are growing in scale and sophistication.

Today's security threats are focused on new multifaceted, interrelated and transnational threats. Aspects such as human rights, environmental degradation, political stability and democracy, social issues, cultural and religious identity or migration need to be taken into account. Moreover, the internal and external aspects of security are inextricably linked. According to estimates, the direct cost of crime, terrorism, illegal activities, violence and disasters in Europe counts for at least EUR 650 billion (about 5 % of the Union GDP) in

2010. 145

This evaluation presents the preliminary results of Secure Societies Challenge and it is based on literature review (review of Horizon 2020/FP7 documents, relevant policy documents), consultation (interviews with EU officials, beneficiaries and end-users), survey of project

coordinators and data analysis (CORDA data). 146

In terms of the number of participants in the survey, 118 project coordinators responded to the online survey, which represents a response rate of 35%. Regarding the interviews, the selection of stakeholders was focused on 12 projects; 9 projects out of 169 under the Horizon 2020 Secure Societies Programme and 3 projects under FP7. In total, 12 interviews have been conducted to date (8 with project coordinators/beneficiaries and 4 with EU officials).

P.1.2. Objectives and intervention logic

In general, the objective of Secure Societies Challenge is to foster secure European societies in a context of unprecedented transformations and growing global interdependencies and threats, while strengthening the European culture of freedom and justice.

The specific objectives of Secure Societies Challenge are:

145 Regulation (EU) No 1291/2013 i of the European Parliament and of the Council of 11 December 2013 establishing

Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No

1982/2006/EC, L 347/104, 20.12.2013

146 Risk & Policy Analysts (RPA), Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon

 To fight crime, illegal trafficking and terrorism  To protect and improve resilience of critical infrastructures  To increase Europe's resilience to crises and disasters  To strengthen border security  To improve cyber security  To ensure privacy and freedom  To enhance standardisation and interoperability of systems  To support the Union's external security policies

Overall, the abovementioned objectives aim to contribute to building a society and an economy based on knowledge and innovation across the Union which is a general objective of Horizon 2020. By supporting research and innovation activities in the area of security, Horizon 2020 helps tackling one of the key challenges for EU society.

Until now there are two Work Programmes within Secure Societies Challenge – the 2014- 2015 Work Programme and the 2016-2017 Work Programme. Grant Agreements for projects resulting from the first call for proposals are running, while projects from the second call for proposals have been signed recently and just started or are about to start in the coming months.

In terms of the differences between the two Work Programmes under Secure Societies Challenge, the 2016-2017 Work Programme introduced a new separate call for the Critical Infrastructure Protection. It covers prevention, detection, response and mitigation of physical and cyber threats to European critical infrastructure, transport infrastructure and means of transportation, communication infrastructure, health services and financial services. In addition to that, under the new Work Programme, the Security call has three sub-areas, namely Fight against crime and terrorism (FCT), Disaster-resilience (DRS) and Border security and external research (BES), whereas in the 2014-2015 Work Programme these areas were listed as separate calls. Lastly, all activities addressing cyber security were grouped under the Digital Security area.

It can be said that as for security research, Horizon 2020 shows a degree of continuity with its predecessor -the Seventh Framework Programme (FP7). The budgets dedicated to security research under these two programmes are very similar, namely EUR 1.4 billion under the FP7 and EUR 1.4 billion under Horizon 2020. Moreover, both research programmes maintain a mission-driven character, support the competitiveness of industry and support EU internal security policies. Regarding the differences, the areas of cyber security the protection of EU external security emerged in Horizon 2020.

The figure overleaf provides a logic diagram summarising the intervention logic for the Secure Societies Programme. It summarises the key needs that the programme sought to address through its objectives, the inputs that were required and activities undertaken. It also shows the anticipated outputs, results and impacts.

Figure 250 - Intervention Logic for Secure Societies

Intervention Logic for Secure Societies (continued)

Source: Risk &

Policy Analysts

(RPA), Interim

1423

Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020

1424 1425

P.2. I MPLEMENTATION STATE OF PLAY

P.2.1. Overview of programme inputs and activities

As of 1 October 2017, the EC contribution allocated to the implementation of the calls included in Work Programmes 2014-2016 and which have been closed at the date of «01/10/2016» has been «0,581» EUR billion, about «1.96»% of total expected budget allocated to in Horizon 2020, which is «1,694» billions for the period 2014-2020.

Through the Horizon 2020 Work Programmes 2014- 2017, each line of activity of «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens», namely, disaster resilience: safeguarding and securing society, including adapting to climate change ('DRS'), fight against crime and terrorism ('FCT'), border security and External Security ('BES') and digital Security ('DS')» was allocated a share of the overall budget as indicated in the table below.

Table 223 - Activities and allocated share of budget dedicated to «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens» for the programming period 2014-2017

Activities in the legal basis Estimated allocated share of Share of the overall SC7 thematic budget budget

(a) Fight crime, illegal trafficking EUR 110,6 million 6.5% and terrorism, including

understanding and tackling terrorist ideas and beliefs;

(b) Protect and improve the EUR 69,3 million 5.68% resilience of critical

infrastructures, supply chains and transport modes;

(c)Strengthen security through EUR 133,3 million 7.87% border management;

(d) Improve cyber security; EUR 217,6 million 12.85%

(e) increase Europe's resilience to EUR 74,9 million 4.42% crises and disasters;

(f) ensure privacy and freedom, EUR 72,97 million 4.26% including in the Internet, and

enhance the societal legal and ethical understanding of all areas of security, risk and management;

(g) enhance standardisation and EUR 29,3 million 1.73% interoperability of systems,

including for emergency purposes;

(h) support the Union's external EUR 27,9 million 1.65% security policies, including conflict

prevention and peace-building

Source: CORDA: 1 October 2016 and Work programme 2016-2017.

Table 224 - Key data on proposals per type of action for the «Societal Challenge 7:

Secure societies – Protecting freedom and security of Europe and its citizens»: Number of eligible and retained proposals, EC contribution requested (in million Euros) and success rates (as % of proposals submitted, and as % of budget available)

Type Of Nr of Nr of EC EC Success Success

Action Eligible Retained Contribution Contribution Rate Rate

Proposals Proposals requested by to Retained Proposals Funding

Eligible Proposals

Proposals (EUR million)

(EUR million)

COFUND- 1 1 8,9 8,9 100,0% 100.0% PCP

CSA 141 18 246,5 29,9 12,8% 12.1%

IA 317 30 1 632,7 164,9 9,5% 10.1%

RIA 456 37 2 158,0 186,8 8,1% 8.7%

SME-1 460 57 23,0 2,9 12,4% 12.4%

SME-2 276 16 395,2 22,7 5,8% 5.7%

1 651 159 4 464,3 416,1 9,6% 9,3%

Source: CORDA data, 1 October 2016, Success Rates by Type of Action (General).

P.2.1.1. Projects and proposals by call

In total, 153 projects have been funded under the Horizon 2020 Secure Societies programme between January 2014 and October 2016. 170 grants have been signed across the various FP7 calls. As would be expected, the number of calls decreases for each subsequent year as projects that start earliest are most likely to finish first. For example, there was only one project under FP7-SEC-2009-1 that was still ongoing after 31 December 2014, but there were 67 projects from FP7-SEC-2013-1.

Of the 1,651 eligible proposals that have been submitted under Horizon 2020 Secure Societies, only 694 were considered high quality and only 159 were retained. This corresponds to a ratio of eligible to retained proposals of approximately 10:1. This represents a significant reduction in success rates relative to FP7, for which the success rate of eligible

proposals to funded projects was 6:1. 147

P.2.1.2. Projects and proposals by topic

A large number of eligible proposals have been submitted for some of the Horizon 2020 topics. For example, there were 159 eligible proposals under Horizon 2020-SMEINST-2- 2016-2017, accounting for 9.6% of the total eligible proposals submitted, and 139 eligible proposals (8.4%) under Horizon 2020-SMEINST-2-2016-2017. DRS-17-2015-2, DRS-17- 2014-1 and DRS-17-2015-1 also account for a large share of the eligible proposals at 9.4%, 8.9% and 6.4% of the total respectively.

A large proportion of the grants that have been signed fall under the following three topics:

147 Technopolis Group (2015): Final evaluation of Security Research under the Seventh Framework Programme for

 DRS-17-2014-1: with 25 signed grants, accounting for 16.3% of the total  DRS-17-2015-1: with 17 signed grants, accounting for 11.1% of the total  Horizon 2020-SMEINST-1-2016-2017: with 14 signed grants, accounting for 9.2%

of the total

P.2.1.3. Projects and proposals by type of action

In terms of the type of action, a clear majority of the Horizon 2020 Secure Societies projects fall under the SME-1 category; 56 of the 153 signed grants, or 37%, are classed as SME-1. A further 33 projects (22% of the total) have been awarded under both IA and RIA. Only 19 (12%) of the projects that have signed grants since the start of Horizon 2020 are classified as CSA.

Table 225 - Horizon 2020 Secure Societies - Number and percentage of signed grants by type of action

Type of action No. signed grants % signed grants

CSA 19 12%

IA 33 22%

RIA 33 22%

SME-1 56 37%

SME-2 12 8%

Total 153 100%

Source: RPA analysis of CORDA data.

The following table provides data on the number and percentage of proposals by type of action for Horizon 2020 Secure Societies.

Table 226 - Horizon 2020 Secure Societies – Number and percentage of proposals by type of action

Type of action No. % eligible No. high % high No. % eligible proposals quality quality retained retained

proposals proposals proposals proposals proposals

COFUND-PCP 1 0% 1 0% 1 1%

CSA 141 9% 72 10% 18 11%

IA 317 19% 162 23% 30 19%

RIA 456 28% 228 33% 37 23%

SME-1 460 28% 94 14% 57 36%

SME-2 276 17% 137 20% 16 10%

Total 1651 100% 694 100% 159 100%

Source: RPA analysis of CORDA data.

P.2.1.4. EU financial contribution

The following table provides the distribution of EU financial contribution across the various Horizon 2020 Secure Societies calls. It shows that while a large proportion of the budget has been allocated to BES, DRS, DS and FCT calls, only a relatively small share (4.6%) has been allocated to the SME calls. It should be noted that the total for SME calls shown in Table 227 below (i.e. EUR18.3 million) understates the total amount of funding awarded to SMEs, as it Table 227 - Horizon 2020 Secure Societies – EU contribution by call (signed grants only)

Call EU contribution % EU contribution

Horizon 2020-Adhoc-2014-20 € 250,000 0.1%

Horizon 2020-BES-2014 € 22,611,889 5.7%

Horizon 2020-BES-2015 € 36,244,240 9.1%

Horizon 2020-DRS-2014 € 59,966,393 15.1%

Horizon 2020-DRS-2015 € 55,953,657 14.1%

Horizon 2020-DS-2014-1 € 49,334,152 12.4%

Horizon 2020-DS-2015-1 € 52,228,821 13.2%

Horizon 2020-FCT-2014 € 59,787,053 15.1%

Horizon 2020-FCT-2015 € 41,671,538 10.5%

Horizon 2020-SMEINST-1-2014 € 1,250,000 0.3%

Horizon 2020-SMEINST-1-2015 € 850,000 0.2%

Horizon 2020-SMEINST-1-2016- € 700,000 0.2%

2017

Horizon 2020-SMEINST-2-2014 € 5,638,318 1.4%

Horizon 2020-SMEINST-2-2015 € 6,167,619 1.6%

Horizon 2020-SMEINST-2-2016- € 3,682,711 0.9%

2017

Total for SME calls* €18,288,648 4.6%

Grand total € 396,336,391 100.0%

• * 

  SME calls have been highlighted in grey. Data for third parties have not been included.

Source: RPA analysis of CORDA data.

Table 228 - Key data on signed grants per type of action for the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens»: number, EC contribution, time-to-grant, projects' total costs, % of EC contribution in projects

TOA Nr of EC Share of Nr of Share of Participan Share of Simplified Signed Contributi EC Grants Signed t Total EC

FILTER Grants on to Contributi within 8 within Costs in Contributi Signed on to months Benchmar Signed on in

Grants Horizon k against Grants Project (EUR 2020 all Signed (EUR Total

million) Signed million) Costs Grants (Signed Grants)

CSA 19 32.6 0.2% 15 78.9% 34.3 95%

IA 33 175.5 0.8% 29 87.9% 209.6 83,7%

RIA 33 170.0 0.8% 30 90.9% 171.1 99,3%

SME-1 56 2.8 0.0% 54 96.4% 4.0 70%

SME-2 12 15.5 0.1% 8 66.7% 22.2 69,8%

153 396.3 1.9% 136 88.9% 441.1 89,8%

Source: CORDA data, 1 October 2016, Selected Projects and Signed Grants by Type of Action.

At the time of the interim evaluation, 0 projects are completed, 70 are ongoing, and 0 are abandoned. The programme has so far been implemented mainly Innovation Actions and Research and Innovation Actions.

P.2.2. Participation patterns

A total number of «70» projects have been selected so far.

Table 229 - Key data on participation per type of organisation for the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens»: number of participants, of project coordinators, of newcomers, of participations, and EC contribution to participations (in million Euros)

Legal Entity Nr of Nr of Nr of New Nr of Average EC type Participants Projects Comers in Participatio Participatio Contributio in Signed Coordinator Signed ns in Signed ns per n to Grants s in Signed Grants Grants Participant Participatio Grants ns in Signed Grants (EUR

million)

HES 203 28 9 285 1.4 110.8

OTH 44 2 25 49 1.1 12.3

PRC 384 89 205 461 1.2 153.8

PUB 119 2 50 185 1.5 32.8

REC 121 19 17 209 1.8 86.6

871 140 306 1 189 1.4 396.3

Source: CORDA data, 1 October 2016, Participants and Participations by Legal Entity.

Table 230 - Success rates (as % of proposals submitted and as % of budget available) per country for the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens»

Legal Entity Success Rate Success Rate Success Rate of type of Applicants of Funding

Applicant Applications (Applicants)

HES 26.0% 9.7% 9.2%

OTH 15.4% 11.2% 9.4%

PRC 13.7% 9.7% 8.2%

PUB 25.8% 16.3% 14.6%

REC 21.9% 11.3% 10.3%

17.7% 10.7% 9.3%

Source: CORDA data, 1 October 2016, Applicants and Applications by Type of Organisation (General).

Table 231 - Key data on participation per country for the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens»: number of participants, of project coordinators, of newcomers, of participations, and EC contribution to participations (in million Euros)

Country Nr of Nr of Nr of New Nr of Average EC Participants Projects Comers in Participatio Participatio Contributio in Signed Coordinator Signed ns in Signed ns per n to Grants s in Signed Grants Grants Participant Participatio Grants n in Signed Grants (EUR

million)

Austria 24 4 8 35 1.4 13.5

Belgium 35 5 13 59 1.8 16.6

Bulgaria 9 6 10 1.1 1.3

Croatia 3 2 3 1.0 0.1

Cyprus 5 1 2 6 1.2 1.5

Czech 7 3 7 1.1 2.0 Republic

Denmark 12 3 6 15 1.2 2.9

Estonia 10 1 2 10 1.0 1.7

Finland 26 4 12 35 1.4 12.7

France 45 8 12 64 1.5 32.9

Germany 79 7 19 115 1.5 42.7

Greece 40 6 7 71 1.8 23.7

Hungary 7 1 5 8 1.1 1.2

Ireland 25 6 8 31 1.3 9.6

Italy 104 20 36 144 1.4 48.4

Latvia 3 1 3 1.0 0.3

Luxembourg 11 1 5 15 1.4 5.4

Malta 2 1 1 3 1.5 2.5

Netherlands 38 7 14 47 1.3 17.0

Poland 23 3 11 28 1.2 7.3

Portugal 25 4 5 39 1.7 10.5

Romania 25 14 29 1.2 6.3

Slovakia 5 1 2 5 1.0 0.7

Slovenia 5 2 9 1.7 2.0

Spain 79 21 25 119 1.6 38.8

Sweden 22 1 5 25 1.1 9.3

United 110 23 43 150 1.4 55.6 Kingdom

Sum: 779 128 269 1 085 1.4 366.5

Source: CORDA data, 1 October 2016, Participants and Participations by EU-28 Member State.

Table 232 - Key data on participation per group of country EU28, EU-13, EU-15, Associated countries, Third Countries for the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens»: number of participants, of project coordinators, of newcomers, of participations, and EC contribution to participations (in million Euros)

Country Nr of Nr of Share of Nr of Nr of Average EC

Groups Partici Projects PJ New Participati Participati Contributio

pants Coordin Coordinat Comers ons in ons per n to

in ators in ors in in Signed Signed Participan Participatio

Signed Signed Horizon Grants Grants t n in Signed

Grants Grants 2020 Grants

Signed (EUR

Grants million)

ASSOCIATED 69 12 0.1% 28 78 1.2 25.0

CANDIDATE 15 4 18 1.2 3.7

EU-13 104 8 0.0% 51 121 1.2 27.0

EU-15 675 120 0.7% 218 964 1.5 339.5

THIRD 8 5 8 1.0 1.1 COUNTRY

Sum: 871 149 0.8% 306 1189 1.4 396.3

Source: CORDA data, 1 October 2016, Participants and Participations by Country group.

Table 233 - Success rates (as % of proposals submitted, and as % of budget available) per group of country for "SC7 Secure societies – Protecting freedom and security of Europe and its citizens"

GROUP Success Rate of Success Rate of Success Rate of

Applicants Applications Funding (Applicants)

AC COUNTRIES 17,3% 11,3% 8,3%

EU-13 14,4% 8,7% 5,7%

EU-15 18,7% 10,9% 9,9%

THIRD_PARTY 9,8% 8,7% 5,6%

Sum: 17,7% 10,7% 9,3%

Source: CORDA data, 1 October 2016, Applicants and Applications by Country groups (General).

P.2.2.1. Participation per type of organisation

The selected proposals represent a total of «871» participations, mobilising «5» distinct participants, namely organisations in the following categories:

• - 

  Higher or secondary education (HES) - Private for profit, excluding education (PRC) - Public body, excluding research and education (PUB) - Research organisations (REC) - Others (OTH)

  P.2.2.2. Attraction of new participants / newcomers

There are «1,3» % (213) of newcomers (excluding SME instrument).

P.2.2.3. Geographical participation patterns

A summary of the geographical participation in the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens» is presented below.

Graph 1 - Number of participations per country, Societal Challenge 7

Geographical Participation Patterns

160

140

120

100

80 Number of participation

60

40

20

0

AT BE BY CY DE EE ES FR HR IE IS LU MK MT NO PT RS SI TR UK XK ZA

Source: European Commission, DG HOME B4, based on CORDA data.

The above graph illustrates the geographical participation in the Societal Challenge 7 under Horizon 2020. It can be seen from the graph that the highest participation in the SC7 comes from the EU Member States, in particular from Germany, Italy, Spain and the UK which took part in more than 100 projects each. Belgium, Greece, France and the Netherlands followed them with having participated in more than 50 projects each. In terms of the participation in SC7 by the third countries, Israel, Norway, Serbia and Switzerland achieved the highest level of participation in comparison to the rest of third countries.

In general, a correlation can be noted between the level of participation and the level of contributions. The high level of contribution in most cases means the high level of participation. However, this is not necessarily true for all countries since Poland and Romania contributed less than, for example Sweden, but they reached the same level of participation.

P.2.2.4. International cooperation

A total of «102» entities from third countries applied to the programme, within the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens» project proposals. «9.8» % of these proposals were retained for funding, involving «10» third countries participants.

These participants are coming from «Belarus, Bosnia and Herzegovina, Georgia, Island, Israel, Kosovo, Macedonia, Mali, Norway, Serbia, South Africa, Switzerland, Turkey, Ukraine, United States of America and Yemen». The projects they are involved in relate to Digital Security, Fight against crime and terrorism (including Law Enforcement Capabilities, Ethical/societal Dimension), Border security and External Security (including Maritime Border Security, External Security, Ethical Societal Dimension, Supply Chain Security) and Disaster resilience: safeguarding and securing society, including adapting to climate change (including Crisis Management, Critical Infrastructure Protection, Communication Technologies and Interoperability).

P.2.3. Cross-cutting issues

In the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens», 41.1% (EUR 183.9 million) of the budget has been so far allocated to Sustainable development topics (the target for Horizon 2020 is at least 60%), 9.7% (EUR 43.4 million) of the budget to Climate related topics (it should exceed 35% of the overall Horizon 2020 budget) and 1.8% (EUR 7 million) of the budget has been so far allocated to biodiversity. 2.3% (EUR 9.3 million) of the EC contribution is ICT Research and Innovation related.

In terms of promotion of socio-economic sciences and humanities, it can be observed that 21.4% of partners are SSH partners, receiving «0.46»% of the EC contribution.

Women represent 19.8% of the project’ participants 148 , and 19.8% of the coordinators; 55 %

• (11) 

  of the members of the EC advisory groups are women/men.19.8% of the projects include a gender dimension in the research content.

Within the projects of the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens», 44.3% (EUR 396.4 million) of EC contribution is allocated to innovation actions. Within the innovation actions, 48.8% (EUR 85.7 million) of EU financial contribution focuses on demonstration and first-of-a-kind activities.

P.2.4. Other issues related to the state of implementation

In the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens», all 85 CSAs/RIAs and IAs have undergone a security scrutiny and an ethics review. 18 (21%) projects have classified results.

P.3. R ELEVANCE

P.3.1. Is the «Societal Challenge 7, Secure societies – Protecting freedom and security of Europe and its citizens» tackling the right issues?

P.3.1.1. The relevance of the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens» given the challenges to address

In December 2003, the European Council adopted The European Security Strategy entitled ‘A Secure Europe in a better World’. This strategy analysed and defined the EU’s security environment and, for the first time, identified the threats facing the Union, defined its strategic objectives and subsequent political implications. The European Security Strategy recognised five key threats – terrorism, proliferation of weapons of mass destruction, regional conflicts, state failure and organised crime.

148

In 2005, the security theme was incorporated into the EU research agenda 149 to reflect the

growing importance the EU had attached to this area in the previous period. Security was subsequently added to the Community’s 7th Framework Programme (FP7), which placed

emphasis on research themes 150 corresponding to major fields of advancement of knowledge,

promising scientific and technological avenues which are currently opening up, and challenging social, economic and industrial issues faced by the EU.

In 2010 the Commission published a Communication on the Internal Security Strategy in Action: Five steps towards a more secure Europe which set strategic goals for the EU's internal security policies and served as a basis for concerted action to address common

security challenges. 151 In 2015, the European Agenda on Security was adopted, setting three

overarching objectives: fighting terrorism and radicalisation, organised crime and

cybercrime. 152 According to it, 'research and innovation is essential if the EU is to keep upto-date

 with evolving security needs. Research can identify new security threats and their impacts on European societies. It also contributes to creating social trust in research-based new security policies and tools. Moreover, Innovative solutions will help to mitigate security risks more effectively by drawing on knowledge, research and technology.'

The security theme under FP7 benefitted from EUR 1.4 billion between 2007 and 2013 has been allocated EUR1.7 billion under Horizon 2020 for the period 2014-2020.

The following key objectives form the focus of the Security theme under Horizon 2020:

 Border security and external security  Fighting against crime and terrorism  Disaster-resilience: Safeguarding and securing society, including adapting to climate

change

 Digital security: Cybersecurity, privacy and trust

Detailed information on the overriding situation in the EU regarding the challenges faced in these different areas over the period covered by FP7 and the design and implementation of Horizon 2020 security research themes and consequently the context to their continuing focus

can be found in the study prepared by Risk & Policy Analysts (RPA). 153

Stakeholders' perspective on the challenges in the area of security in the EU and the relevance of the SC7

Stakeholders have noted that, unfortunately, the threats facing the EU are increasing and that more needs to be done to increase the EU’s resilience at all levels. It has been identified that the threats facing the EU are not only facing a single Member State but the European community as a whole. Thus, one REC from Sweden has noted that Secure Societies has an important role to fill in improving citizens’ safety and for creating a common Security strategy and market across the EU. This stakeholder has elaborated that projects often address areas of operation and expertise where information exchange is difficult due to

149 COM (2005) 118 i Communication from the Commission, Building the ERA of knowledge for growth

150 As opposed to The 6th Framework Programme, which was designed to help realise the European Research Area and

which placed strong emphasis on new instruments to structure research efforts and overcome fragmentation.

151 European Commission (2010): Communication from the Commission to the European Parliament and the Council, The

EU Internal Security Strategy in Action: Five steps towards a more secure Europe, COM(2010) 673 i Final

152 European Commission (2015): Communication from the Commission to the European Parliament, the Council, the

European Economic and Social Committee and the Committee of the Regions, The European Agenda on Security,

COM(2015) 185 i Final

153 Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk &

security classification aspects but where improved sharing of tools, concepts and innovation has high impact. Expressing a similar view, a HES from Portugal has noted that it is difficult to secure funding from alternative sources due to secrecy around sharing information on security issues and that it is also harder to get funding because stakeholders are more hesitant to collaborate on such sensitive issues. This stakeholder has identified that this means there is real added value to the Secure Societies programme.

A micro-enterprise from Denmark has noted that the security sector is strongly supported in Asia and the USA and that the EU needs to have the most efficient funding possibilities and political focus in order to ensure that, in the global context, the EU has a strong security sector. Similar views have been expressed by several other stakeholders.

P.3.1.2. The relevance of the «Societal Challenge 7: Secure societies – Protecting freedom and security of Europe and its citizens» to address European objectives

The objectives described at the previous section were identified in the Juncker Commission's priorities and Commissioner Moedas'3O Strategy as key challenges. In particular, those policy documents pay specific attention to:

 Fighting crime, illegal trafficking and terrorism, including understanding and tackling

terrorist ideas and beliefs

 Strengthening security through border management  Ensuring privacy and freedom, including in the Internet, and enhancing the societal

legal and ethical understanding of all areas of security, risk and management

 Improving cyber security  Increasing Europe’s resilience to crises and disasters.

P.3.2. Flexibility to adapt to new scientific and socio-economic developments

It has been noted that provided that the security threat is constantly evolving, the work programme for Secure Societies needs a balanced portfolio of topics that will meet Europe’s long term strategic needs and have the flexibility for a short term submission and selection process to cope with new security needs in a changing world. One stakeholder (a coordinator/beneficiary of multiple Horizon 2020 projects) has noted that there is currently a lack of flexibility to address the practitioners, industry and market needs. It has been suggested that future Horizon 2020 Secure Societies work programmes should be more flexible and, for example, open placeholders should be included in the work programme and flexibility should be introduced during the update of yearly work programmes to insert some topics corresponding to new short term needs. It has also been suggested to set up a new European Security Research Advisory Board to prepare the content of the security research programme beyond 2020, taking into account the EU policies, the new challenges, the market opportunities and the practitioners’ needs. The security cycle is very short (because security challenges are constantly changing) and so there needs to be flexibility in the work programme. With a long (e.g. 3 year) project, the security needs may have changed by the time the study finishes. It has been noted that in the UK the military has a one year project cycle and that this allows for fast innovation.

P.3.3. Addressing specific stakeholder needs

According to the project coordinators who participated in the online survey, all eight original objectives of the Secure Societies programme are still considered relevant to the needs and problems facing the EU by the majority of stakeholders. For example, the programme’s objective to “protect and improve the resilience of critical infrastructures, supply chains and transport modes” is judged to be “very relevant” by 65% of survey participants, while its objective to “fight crime, illegal trafficking and terrorism including understanding and tackling terrorist ideas and beliefs” is judged to be “very relevant” by 60% of respondents. It is interesting to note, however, that 18% of survey participants rated “strengthening security through border management” as being “not relevant” to the needs/problems facing the EU. This appears to conflict somewhat with the perception of EU citizens that the Commission should do more to protect the EU’s external borders (Figure 5 5).

Regarding the emerging needs or challenges that the Secure Societies programme does not yet (adequately) cover, only 14% of project coordinators identified the following emerging needs/challenges:

 Cyber-attacks and privacy via the Internet of things. One research institute from

Germany has noted that research alone is not sufficient for counteracting this and that new security standards and products are needed;

 Pollution and waste crime. One end user organisation has noted that more funds are

needed, but not for more research without implementation and coordination of law enforcement action;

 Defence and military. A micro-enterprise from the United Kingdom has noted that

there are gaps between the areas of defence and military;

 Integrating emerging risk management and resilience assessment;  Personal security (including tools, awareness, training);  Increasing the resilience of EU neighbourhood partners;  Security and safety in workplaces.

In addition, the project coordinators were asked to identify which of the aims of the Secure Societies programme are most relevant to the needs of key stakeholder groups. The results show that fighting crime, illegal trafficking and terrorism, including understanding and tackling terrorist ideas and beliefs, for example, was widely judged by project coordinators as being relevant to the needs of public and private security citizens, while protecting and improving the resilience of critical infrastructure, supply chains and transport modes was, unsurprisingly, judged to be most relevant to operators of/companies with critical infrastructure.

In respect to the relevance to EU citizens, the recent Eurobarometer survey on public opinion in the European Union highlights terrorism as a clear concern for European citizens. 39% of people surveyed indicated that terrorism is one of the most important issues facing the EU. Crime is also seen as an important concern among European citizens, and was mentioned by 9% of respondents. Interestingly, while concern about other issues, such as immigration and the economic situation appears to be shrinking over time, and concern about terrorism and crime appears to be growing.

Some of the most recent achievements of the Secure Societies programme are available to view to online on the Horizon 2020 website. On this site, there are multiple examples of how the outputs from the various projects under the Secure Societies programme will benefit stakeholders in the area. Two such examples are shown in the table overleaf.

Table 234 - Examples of how the projects under the Secure Societies programme are

relevant to stakeholders in the area 154

Project Description Relevance to stakeholders

iSAR+ An international team has The iSAR+ platform has been developed for a wide range of developed a system that can public protection and disaster relief services — including law exploit the widespread use of enforcement, search and rescue and medical help.

mobile electronic devices and social media to improve the

official response to emergencies The prototype iSAR+ system was tested in an exercise in

and other crises. Portugal involving local security organisations. On top of being a means to enable communication between the public and the

authorities, iSAR+ was also used as an information The iSAR+ platform enables management system, typically to manage resources including citizens using new mobile and ambulances and other vehicles, to control traffic, to call for online technologies to actively back-up and to define restricted areas. participate in the response effort in emergencies or crises,

through the bi-directional The consortium is now prototyping a version intended to provision, dissemination, optimise the use of resources by emergency teams and security sharing and retrieval of forces during emergencies.

information.

OPTIX Based on advanced optic The Optix consortium has successfully developed and tested a technology, the developed portable prototype capable of detecting extremely small device can detect quantities of quantities of explosives at a 20- metre distance, providing less than 1 mg of explosives at a police and security forces with an invaluable asset in the fight 20-metre distance. against bomb attacks.

To guarantee the programme’s success, an effort has been made to actively involve end users, European forces and security bodies specialised in detecting and neutralising explosive artifacts. Sessions have therefore been organised with experts from the Guardia Civil (Spain National Police), Mossos de D´ Esquadra (Catalan Police), Ertzaintza (Basque Police), as well as Police Forces from Romania, Poland and Italy to show them the technology and its possibilities.

Participation of end-users

It would appear that many of the projects under the Secure Societies programme have involved end-users either as a formal consortium member, or in an indirect or informal capacity, as evidenced by the information gathered from project coordinators during the

online survey. 155 A very broad range of end-user organisations have been mentioned by

survey participants as having been involved in projects directly and/or indirectly, including public and private security services, emergency services, operators of/companies with critical infrastructure, disaster relief and crisis management organisations, policy makers and regulators as well as operators in the financial services sector (e.g. banks and insurance providers), internet and digital service providers, social media companies, certification bodies and certification scheme operators, operators in the construction industry (architects, engineers), SMEs, municipalities, and charities, to name just a few.

154 Source: European Commission (2016): Horizon 2020, Achievements Secure Societies – Protecting freedom and security

of Europe and its citizens. Website available at: https://ec.europa.eu/programmes/horizon2020/en/newsroom/achievements/secure-societies-%E2%80%93-protectingfreedom-and-security-europe-and-its-citizens

155 Risk & Policy Analysts (RPA), Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon

One official from the European Commission explained that society is at the core of the research activities in the Secure Societies programme and that civil society and communities have to be included as major stakeholders. The official noted that, in this respect, the Secure Societies programme is different to other programmes that focus on one set of stakeholders. The official explained that there has been clear progress and that the Secure Societies programme is not just industry oriented, but society-centric.

P.3.4. Other issues related to relevance

Overall, it appears that the original objectives of the Secure Societies programme do still

correspond to the needs/problems in the EU. 156 Three of the objectives/activities are still at

the forefront of challenges to be address at the EU and international level, namely:

 Fighting crime, illegal trafficking and terrorism, including understanding and tackling

terrorist ideas and beliefs (mentioned in six out of the nine policy documents reviewed);

 Strengthening security through border management (mentioned in four out of the nine

policy documents reviewed); and

 Improving cyber security (mentioned in five out of the nine documents reviewed).

Even though the other objectives/activities are not identified as key needs/challenges, they still address important societal challenges. It should be noted that the three objectives above

157

mentioned are currently at the centre of all attention from stakeholders and citizens.

P.3.5. Lessons learnt/ Areas for improvement

It is clear that the original objectives of the Secure Societies programme still correspond to the needs and problems facing the EU, with this being confirmed through both the literature review and the consultation. The concerns addressed by the Secure Societies programme appear to align well with the concerns of EU stakeholders (emergency services, public and private security services, operators of / companies with critical infrastructure, disaster relief and crisis management organisations, policymakers and regulators, EU research community, EU industry, EU SMEs) and citizens and, by and large, it is anticipated that Secure Societies projects will provide benefits to these various stakeholder groups.

P.4. E FFECTIVENESS

P.4.1. Short-term outputs from the programme

To date, the Secure Societies programme has produced relatively few outputs such as publications in peer-reviewed journals, conference proceedings, books and intellectual property applications. In terms of publications, there were 111 of them to date. To some extent this is not unexpected given the limited amount of time that has elapsed since the beginning of the programme. It is likely that the total number of publications and intellectual property applications produced by the end of the Secure Societies programme in 2020 will be higher.

156 Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk & Policy Analysts (RPA), pp 73-93

157 Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk &

The Horizon 2020 Key Performance Indicators set a target of, on average, 20 publications per EUR 10 million of funding for all societal challenges. The Secure Societies programme is still some way off achieving its publications’ target at 2.8, but only just below the average of

3.2 for all Horizon 2020. 158 An alternative view of the same data is the cost per publication:

on average each publication currently costs EUR 3.6 million. The ratio of publications to projects is 0.72. During the interviews, the project coordinators of two Horizon 2020 Secure Societies projects each worth approximately EUR 5 million mentioned an expectation of six and ten publications, or 12 and 20 publications per EUR 10 million respectively. The overall number of publications per EUR 10 million and overall cost per publication for the FP7 Security projects running beyond 2014 are similar to the Secure Societies programme: this is surprising given that the projects are large and have been running for several years. The ratio of publications to projects for FP7 Security projects is higher at 2.0.

Further data on the number of intellectual property applications in FET, LEIT and SCs for Horizon 2020 Secure Societies and FP7 show that there are a relatively low number of intellectual property applications – just two patent applications and three utility model

applications 159 . The Horizon 2020 Key Performance Indicators set a target of two patent

applications and patents awarded in the area of the different Societal Challenges per EUR 10 million of funding. Again, it appears that the Secure Societies programme is still some way off achieving this target with 0.126, however the average for all Horizon 2020 projects is even lower at 0.09. The average cost per IP application is EUR 79.9 million. The overall number of IP applications per EUR 10 million and overall cost per IP application for the FP7 Security projects running beyond 2014 are approximately three times better than the Secure Societies projects: probably because these projects have been running longer.

The low number of patents for the Horizon 2020 projects to date could be partly explained by a comment from one of the interviews that the project was based on adapting existing technologies for new purposes through the development of tailored applications that were already patented before the start of the project and that the only IP application likely was for a trademark. This remark may apply to many of the RIA and IA projects which account for 71 of the 94 non-SME Horizon 2020 Secure Societies projects. This was confirmed through some of the interviews with project coordinators. Several examples were identified of the innovative use of existing legacy technologies. This comment was backed up by the coordinator of a BES project, who explained that much of their work involves either building on existing technology used, for example, in merchant shipping, to make it relevant within a border security context or in seeing how it might best be used within complex settings, such as land border posts (which have a particularly harsh and changeable environment). Furthermore, several projects involved the development of IT applications and software where patenting is not appropriate due to a tradition of open source collaboration.

Dissemination

Several interviewees mentioned the difficulties of working on restricted or classified projects; respondents questioned how their results can be exploited and disseminated if the project deliverables and research outcomes are at least partially classified.

On a more positive note, it was noted by some that the focus on dissemination within Horizon 2020 projects has helped to build awareness in multiple countries as to what is being developed. This has attracted some attention and interest in take-up from both the public and

158 Risk & Policy Analysts (RPA), Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon

2020 , pp. 143-144, Annex 3, Section A3.2.1

159

private sectors. One CSA project, for example, has received private sector co-funding to carry out more training workshops than the number originally specified within the proposal, as these training workshops were perceived to be both relevant and useful. Coordinators of two separate RIA actions mentioned that they had received interest from end-users (one from public sector bodies, another from industry) in piloting and further developing the work they were carrying out under Horizon 2020 – suggesting that the early-stage dissemination activities are already bearing fruit in terms of visibility and potential future uptake.

P.4.2. Expected longer-term results from the programme

The legal basis of Horizon 2020 specifies that one of the key objectives of Horizon 2020 is to leverage additional research, development and innovation funding, particularly from the private sector. By the end of Horizon 2020, it is hoped that EUR 35 billion of private funds will have been leveraged covering Horizon 2020 overall (i.e. not just the Secure Societies programme).

The external study performed for this evaluation (Risk & co, forthcoming) provides data on

the total value of funds leveraged through the Secure Societies programme. 160 It shows that

over EUR45 million in funding has been provided by HES, PRC, PUB, REC and OTH organisations alongside the EUR399 million EC contributions. The vast majority of this additional funding has come from the private sector, with 88% of the leveraged funding coming from PRC.

The EUR 45 million in private funds leveraged so far equates to 0.13% of the overall Horizon 2020 target of EUR 35 billion. Non EU countries have provided EUR 2 million: 70% of which comes from Israel.

To obtain an indication of likely direct results of the projects, their project descriptions were analysed. Of the 169 Secure Societies projects, 17 mention “prototypes” and 3 mention “demonstrators” in their descriptions. For the 170 FP7 Security projects running after 2014, 17 mention “prototypes” and 2 mention “demonstrators”. This element of the two funding rounds appears similar.

Technology Readiness Levels (TRLs)

The nine technology readiness levels are used to indicate the research stage from initial investigation to fully available to the market. The different types of action (TOA) are each broadly associated with different ranges of TRL, with some overlaps and exceptions:

 CSA – TRLs 1-4  RIA – TRLs 4-7  IA – TRLs 6-9

According to a commission official, some academic end users struggle with high TRL projects, but the political priority is on projects with higher TRLs and a more specific output rather than basic research. There were many different views about the need to concentrate on higher or lower TRL projects. Overall, the consensus is that the Horizon 2020 programme should continue to include research at all TRLs.

160

More generally, collaboration with third countries tends to become more selective and sensitive for research with higher TRL levels (as the EU may be competing with the industries of third countries), thus it can be expected that cooperation becomes somewhat more limited for projects that are closer to commercialisation.

The TRLs differ between the four major 2014-2015 themes as explained below.

Border and external security (BES)

Five of the BES Research & innovation Actions (RIA) topics have a target TRL of five or above reflecting the fact that RIA aims to bring technology from the laboratory and prepare it for market: all the projects in these topics appear to be planning substantial demonstrations in a relevant environment (TRL = 6) or in operational environments (TRL = 7). The topic supply chain security 2: “Technologies for inspections of large volume freight” has a target TRL of seven and the project in this topic, C-BORD says in its description “on 3 custom sites integrated solutions will be trialled, respectively addressing the needs of big seaports, small seaports and mobile land-borders” indicating that it is planning to reach a TRL of seven.

Fight crime and terrorism (FCT)

The expected TRLs for FCT topics, where specified, range from four to six. One project, DANTE, indicates that they expect to achieve a TRL of seven: the other projects appear to be intending to achieve the required TRL. There are two exceptions, ASGARD and ICT4COP, both of which have unclear descriptions and no indication of the likely TRL.

Disaster resilience (DRS)

The DS projects in topics DRS-01, DRS-02 and DRS-03 are Innovation Actions (IA) where a TRL is not specified, but the descriptions indicate TRLs are likely to be at least six which is appropriate for IA projects. In topic DRS-07 “Crises and disaster resilience – operationalizing resilience concepts”, the expected TRL for this topic is seven, however none of the projects appear to be aiming at any more than a five or six and it is hard to estimate a TRL for projects RESOLUTE and SMR. In topic DRS-19, “next generation emergency services”, a TRL of seven is expected and both projects, EMYNOS and NEXES, are aiming for this. For topic DRS-13 “2: Demonstration activity on tools for adapting building and infrastructure standards and design methodologies in vulnerable locations in the case of natural or manoriginated catastrophes” project LIQUEFACT ’s website talks about “a case study validation”, which sounds like a TRL of four rather than a TRL of seven that the topic requires.

Digital security (DS)

All but one of the DS projects are IA (the exception, CANVAS, is a CSA.) However, an expected TRL is only provided for DS-03 to DS-05 and these are either six or seven. All the IA projects in topics with no expected TRL appear to be working to a TRL of seven, with the exception of some of the DS-01 projects, where it is hard to evaluate the likely TRL. Interestingly in the topics where the TRLs are defined, it is hard to estimate the expected TRL for the individual projects, although where they can be teased out, they appear to be at least five and often seven.

P.4.3. Progress towards the overall Horizon 2020 objectives

P.4.3.1. Fostering excellent science in scientific and technological research

Detailed data tables are provided in Annex of the evaluation study 161 .

Organisations involved

Horizon 2020 Secure Societies’ signed grants were awarded to the different organisational types. Broadly speaking, PRC receive approximately 40% of the projects and EC contributions, HES about 25% and REC about 20%. CSA contracts are much more likely to be awarded to HES than to PRCs, however, IA contracts are more likely to be awarded to PRCs.

Cross-country cooperation

Fourteen cross-cutting issues are tracked within Horizon 2020 162 , including international

cooperation.

The involvement of third countries is approximately 8% for both Secure Societies and FP7 Security projects. The percentages for EC contributions for participations split by country type were similar. Detailed data on the country of the projects, participants and participations for FP7 projects ending after 2014 and Horizon 2020 projects respectively are available in the external study performed for this evaluation.

To date 13 Secure Societies projects are flagged as mentioning at least one third country or region representing EUR 71 million of the EC contribution to Secure Societies (18.8%), a little lower than the average EC contribution for all Horizon 2020 projects at 23%.

The percentage of projects involving all country groups (EU-15, EU-13, associated countries and non EU countries, excluding SMEs) is the same for FP7 and Horizon 2020, but the percentage involving three types has increased from 29% for FP7 Security projects to 36% for Secure Societies under Horizon 2020 and the percentage with only one country type has decreased slightly. This shows that the Secure Societies programme has greater cross-country collaboration than the FP7 Security projects ending after 2014.

Cross-sectoral cooperation

A further indication of collaboration is that between different types of organisation. The external study performed for this evaluation shows the organisation types of participations involved: percentages for EC contributions for participations split by organisation type were similar.

Investigating the number of different organisation types involved in the projects illustrates the degree of collaboration. The number of projects involving all four types (HES, PRC, PUB and REC) increased from 47% in FP7 projects after 2014 to 56% to in Secure Societies (OTH was not included in this analysis). This compares with earlier research in the Final Evaluation

161 Risk & Policy Analysts (RPA), Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020

162 European Commission (2015): Horizon 2020 indicators: Assessing the results and impact of Horizon 2020, available

of Security Research under FP7, undertaken by Technopolis Group in 2015 163 where “Most

Security Research projects involved participants from three (45%) or all four (42%) of the main types of organisations (HES, PRC, PUB and REC), while 11% involved just two types, and 2% only one.” The percentage of projects involving three or four types of organisation is approximately 90% in all three instances but the percentage involving all four organisation types, and thus gaining the maximum cross-sectoral impact has increased significantly from 42% to 56%.

More participants per project

Secure Societies projects display a greater number of participants on each project and a greater level of collaboration. There was a significant increase in the average for Secure Societies projects in the DRS and FCT work programmes between 2014 and 2015, whereas the average fell in BES and remained stable for DS. Overall, the average increased from 12 for FP7 Security projects evaluated in the Final Evaluation of Security Research under FP7,

undertaken by Technopolis Group in 2015 164 , to 13 for FP7 Security projects ending after

2014, and then 14 for Secure Societies. This reinforces the evidence from the cross-country and cross-sectoral analysis that international collaboration is steadily increasing.

However, whilst larger project consortia are an indication of strengthened international collaboration, with positive associations in terms of networking, there is no conclusive evidence that larger consortia are more effective than smaller consortia in implementing research projects and in delivering research outcomes. Indeed in the previous Interim Evaluation of FP7 Security, concerns were expressed by quite a few participants that larger consortia can be less effective than smaller consortia in achieving project objectives.

Through the interviews with Horizon 2020 Secure Societies projects, the difficulties of managing projects in which there are large consortia were identified as placing a high administrative burden on coordinators and can cause delays in the project cycle.

However, there was no strong view or consensus among project coordinators in Horizon 2020 Secure Societies as to whether the size of the consortium influences the optimisation of research outcomes, other than that the size can slow down the research implementation process.

Innovation actions

Excellent science is also indicated by projects undertaking innovative actions and focussing upon demonstration or first-of-a-kind activities. Of the Secure Societies projects, 33 are flagged as innovative actions, representing 21% of the signed grants for Secure Societies worth EUR175.5 million of EC contributions, 44% of the Secure Societies spend. Of these 33 projects, 11 are focussed on demonstration or piloting, representing EUR 85.7 million of EC contribution or 22% of the of the EC contribution to Secure Societies. Secure Societies has the highest percentages of projects and EC contribution allocated to innovative actions compared with the other Horizon 2020 programmes, but the proportion of innovative action projects focussing on demonstration or piloting is lower than most.

Project coordinators’ responses

163 Technopolis Group (2015): Final evaluation of Security Research under the Seventh Framework Programme for

Research, Technological Development and Demonstration, Final Report for DG Home.

164 Technopolis Group (2015): Final evaluation of Security Research under the Seventh Framework Programme for

Project coordinators that participated in the online survey were asked whether they agree that the Secure Societies programme plays an adequate role in supporting “open science” in the field of security, whether the programme promotes excellence in scientific and technological research, development and demonstration in the field of security and whether it has helped to position Europe on the global map of science and technology in the field of security research.

Pproject coordinators have provided a largely positive response. 165

European research teams are producing relevant scientific publications and a considerable part is the product of EU funded research. However, it has been identified that competition worldwide is very strong, mainly emanating from the US and China. During the interviews, one beneficiary noted that when the Horizon 2020 and FP7 programmes are compared to what is going on in the USA, there is a huge difference between them. The stakeholder explained that, in the USA, billions of dollars are given to some areas of research. In contrast, in the EU, there is a lack of prioritizing important areas and balancing and managing all issues.

This corresponds with the findings of the Societal Challenges Horizon 2020 assessment report carried out by CSES for the European Parliament’s ITRE Committee in late 2016, which found that the efficiency of programme management and implementation depends on having staff with the appropriate knowledge and experience in place. In a few instances, alternative management approaches were put forward, including that of the US Defence Research Agency (DARPA). In contrast to EU programmes, DARPA programmes are driven by highly specialised project officers who proactively seek out potential applicants based on the specific R&I needs perceived under the remit of the programme in question. Although one could take the view that this approach is contradictory to the ‘bottom-up’ approach of the Excellent Science Priority, the DARPA approach has been effective in establishing and maintaining a high-quality portfolio of research, which prioritises not only R&I production but equally the effective and strong management of R&I portfolios. It may be that Horizon 2020 as a whole can learn lessons from this approach.

P.4.3.2. Boosting innovation, industrial leadership, growth, competitiveness and job creation

Job creation

The majority (60%) of project coordinators that participated in the online survey agreed (either strongly or somewhat) that the Secure Societies programme has contributed to strengthening the competitive position of the European security industry. Most also agreed (strongly or somewhat) that the Secure Societies programme has contributed to the diffusion of innovation in the European economy (63%) and to better trans-national and cross-sector coordination and integration of research and innovation efforts (56%). A slightly smaller, but still significant, proportion (51%) of project coordinators indicated that the Secure Societies programme has contributed to generating jobs, growth and investment in the European economy.

Out of the 170 FP7 projects that are ongoing or that have been completed since the 31 December 2014, there are only 19 projects in the CORDA dataset which provide data on the number of direct full time equivalent jobs. In total, these 19 projects have reported 400 direct full time equivalent jobs, which is equivalent to, on average, 21.1 direct full time equivalent

165 Annex 3 to the Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study

jobs per project. As a crude estimation of the employment impact of FP7 projects, this figure can be extrapolated to all 170 FP7 projects that have been completed since the 31 December 2014 and that are ongoing. Based on this extrapolation, it can be estimated that approximately 3,580 direct full time equivalent jobs are associated with these projects.

During the interviews, one beneficiary noted that FP7 had created approximately 50 jobs in its organisation and that these had been maintained in Horizon 2020 (across multiple projects). Another organisation noted that it is still too early to estimate the number of jobs created.

When asked whether the Secure Societies programme has contributed to the diffusion of innovation in the European economy, one HES from Portugal has noted that there is an insufficient number of participants involved in the Secure Societies programme to take innovations to market.

Digital agenda

The digital agenda is one of the cross cutting issues (CCMI 10.1) indicating innovation and is measured with a flag of 100, 40 and 0%, indicating whether or not ICT R&I appears in the project’s objectives. Overall, 30% of the EC contribution to Secure Societies is to projects contributing to the digital agenda, slightly higher than the overall percentage of 29% for all Horizon 2020 projects.

Pre-Commercial Procurement Cofund

The Horizon 2020 Secure Societies programme includes two specific instruments that are designed to bring research closer to the market by bringing together industry, public authorities and end-users from the very beginning of a research project. These are:

 Pre-Commercial Procurement (PCP) which is designed to steer the development of

solutions towards concrete public sector needs; and

 Public Procurement of Innovative Solutions (PPI) which is designed to bring

innovative commercial end-solutions earlier to the market by enabling contracting authorities to act as a launch pad for innovative goods or services

These two instruments are funded through the Pre-Commercial Procurement Cofund.

Project coordinators that participated in the online survey were asked whether their project falls under either of these two instruments and, if so, whether they have provided value for their organisation and other participants in the project. Out of the four project coordinators that indicated that their project does fall under one of these two instruments, three agreed that the instruments of Pre-Commercial Procurement and Public Procurement for Innovation have provided value for their organisation and one indicated “don’t know”). It should be noted that the CORDA data indicate that no grants have yet been signed for the Pre-Commercial Procurement Cofund, identified by the entries where the TOA is P2P.

Major barriers for the successful use of pre-commercial procurement include lack of awareness and aversion to the risks involved. This risk-averse culture is inevitably reinforced within the FPs, where funding is based on attaining key indicators to ensure that value for money is guaranteed. Unfortunately, developing new and innovative solutions runs a much higher risk of failure than research, scoping or adapting pre-existing solutions to new purposes. In order to fully leverage PCP, a new approach may be required. This could perhaps be overcome by more direct intervention. It has been suggested that following the American model and setting up specialised innovation centres would allow for short-term projects focused on demand-led experimentation and innovation. These centres could help to cultivate entrepreneurship and remove some of the risk associated with PCP.

P.4.3.3. Addressing the major societal challenges

Two thirds of project coordinators that responded to the online survey agreed (either strongly or somewhat) that the Secure Societies programme has contributed to increasing the security of Europe’s citizens. Nevertheless, during the online survey, one HES from Ireland has noted that:

“We are interested in extremism and terrorism, Secure Societies increasing emphasis on competitiveness, diffusion of innovation, generating jobs and growth, markets, etc. to the exclusion of necessary basic research on some of these issues is potentially ultimately quite wrongheaded. It means that instead of figuring out the basic workings

of, say, (online) radicalisation, researchers are instead encouraged to come up with 'innovative' 'platforms' and 'tools' to solve issues that are not yet sufficiently well understood.”

This may however be a reflection of the lack of integration of SSH in SC7 and not directly related to the security of EU citizens. i.e. the expected outputs are not relevant to SSH research goals.

Sustainable development, climate change and bio-diversity are cross cutting issues (CCMI 8.1 – 8.3) measuring the impact on some major societal challenges. Climate change is measured with a flag of 100, 40 and 0%, indicating whether or not climate change appears in the project’s objectives. The percentage of EC contributions that are climate change related is 10% compared with an average for all Secure Societies projects of 25%. Three projects are flagged as 100% climate change objectives (climate change is a primary project objective) and these are

 ANYWHERE 166 empowering exposed responder institutions and citizens to enhance

their anticipation and pro-active capacity of response to face extreme and high-impact

weather and climate events

 I-REACT 167 increases resilience to natural disasters though better analysis and

anticipation, effective and fast emergency response, increased awareness and citizen engagement

 CLISEL 168 proposes an innovative approach to the question of how Europe can be

secured from the impacts of climate change in Third Countries.

The percentage of EC contributions that are related to sustainable development is 41%, which is lower than the average for all Horizon 2020 projects of 46%. All three projects described above under climate change are also flagged as 100% for Sustainable development, along with nine further projects including “Reaching Out” an 18 million euro project with 27 participants that aims to improve external disaster and crisis management efficiency for large external crises and includes five large scale demonstrations in Africa, Asia and the Middle East.

166 http://anywhere-Horizon 2020.eu/

167 http://www.i-react.eu/

168

Bio-diversity related contributions for Secure Societies are 2% compared with 4% for all Horizon 2020 projects. The three projects with a 40% bio-diversity measure are ANYWHERE and I-REACT described above and HOLOSCAN, a security scanning system which allows true real-time scanning of multiple moving persons and their bags.

Secure Societies projects’ indicators for climate change, sustainable development and biodiversity are all lower than the overall Horizon 2020 programme, but perhaps not as low as might be expected given the type of work involved.

A recent study looking at the Added Value of the whole Horizon 2020 programme 169 asked

respondents which of the seven societal challenges (SC) they expected to have an impact over the next ten years and the results for the SC7 respondents. Unsurprisingly, over 90% believe that their project will have an impact on the SC7 challenge. However, the expected impacts on all the other societal challenges was 25% or more and over 50% believed that their project would have an impact on SC6 making Europe into a more inclusive, innovative and reflective society.

P.4.3.4. Spreading excellence and widening participation

EU-13

Extending and involving the MS13 countries is fundamental to spreading excellence and widening participation in Horizon 2020 programmes. The percentage of participations in MS13 countries has grown significantly from 7% in FP7 Security projects ending after 2014 to 10% in Secure Societies projects to date. Four Secure Societies projects have over six MS13 participants involved:

 CARISMAND 170 - Culture And RISkmanagement in Man-made And Natural

Disasters

 CITYCoP 171 - Citizen Interaction Technologies Yield Community Policing  iCROSS 172 - Intelligent Portable ContROl SyStem  SEREN 3 173 - Security Research NCP Network 3

A further consideration is whether the investment is going predominantly to countries that already have a major security research programme in place or is it being used to build new research capacity. The FP7 SecRes Evaluation Report indicated that there were six MS with substantial capacity (Austria, Belgium, France, Germany, Sweden and United Kingdom) and three more with some capacity (Italy, Netherlands and Spain.) For the FP7 Security projects running after 2014, the first six countries took 50% of the signed grants and a further 30%

went to the other three countries. 174 Looking at the Horizon 2020 Secure Societies projects,

the allocation to the first six countries reduced to 34%, whereas the other three countries allocation rose to 35%, and all other countries allocations rose from approximately 20% to 30%. The investment appears to be shifting from the countries with established networks to countries where capacity is being built.

169 PPMI Overview of Horizon 2020 projects’ coordinators survey results, 2016

170 http://www.carismand.eu/

171 http://citycop.eu/

172 http://www.icross-project.eu/

173 http://www.seren-project.eu/

174 Annex 3 to the Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study

It was suggested that one means of strengthening the participation of the EU-13 would be to encourage their greater integration into networks of excellence, for example, through the ERANETs.

New comers

Another measure of widening participation is how easily newcomers can get involved. Overall, 306 (35%) of the participants so far are newcomers. Newcomers are more likely to be PRCs and a lot less likely to be HES or REC, presumably because most of the latter organisations were already involved if this type of funding is relevant to them and because there are, relatively speaking, far fewer of them.

Newcomers are more likely to be from MS13 countries and less likely to be from MS15 countries, with the percentages of all participants to newcomers relatively similar for the other country types.

Stakeholders that participated in the consultation were asked how easy or difficult it is for new players to participate in the Secure Societies programme. Approximately a quarter of respondents of the online survey thought it was very or quite easy for newcomers to

participate, whilst approximately a fifth though it was quite or very difficult. 175

A number of SC7 interviewees commented that there is a need to attract more newcomers to Horizon 2020 partly to avoid a situation in which funding is given to applicants where the coordinator and participants have well-established consortia and networks of partners. There are brokerage events (e.g. the Commission has a brokerage event, and there are similar events held in the Member States (e.g. by the Home Office in the United Kingdom) that help organisations to find suitable partners. However, many of the consortia are already formed before these events take place. Without an established and cohesive research network, it is apparently difficult to compete in the Secure Societies programme. Stakeholders have noted that it is difficult to find project partners and that established players appear to be favoured. This is especially difficult from an SME perspective.

Project beneficiaries have also noted that in order to bid for projects in the Secure Societies programme, organisations require a certain background and understanding of what has previously been done. Applicants need to be aware of ongoing work in different domains so they can demonstrate added value of their own project proposal.

Consequently, it was suggested by an interviewee that the EC should give consideration during the remainder of Horizon 2020 and in planning for FP9 as to how to stimulate new partnerships, since many consortia work together on successive FP projects. The interviewee suggested that in addition to giving additional award points to applications from consortia depending on how many SMEs are in the proposed consortium, the number of SMEs that are newcomers could be prioritised as to encourage the lead partner at the consortium formulation stage to team up with potential partners outside their known network. A further problem in terms of widening participation is that although the participation levels of the new Member States has increased, a small number of EU-15 Member States account for a disproportionate share of participation, in Secure Societies and in Horizon 2020 more generally. The small number of EU Member States that dominate in terms of the number of participations in Horizon 2020 overall, such as Spain, the UK, France, Germany, Italy,

175

Netherlands, Sweden and Greece, also have a strong position in Secure Societies. The same applies to Associated Countries, where Israel, Norway and Switzerland dominate.

P.4.3.5. Science with and for society

Gender

Three cross cutting measures relate to gender and have data available (CCMI 6.1, 6.2 and 6.4). The first evaluates projects that take into account the gender dimension in research and innovation content. It is believed that integrating the gender dimension in research and innovation content should help to improve the scientific quality and societal relevance of the

produced knowledge, technology and/or innovation. 176 The analysis of CORDA data shows

that 14.9% of the grants signed under Secure Societies have taken into account the gender

dimension, this is significantly lower than the 24.7% average for Horizon 2020 as a whole. 177

Approximately half of the projects with signed grants under Horizon 2020 Secure Societies have a female coordinator, and this is higher than the 35% average for Horizon 2020 as a whole. However, separate data on the gender of the Legal Entity Appointed Representative (LEAR) contact for each of the signed grants paint a yet another picture. Where available (data for 5 signed grants are missing), these data show that 68% of the LEAR contacts for signed grants are male and 32% are female. The proportion of female LEAR contacts has increased since the FP7 Security programme, when the vast majority (83%) of LEAR contacts for participations in Security projects were male (17% female), from the Final

Evaluation of Security Research under FP7, undertaken by Technopolis Group in 2015 178 .

Another set of data measures the gender of project staff, and indicates that just over a quarter are female. The Final Evaluation of Security Research under FP7, undertaken by Technopolis Group in 2015 does not provide a direct comparison but again the vast majority of key personnel in FP7 projects were male (80%). Overall, gender equality has improved since FP7, but the balance between males and females under Secure Societies is not yet even and is poorer than Horizon 2020 overall.

Social Sciences and Humanities (SSH) & Responsible Research and Innovation (RRI)

Two cross cutting issues indicate the impact of Horizon 2020 Secure Societies projects upon science for society: SSH (CCMI 4.1) and RRI (CCMI 5.1). Overall, 33 projects are flagged as SSH, 21% of all projects, representing an EC contribution of 106 million euros or 26% of the contribution to Secure Societies. 12 projects are flagged as RRI or 8% of all projects, representing an EC contribution of 40 million euros or 10% of the EC contribution to Secure Societies. On both measures, Secure Societies is performing at approximately the average level for Horizon 2020.

An example of SSH and RRI in action is the TOXI-triage project, 179 one of eight projects

flagged as taking into account the Gender dimension, RRI and SSH: it also has ICT as one of its primary objective so it 100% supports the digital agenda. This EUR12 million project has

176 European Commission (2016): Promoting gender equality in research and innovation. Article available at:

https://ec.europa.eu/programmes/horizon2020/en/Horizon 2020-section/promoting-gender-equality-research-andinnovation

177 Annex 3 to the Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study

prepared by Risk & Policy Analysts (RPA), p.152.

178 Technopolis Group (2015): Final evaluation of Security Research un the Seventh Framework Programme for Research,

Technological Development and Demonstration, Final Report for DG Home.

179

18 participants and aims to develop and field/trial a new level of medical care and site management during triage within rescue efforts in a CBRN (chemical, biological, radiological and nuclear) incident.

P.4.3.6. Science for policy

Information from interviews has identified that project coordinators do not always know the policy or legal landscape and so it is important that project coordinators are given access to the right information on the links between policy and research. One official from the European Commission has noted that in the topic description there is usually a reference to the main legislations of relevance, to facilitate coherence and that projects are actively encouraged to provide policy recommendations at all levels of implementation, from very local to EU level.

Results from the Final Evaluation of Security Research under FP7, undertaken by

Technopolis Group in 2015 180 indicated that nearly 60% or more of respondents though that

the programme had had a high or medium impact upon all seven indicators relating to the European Security Industrial Policy and Industry Market.

P.4.4. Early success stories

The early success stories identified here below demonstrate progress towards achieving the primary aims of this societal challenge: DARWIN – contributes to enhance the resilience of our society against natural and man-made disasters; WOSCAP – contributes to support the Union's external security policies through conflict prevention and peace building; ReCRED contributes to enhanced cyber-security services.

"DARWIN 181 - Expecting the unexpected and know how to respond" - Call Horizon 2020-

DRS-2014

DARWIN is a Research and Innovation Action, with a budget of EUR 4,998,896.25, including EUR 4,998,896.25 of EU contribution, which started on 1 June 2015 and will end on 31 May 2018 (36 months).

DARWIN is contributing to improve responses to expected and unexpected crises affecting critical societal structures during deliberate man-made disasters (e.g. cyber-attacks) and natural events (e.g. earthquakes). The project is developing European Resilience Management Guidelines (ERMG), which will support the ability of crisis management experts and those responsible for public safety to anticipate, monitor, respond, adapt, learn and evolve, to operate efficiently in the face of crises. After one year, DARWIN achieved promising results: i) definition of the catalogue of resilience concepts and requirements for the development of the ERMG; ii) launched the Community of Resilience and Crisis Practitioners; iii) and presented the initial evaluation plan for the pilots. The guidelines will be user-friendly and presented in formats for easy usage and maintenance. Furthermore, the project is exploring innovative tools such as serious gaming and training packages to facilitate the adoption of the ERMG. The target beneficiaries of DARWIN are infrastructure operators: service providers and related stakeholders who have responsibility for critical infrastructures that might be affected by a crisis as well as the public and media.

180 Technopolis Group (2015): Final evaluation of Security Research un the Seventh Framework Programme for Research,

Technological Development and Demonstration, Final Report for DG Home.

181

"WOSCAP 182 - Whole-of-Society Conflict Prevention and Peacebuilding" – Call Horizon

2020-BES-2014

WOSCAP is a Coordination and Support Action, with a budget of EUR 2,018,034.75, including EUR 1,990,114.25 of EU contribution, which started on 1 June 2015 and will end on 30 November 2017 (30 months).

WOSCAP seeks to enhance the capabilities of the EU for implementing conflict prevention and peacebuilding interventions through sustainable, comprehensive and innovative civilian means. The project is exploring the principles, processes and tools that can enhance EU capabilities, by focusing on three types of EU interventions: multi-track diplomacy, security sector reform, and governance reform. The assessment is based on field research in Georgia, Mali, Ukraine and Yemen, and desk reviews looking beyond these countries. In addition, WOSCAP is creating a ‘community of practice’, for validating the project results, bringing together policymakers, civilian and military practitioners, academic experts and beneficiaries of EU interventions.

WOSCAP has the potential for having a high impact on EU policies (namely on the Common Security and Defence Policy and the EU Global Strategy in the area of Security and Defence), since the project has started to generate new knowledge on the EU capabilities in conflict prevention and peace building (especially in the sub-fields of ICT and Security Sector Reform in EU peacebuilding).

"Re-CRED 183 - 'From Real-world Identities to Privacy-preserving and Attribute-based

CREDentials for Device-centric Access Control' - Call Horizon 2020-DS-2014-1

Re-CRED is an Innovation Action, with a budget of EUR 6,325,156.00, including EUR 4,997,242.00 of EU contribution, which started on 1 May 2015 and will end on 30 April 2018 (36 months).

ReCRED is developing an integrated next generation access control solution, by designing and implementing mechanisms that anchor in one single point all access control needs to mobile devices that users regularly use and carry.

The project has successfully developed a programme that collects different online identities into one online persona, which can then be securely linked to the real-world identity and authenticated via security measures (different attributes, such as typing patterns, face recognition, and mobility signatures) on the phone or computer. The results achieved so far demonstrate a high innovation potential and contribute to prevent cybercrime, and more specifically identity theft and online fraud.

P.4.5. Lessons learnt/Areas for improvement

A barrier to the full effectiveness of the programme lies in the significant differences in terms of success rates across the various call topics, which may cause deviations between the expected and the actual topic coverage and therefore lead to an unbalanced project portfolio, where some areas of research may be over represented and others neglected.

182 http://www.woscap.eu/

183

For instance some topics were very popular and attracted a higher number of proposals than expected with respect to the programme size. For instance in the same call 'Disaster Resilient Societies', the topics DRS-07-2014 and DRS-01-2015 received massive numbers of proposals, leading to respectively to 5 and 3 grants, whereas some other topics did not attract applicants and led to 'orphan topics', without any proposal retained for funding, e.g. DRS-5- 2014 and DRS-15-2015. The topic DRS-5-2014 had to be rewritten and reintroduced in the Work Programme 2016-2017, causing a two year delay in addressing a critical need, namely paving the way for a pre-commercial procurement for situation awareness system for civil protection authorities in the EU.

Such a situation was particularly acute for the Work Programme 2014-2015 and has been mitigated in the Work Programme 2016-2017, with the introduction of dedicated budgets for a majority of topics.

As a consequence of the above, there are parallel grants which are funded under the same topic and are expected to deliver similar results, in line with their respective topic requirements. Whereas complementary approaches can be beneficial and respond to a variety of stakeholders needs, the competition between projects to reach out to the main stakeholders e.g. at the EU level can be largely counterproductive and have an adverse impact.

The lack of clearly defined and contractually binding mechanism to make sure such "parallel" grants are coordinating their effort could be perceived as a barrier to ensure full effectiveness of the programme.

Actions to take advantage of synergies between projects can be implemented, but remain too dependent on the good will of the project coordinators and the project officers, and represent an additional project monitoring task which is not acknowledged in the current state of play.

Projects are very often expressing the need for an ad-hoc post project support, when successful results are being delivered. There is currently no support mechanism which can directly be granted to projects which have achieved a very high TRL or very promising results in order to bring them to actual exploitation and implementation.

In regards to consultation, most stakeholders have agreed that the Secure Societies programme promotes excellence in scientific and technological research and that it also plays an important role in supporting “open science” in the field of security.

Nevertheless, it would appear that the Horizon 2020 Secure Societies programme has, to date, produced relatively few publications and intellectual property applications – although, to some extent, this is not unexpected given the limited amount of time that has elapsed since the beginning of the programme.

Low success rates for proposals are a key source of dissatisfaction among stakeholders and have a disincentive effect on the research and technology community. Where proposals are successful, it would appear that the amount of funding allocated to projects is generally satisfactory to achieve the projects’ objectives; however, a large share of project coordinators (35%) have also indicated that better research outcomes could be achieved with a larger budget.

To date, over EUR 45 million in funding has been leveraged alongside the EUR 399 million contributed by the European Commission. Most (88%) has come from the private sector.

The analysis of the CORDA data shows that the average time to grant for the Horizon 2020 Secure Societies programme has improved considerably since FP7 and is now just 213 days across Secure Societies as a whole. This compares to an average time to grant of 546 days for FP7 projects that are still ongoing or that have finished since the 31 December 2014. Although only 58% of Horizon 2020 Secure Societies projects have achieved the Commission’s target of 240 days, 90% of Horizon 2020 Secure Societies projects have managed to achieve a time to grant of 250 days or less. It has been noted during the consultation that delays in receiving funds from the Commission may be problematic for SMEs.

Most project coordinators (75%) have indicated that their project will achieve its aims in full. Most (90%) have also indicated that end-users are somewhat to very likely to use the research results/outputs. There are, nevertheless, some issues that have been identified as potentially posing a barrier to greater uptake by end-users, with the most prominent including: confidentiality arrangements, resource/financial constraints, the need for further research/development and a lack of knowledge/trust among end-users. It has been identified that one of the main gaps in the programme is that, at the end of the project, there is insufficient financial, technical, policy and project support to implement project results and take innovations to market.

It has been identified that global competition in the security industry is very strong, mainly emanating from the US and China. Most project coordinators that participated in the online survey agreed that the programme is helping to position Europe on the global map of science and technology in the field of security research and that it has contributed to strengthening the competitive position of the European security industry.

Most also agreed (strongly or somewhat) that the Secure Societies programme has contributed to the diffusion of innovation in the European economy (63%) and to better transnational and cross-sector coordination and integration of research and innovation efforts (56%).

Two thirds of project coordinators that responded to the online survey have agreed (either strongly or somewhat) that the Secure Societies programme has contributed to increasing the security of Europe’s citizens.

Stakeholders have explained that submitting a proposal is very difficult and that, although the participant portal has improved access to information for new players, it is still an expert task to know how to bid. It has been identified that there tend to be established consortia and that this can make it difficult for newcomers to participate. Identifying additional sources of funding has also been identified as a source of difficulty by a large proportion (28%) of project coordinators.

P.5. E FFICIENCY

P.5.1. Budgetary resources

Analysing the success rates from eligible proposals to signed grants gives an indication of the efficiency with which the funding is distributed. The highest proposal success rates were achieved under CSA. There is a substantial difference in the proposal success rates for SME-1 versus SME-2, the former being amongst the highest rates at 12% and the latter having the lowest of all at 4%.

Comparing the proposal success rates for different countries, some countries have secured more EU grants than others. Finland and Luxembourg, for example, have both achieved an application success rate of 18%. At the other end of the spectrum, whilst Slovakia only achieved five participations in signed grants for its 112 applications in eligible proposals, a success rate of only 4%, Lithuania did not succeed with any of its 41 applications.

Comparing success rates from applicants in eligible proposals to participants in signed grants by country, there is a disparity between EU countries. Luxembourg has the highest applicants’ success rate at 37%, while Hungary and Croatia have only 8% and Lithuania is zero.

Looking at the success rates for both proposals and applicants, it is clear that those from the EU13 fare less well than from EU15 countries. The average success rate for all Secure Societies proposals is 11% and for applicants is 18%. Only two of the EU13 countries managed a success rate higher than the average, Estonia at 14% for proposals and 29% for applicants and Romania at 13% for projects and 20% for applicants. Worryingly, 25 Lithuanian organisations put in 40 applications and failed to win any: Latvia and Malta had similar levels of entry and each succeeded with three projects.

Several evaluation studies have attributed this to the fact that such countries have strong domestic and international networks of research actors (e.g. through participation in the security-specific ERA-NETs), making it easier for them to find partners and get into strong consortia. The participants that were among the biggest players in FP7 Security Research still have a dominant position in Horizon 2020 Secure Societies. Interestingly, Austria and Belgium have been slightly less successful to date than in the previous period. However, it is important to caution that a direct comparison in terms of the number of participations can be misleading since in Horizon 2020, there is a general trend towards larger projects involving a greater number of partners than in FP7 so the data is not directly comparable. Moreover, a direct comparison at this early stage in implementation (only two years of Secure Societies calls) means that any conclusions with regard to country participation are necessarily caveated by the fact that this evaluation takes place at a relatively early stage in implementation.

It was also observed in the FP7 SEC evaluation that the existence of national civil security research programmes was a factor influencing success rates and readiness to participation in the FPs. Whilst this continues to be at least partially the case in Horizon 2020, since several national programmes have been continued, an interviewee from Germany was of the view that the types of calls that have been issued to date under Secure Societies are different from the types of research projects that are funded under the German federal security research programme.

The success rates have decreased for every work programme between 2014 and 2015. 184

The decreases for four work programmes are particularly significant:

 BES from 20% to 10%  DRS from 12% to 7%  SME11 from 17% to 11%  SME-2 from 16% to 5%

As the grants for 2016 are still being awarded as this data was taken, the figures for 2016 will probably change significantly so these are not considered. The reduction in success rates is

184

partly due to the 25% rise in eligible proposals from 582 in 2014 to 723 in 2015. The rise was particularly marked for BES proposals, which nearly quadrupled and SME proposals, which tripled.

Low success rates for proposals appears to be a key source of dissatisfaction among project coordinators and other beneficiaries and several stakeholders have noted that success rates for proposals appear to be falling. One large organisation that has participated in multiple FP7 and Horizon 2020 studies noted that it used to have a success rate of about 40% for proposals but this has now dropped to about 20%-25%. The stakeholder explained that because national funding is decreasing, a larger number of organisations are trying to obtain EU funding. Stakeholders have identified that there is not enough funding globally to cover the broad scope of challenges facing the EU and that there is a need for more investment in security research. One project coordinator has noted that with the new requirement on ‘practitioners’ involvement introduced for 2016 calls, the oversubscription has been partly reduced but there are large variations between the various calls.

It would appear that having a low success rate has had a disincentive effect on the research and technology community.

Another indication of efficiency of funding allocation is the type of organisation awarded grants for each call. Overall, there is a significant change in the proportion of grants awarded to RECs (decrease of 5%) to PUBs (increase of 5%) between FP7 Security projects ending after 2014 and Secure Societies projects. This trend can be seen continuing in the detailed data for Secure Societies: the proportion rises from 2014 to 2015 for PUB for all work programmes. A further detailed observation is that the proportion for HES decreases for all work programmes. This appears to reflect the move to involve more end-users in projects as end-users like police forces and border security are all public sector organisations.

In terms of the timing of funding, the majority (88%) of project coordinators that participated in the online survey have indicated that project funding from the European Commission was received on time. Only 10% of project coordinators have indicated that funding from the European Commission was delayed. Most project coordinators (58%) have also indicated that funding from other sources was received on time, although a larger proportion (33%) indicated “don’t know”.

One organisation that has been involved in multiple FP7 and Horizon 2020 studies has noted that although initial funding (pre-financing) is usually sent on time, final payments can be sometimes be delayed. The stakeholder gave one example under FP7 where it took two years after the final project was submitted to get paid.

One small PRC that participated in the consultation noted that six months passed before they received the initial round of pre-funding. This meant that the company had to finance some of the initial stages itself before the monies were received. While this did not have any substantive effect on the project, the stakeholder noted that such delays could be problematic for SMEs in general and this finding was corroborated by other participants in the consultation.

P.5.2. Programme's attractiveness

P.5.2.1. Mobilisation of stakeholders

A central instrument to promote and support participation to Horizon 2020 secure societies under dedicated projects (SEREN 1-3). The project SEREN 2 was completed in December 2013. It partially laid the foundations of the successor project SEREN 3 which aims at supporting the cooperation of NCPs under the Horizon 2020 secure societies programme. SEREN 2 organised partnering days and brokerage events already for the relevant FP 7 community. It established a database for potential project partners (see below) and also aimed at enhancing the capacities of the Security NCPs to enable them to assist projects.

SEREN 3 builds upon these efforts and aims at widening them in both content and audience:

 Database of potential partners in security projects: One major activity to promote and

facilitate participation to projects has been the development of the Security Research Map (SeReMa) database which was created as part of the security NCP project SEREN 2. The objective of SeReMa is to increase the visibility of security related research in Europe and to optimise the networking between relevant organisations, such as universities, public authorities, end-users, suppliers of security solutions or

operators of critical infrastructures.

 Until the closure of SEREN 2, 532 organisations were registered and their profiles

published thorough the database. Currently, in the context of SEREN 3, new features are being developed to facilitate the search for partners and constant efforts are undertaken to enhance the number of registered entities which has reached 665 from

more than 32 countries by November 2016.

 Brokerage events/ Infoday: As a central brokerage event, the first Info Day was

organised on 5-6 April 2016 in Brussels. The event was divided into three main parts, namely:

o Information on the 2016-2017 Secure Societies Call and complementary presentations by REA and EC about legal conditions, rules for participation, etc.,

o Project ideas and capabilities presentations, o Bilateral networking meetings.

At this event, representatives from academia, research institutes, industry, NGOs and practitioners as well as from REA, EASME, and several relevant DGS came together. 380 entities from 30 different countries registered, of which only 200 finally participated, mostly due to the repercussions of the March attacks in Brussels. In total 189 bilateral meetings took place and 44 project ideas and capabilities were presented.In addition to the Infoday, 455 representatives of potential partners for security projects were reached in other brokerage events that focussed on specific topics in the course of 2016.

Moreover, a number of networking events took place with relevant players in the security sector and with security NCPs of third countries but also beyond the security sector with other NCP networks.

 Training: Three capacity building trainings and one webinar were organised in the

framework of the 3rd NCP network project (SEREN 3), to which altogether 72 trainees participated, mostly multipliers from NCPs. The content of the trainings spanned from how to write a proposal, to how to take advantage of synergies between Horizon 2020 and other funding instruments to how to engage SMEs in security

research. A fourth training was held in November of which data is not yet available.

 Dissemination: The network of security NCPs functions also as hub for dissemination

activities, such as:

o Maintaining a dedicated website (www.seren-project.eu) o Production and distribution of leaflets with basic data about Horizon 2020 Secure o Production of a video about success stories (planned)

Providing personalised programmes to NCPs to increase their capacities, taking into account their different readiness level and experiences is also a key objective to be achieved within SEREN3. Of major importance is the need for networking with key players outside Europe, in order to promote international cooperation activities and in particular, with those from non EU countries interested in the Security research field. First activities carried out in SEREN3 are targeting Canada and South Africa.

A first interim evaluation of the stakeholder engagement through SEREN3 will be presented in May 2017, when there is sufficient data available of the different engagement activities to proceed with a meaningful analysis, whereas the final evaluation will be available in April 2018.

P.5.3. Cost-benefit analysis

Stakeholders that participated in the consultation were asked whether the costs of the Horizon 2020 Secure Societies programme outweigh the benefits that have been achieved so far.

One large organisation that has participated in multiple FP7 and Horizon 2020 projects noted that this really depends on the country. For instance, a comparison of the contribution that each Member State gives to the EU versus what it gets in return shows that in France the costs of participation outweigh the EU funding received through successful participations, while in the UK, the opposite is true. This stakeholder noted that ultimately the benefits (monetary, non-monetary) should outweigh the costs. Another large organisation that has participated in multiple projects noted that the costs slightly outweigh the benefits. This was attributed to low success rates and calls for proposals lacking clarity due to being insufficiently prescriptive about ultimate research needs that should result from the call. Interview feedback confirmed that in the views of some participants and end-users, not all calls under all research topics were regarded as sufficiently clear and the consequence of this was difficulty in knowing how to develop a proposal that met the expectations of the call authors at the Commission. The counter-argument in this regard is that the Commission has purposely sought to make calls less prescriptive in order to be more open and flexible. Whilst some applicants appreciated this flexibility, they were equally concerned that some research call topics were quite vague and therefore difficult to know whether they had successfully met the criteria.

P.5.4. Other issues related to efficiency

Project coordinators that participated in the online survey were asked to indicate the extent to which they are satisfied or dissatisfied with some of the main aspects of the Secure Societies project cycle.

Overall it would appear that most project coordinators are satisfied (somewhat or very) with the information that is available about Horizon 2020 Secure Societies Actions and Calls for Proposals, with 87% of project coordinators indicating that this is the case. A similarly high proportion (76%) also indicated that they are somewhat satisfied or very satisfied with information about the application process. During the consultation:

 One PRC, a micro-enterprise from Ireland, noted that there was a significant delay

between approval and payment.

 A large PRC from Spain noted that the Commission’s IT tools have worsened in

Horizon 2020 relative to FP7. This stakeholder suggested that the PDF-upload facility to upload proposal information to the system should be removed and replaced with a proper HTML5 interface. Another stakeholder (a HES from the Netherlands) has similarly noted that the portal and several tools were clearly not ready, or had teething problems.

 A REC from Sweden noted that the evaluation outcome seems somewhat arbitrary

and may also be dependent on efficient lobbying prior to evaluation.

When asked whether there are any other aspects of the Secure Societies project cycle that could be improved, stakeholders have noted that:

 There needs to be better understanding/recognition that some projects are led by

social science institutions, rather than technology/sciences institutions;

 There needs to be sufficient time for kick-off. Currently the time between signing a

grant and getting multiple partners to meet is unrealistically short and causes delays.

One HES from Italy has noted that several months can pass between the start of the

project and the contracting of all the required personnel, which means that few people

are working on the project at the very start. It has been suggested that the date of

starting the project should be postponed until the majority of personnel are under

contract.

 The deadline for submitting applications for Secure Societies tends to be in late

August and therefore falls within the summer holiday period. This means that the funded projects do not necessarily include the most qualified people, which is bad for EU taxpayers.

 Sometimes it would appear that the project reviewers at the Commission are lacking

the necessary technical/in-depth expertise. In some of the bigger (security) projects, there have been very few comments from the Commission, which may mean that the technical expertise are lacking.

P.5.5. Lessons learnt/Areas for improvement

The results of the online survey show that there is a high degree of satisfaction among project coordinators with most aspects of the Secure Societies project cycle. The amount of information available about Horizon 2020 Secure Societies Actions and Calls for Proposals and information about the application process were both judged to be somewhat/very satisfactory by the majority of survey participants. The amount and nature of financial support available was also judged to be somewhat/very satisfactory by more than half of the survey participants.

Areas of the project cycle that were identified as being less satisfactory by survey participants include the commercialisation of project results and the commission’s IT tools, which were both identified as somewhat/very dissatisfactory by 17% of project coordinators. Other areas of dissatisfaction include the time to grant and time allowed for submitting proposals, which were each ranked as somewhat/very dissatisfactory by 15% of respondents.

P.6. C OHERENCE

P.6.1. Internal coherence

P.6.1.1. Internal coherence of the actions implemented for SC7

Within the Secure Societies itself, the actions supported appear to be internally coherent. Most projects have a differentiated research approach, although strong similarities between projects in some calls was identified. However, some element of overlap between projects is quite common within the FPs, given that applicants responding to calls for proposals under a particular research topic are seeking to address the same research objectives. As noted above under the DS theme, even when projects with similar objectives are funded, different research teams will approach the research objectives and desired research outcomes differently.

However, a concern expressed by some interviewees was whether sufficient consideration is given to checking the coherence of funded projects between programming periods. Some project coordinators said that some projects funded in Horizon 2020 were too similar to projects funded previously under FP7 Security. However, this was a general observation and there was a lack of specific examples provided.

A study prepared by Risk & Policy Analysts explores in detail the internal coherence within

the main themes (BES, FCT, DRS, DS) for the 2014-15 work programme on pages 77-92. 185

There are a number of initiatives for coordinating projects dealing with security across different Commission services, even beyond Horizon 2020 (e.g. DG ECHO, with participation of FP7 and Horizon2020 projects in the EU Civil Protection Forum, DG DEVCO with the CBRN centres of excellence). Mechanisms for cross-fertilisation of projects e.g. via clustering of projects could be envisaged to ensure convergence of their outputs, hence increasing the effectiveness of the programme. This could also be facilitated by providing more systematically explicit references to previous call topics or projects (including non-Horizon2020) within the Work Programmes.

Joint events and final conferences for projects dealing with similar topics, joint publications of their results is also a best practice which was already used in FP7 Security Research and which should be more systematically promoted in SC7. This can significantly multiply the audience reached by one single project while sharing the costs across several projects, and demonstrate to various target groups the critical mass obtained and the high quality of EU funded projects.

P.6.2. External coherence

P.6.2.1. Coherence with other EU funding programmes

In order to assess the extent to which the Secure Societies programme is coherent with other EU interventions which have similar objectives, a structured review of the Work Programmes for 2014-15 and 2016-17 has been carried out.

185 Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk &

The results of this process are shown below in Table 235 (2014-15 Work Programme) and Table 27 (2016-17 Work Programme). Note that these tables only include the funding programmes, EU policy areas and other EU strategies/agendas for which some form of synergy was identified; many of the funding programmes, EU policy areas and EU strategies/agendas that were initially identified as potentially relevant to the Horizon 2020 Secure Societies Programme have subsequently been screened out.

It is important to note that the 2014-15 Work Programme appears to have stronger - or at least clearer - synergies with some of the Commission’s wider funding programmes, policy areas and strategies/agendas than the 2016-17 Work Programme. For instance, the 2014-15 Work Programme implies that there is a link between the Digital Security Call and Internal

Market Policy 186 , while the 2016-17 Work Programme does not. Similarly, the 2014-15

Work Programme (on page 23) explicitly references the need for proposals under the Disaster Resilience Call to “strengthen complementarity with other EU funding mechanisms, and particularly with the European Structural and Investment Funds”, while the 2016-17 programme does not.

In relation to other EU programmes where there are potential synergies with Horizon 2020 Secure Societies, the Eurostar-Eureka programme ( www.eurostars-eureka.eu/ ) is relevant. The Eurostars programme is SME-driven, and supports innovative product development, concerns promoting a fast track to market and International Cooperation. As such, from an SME perspective for SMEs engaged in security research, this programme offers potential. In addition, the potential role of the ERA-NET in strengthening research actors in the security research field in FP7 and Horizon 2020 should be emphasised.

Lastly, the new Seal of Excellence initiative 187 launched by DG RTD in conjunction with the

Member States and ACs is worth mentioning. This quality label is awarded to project proposals which were submitted for funding under Horizon 2020, passed stringent selection and award criteria but could not be funded due to budget constraints. The Seal of Excellence highlights proposals that should be funding but where there was insufficient funding available from alternative sources (public, private, national, regional, European or international). Since the initiative was launched only recently, it is too early to assess whether it has led to any unsuccessful SC7 projects being funded, for instance by national security research funding. However, its potential in future could be explored since several interviewees have attested to the problem of a high level of competition for funding and low success rates.

There are strong potential synergies in addition with other EU funding programmes designed to strengthen access to finance for innovators, should Secure Societies research projects develop technologies which require follow-on funding in order to exploit and commercialise research results, such as the InnovFin SME Venture Capital and the SME guarantee scheme and the Fast Track to Innovation Pilot scheme, all funded within the Access to Risk Finance programme within Horizon 2020. The first two instruments work through financial intermediaries, and at this early stage in Horizon 2020, it is not possible to determine how far Secure Societies participants have made use of these schemes. However, theoretically, there are synergies if project participants wish to exploit technologies/ innovations with a high TRL, then Horizon 2020 has been designed to boost the supply of finance for innovators that might not otherwise obtain funding support without public intervention due to their high risk nature.

186 In the description of the Digital Security Call on Page 94 of the 2014-15 Work Programme, it mentions that the Call will aim to promote “a Single Market for cybersecurity products” and that the proposed activities will aim to ensure “the wellfunctioning of the internal market”.

187

Table 235 - Synergies between the 2014-15 Work Programme and other European funding programmes, policy areas & strategies or agendas

The 2014-15 Work There are implied The 2014-15 Work The 2014-15 Work Programme makes an synergies between Programme contains Programme contains explicit reference to the the 2014-15 Work specific calls fostering specific topics/actions need for synergies with Programme and this synergies with this fostering synergies this programme/ programme/ programme/ with this programme/ policy/strategy policy/strategy policy/strategy policy/strategy

Funding programmes

European Structural and Investment Funds Yes Yes

Internal Security Fund Yes Yes

European Defence Agency funding Yes Yes

EU Policy areas

EU Space policy Yes Yes

EU Environment policy Yes Yes Yes

EU Energy policy Yes Yes

EU Industry, competitiveness and SMEs policy Yes

EU Internal security policy Yes Yes Yes

EU Trade policy Yes

EU Humanitarian aid and civil protection policy Yes Yes

EU Internal market policy Yes

EU Transport policy Yes Yes Yes Yes

EU Public Health policy Yes

EU Customs policy Yes Yes

EU Security industrial policy Yes Yes Yes

Other EU strategies / agendas

Europe 2020 strategy Yes Yes

Innovation union strategy Yes

Internal security strategy Yes Yes

Cyber security strategy Yes Yes Yes Yes

European agenda on security Yes Yes Yes

3 O Strategy Yes

European agenda on migration Yes

Digital single market strategy Yes

Source: Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk & Policy Analysts (RPA).

1462

Table 236 - Synergies between the 2016-17 Work Programme and other European funding programmes, policy areas and strategies or agendas

The 2016-17 Work There are implied The 2016-17 Work The 2016-17 Work Programme makes an synergies between the Programme contains Programme contains explicit reference to the 2016-17 Work specific calls fostering specific topics/actions need for synergies with Programme and this synergies with this fostering synergies with this programme/ programme/ programme/ this programme/ policy/strategy policy/strategy policy/strategy policy/strategy

EU funding programmes

European Defence Agency funding Yes

EU policy areas

EU Space policy Yes Yes

EU Environment policy Yes

EU Energy policy Yes

EU Industry, competitiveness and SMEs policy Yes Yes

EU Internal security policy Yes Yes Yes

EU Trade policy Yes Yes

EU Humanitarian aid and civil protection policy Yes

EU Transport policy Yes

EU Public Health policy Yes

EU Customs policy Yes Yes

EU Security industrial policy Yes Yes

Other EU strategies / agendas

Europe 2020 strategy Yes

Innovation union strategy Yes

Internal security strategy Yes

Cyber security strategy Yes Yes

European agenda on security Yes Yes

3 O Strategy Yes

European agenda on migration Yes

Digital single market strategy Yes

Source: Interim Evaluation of the Activities under the Secure Societies Challenge under Horizon 2020 - study prepared by Risk & Policy Analysts (RPA).

1463

P.6.2.2. Coherence with other public support initiatives at regional, national and international level

Concerning the consultation, when asked whether the Secure Societies programme is coherent with national and regional initiatives in the Member States, most stakeholders noted that there is common ground and that national and EU strategies are pretty well aligned. However, it was identified that there does not appear to be much coordination between EU and national funding programmes. For example, stakeholders noted that research projects in the UK and the EU are sometimes being duplicated, and that the same is happening between Member States (e.g. in relation to drones). It was also noted that there is some overlap between security and military research.

Stakeholders that participated in the consultation were asked whether they were aware of any international or EU level funding programmes which have similar objectives to Horizon 2020 Secure Societies. The following programmes were identified by project coordinators, beneficiaries and end-users:

 Department of Homeland Security International First Responders  Military International projects on maritime border surveillance and chemical,

biological, radiological and nuclear defence (CBRN)

 SESAR Drones  European Defence Agency studies on CBRN  Eureka Eurostars

One official from the European Commission that participated in an interview noted that in the US there are many programmes funding security research, but that it is not certain to what extent these have similar objectives to the Horizon 2020 Secure Societies programme. However, a HES from Portugal has identified that US programmes with a focus on resilience are coherent with the Horizon 2020 Secure Societies programme, even though resilience is often interpreted and addressed quite differently.

It has also been suggested that although the United Nations Crime Research Institute and Europol both do research, neither focus on science and technology.

An official from the European Commission explained that there is a NATO science programme on security – civil and military. The stakeholder explained that although this is an international programme, it is smaller and less structured but that some aspects may, nevertheless, be complementary. The official explained that NATO could possibly be a beneficiary of some future projects, so there could be connections at the project level too.

In terms of the coherence of the Secure Societies programme with other international initiatives, stakeholders noted that there are there are special challenges facing Europe – e.g. the migrant situation and border management – which may not be obvious to all countries to solve.

Stakeholders noted that while the Secure Societies programme is coherent with other EU initiatives, there needs to be better coordination with other initiatives in other DG’s who also deal with security. For example, one stakeholder gave the example of civil transport security, for which it is apparently very difficult to know which part of the Commission is responsible (e.g. DG Home, DG Move, Cyber PPP, SESAR, DG Connect, etc. are all involved). Similarly, research on drones and cybersecurity were also mentioned (by different stakeholders) as falling under the remit of multiple DGs.

Project coordinators that participated in the online survey were asked whether they are aware of any gaps between the Secure Societies programme and other international, EU and/or regional initiatives that have similar objectives. Only a small proportion of stakeholders (8 out of 78, or 12%) were aware of any gaps.

Project coordinators that participated in the online survey were asked whether they are aware of any overlaps between the Secure Societies programme and other international, EU, national and/or regional initiatives which have similar objectives. Only a relatively small proportion or respondents (6 out of 79, or 8%) were aware of any overlaps.

When asked to elaborate on where these overlaps arise, one REC from Sweden noted that there are overlaps between FP7, the European Reference Network for Critical

Infrastructure Protection (ERNCIP) and the European Programme for Critical Infrastructure Protection (EPCIP) for example.

It has also been noted that there is an overlap between national research agendas but that this is “intended and needed”. A HES from Portugal explained that some overlaps may exist between the Secure Societies programme and other initiatives in areas such as big data management and processing.

 Project coordinators that participated in the online survey were asked whether they are aware of any complementarities or synergies between the Secure Societies programme and other international, EU, national and/or regional initiatives that have similar objectives. Out of the 77 project coordinators that provided a response to this question, nine (or 12% of the total) said “yes”.

One official from the European Commission that was interviewed by the study team explained that there may be some overlap between DG Devco funding of CBRN Risk

Mitigation Centres of Excellence Initiative and some parts of the Secure Societies programme (as regards CBRN and disaster risks), but that the former focuses on training, while the latter’s focus is on research. As such, the DG Devco funding of CBRN Risk Mitigation Centres of Excellence Initiative can be viewed as complementary to Horizon

2020 Secure Societies. Similarly, it has been noted that DG Home’s Internal Security Fund (ISF) focuses within the EU on training and capacity developments for the EU’s border force and can therefore be viewed as complementary.

P.6.3. Other issues related to Coherence

The intervention logic is directly related to a support given to a large span of sectors and policies covering secure, safe and resilient society's issues in a direct or indirect way.

Figure 1 gives an illustration of the different DGs, Intergovernmental Agencies and International Organisations which are in regular contacts with the programme.

Figure 251 - The EU Policy Landscape

Source: European Commission, DG HOME B4.

Interventions related to Crisis Management policies are related to the integrated approach followed for the management of natural and man-made hazards focusing on disaster risk reduction (prevention and preparedness) and disaster response. Support is given in this

respect to the EU Civil Protection Mechanism (UCPM) 188 , and its operational dimension

coordinated by the Emergency Response Coordination Centre (ERCC). Disaster risk

management is also addressed through the EU Internal Security Strategy 189 and the resulting European Agenda on Security adopted in April 2015 190 (DG HOME) and

Consumer Health Protection policies (DG SANCO) 191 . In addition, climate-related

disasters are covered by environmental and climate policies (DG ENV, in particular the

Flood Directive 192 and DG CLIMA through the EU climate change adaptation strategy 193 ). Finally, intergovernmental agencies are also involved in security policies,

namely the European External Action Service (EEAS) – which implements the EU Common Foreign and Security Policy – and Europol – which is the EU Law Enforcement Agency. Both agencies assist EU Member States. There are also links with the Council Decision 2014/415 i/EU on the arrangements for the implementation by the Union of the solidarity clause, which covers response, situational awareness and analysis and threat assessment at Union level.

Other key EU policies concern industrial competitiveness and innovation, namely the EU

Industrial Policy 194 which aims to boost industrial competitiveness and innovation (thus

188 Decision 1313/2013

189 Internal Security Strategy for the European Union: Towards a Eropean Security Model, 5842/2/2010

190 The European Agenda on Security, COM(2015) 185 final i

191 Decision 1082/2013

192 Directive 2007/60/EC i

193 COM (2013) 216 final i

194 COM(2012) 417 final i

the access to market of developed technologies) and the EU research policy represented

by Horizon2020 195 .

With regards to CBRN-E (Chemical, Biological, Radiological, Nuclear and Explosive)

threats, support is provided to key EU policies represented by the CBRN Action Plan 196 (DG HOME) and the EU Action Plan on Enhancing the Security of Explosives 197 which

expired at the end of 2015 but for which actions are to be prolonged from 2017 onward. Support is also given to other EU policies which include CBRN as a focal point, namely in the sectors of Civil Protection and Consumer Health Protection (see above), as well as

Energy Infrastructure and Transport Networks 198 (DGs ENER and MOVE), Customs 199 (DG TAXUD), Environment and Industrial Risks 200 (DG ENV) and International

Cooperation, e.g. CBRN-E Centres of Excellence (DG DEVCO).

Complementary to EU policies, interventions are also occurring at international level in relation to policies relevant to Disaster Risk and Crisis Management. In the case of CBRN-E, various conventions exist, namely the United Nations Security Council

Resolution 1540, the Chemical Weapon Convention (CWC controlled by the Organisation for the Prohibition of Chemical Weapons, OPCW), the Biological and Toxin Weapon Convention (BTWC without control mechanisms), and the Nuclear Nonproliferation Treaty (NPT controlled by the International Atomic Energy Agency,

IAEA). In the field of Disaster Risk Management, Disaster Risk Reduction has been the core action line of the United Nations Hyogo Framework for Action on how to mitigate the impact of natural and man-made disasters, now continued by the Sendai Framework for Action setting priorities for the 2015-2025 period, among which the promotion of a better understanding of disaster risk management through the building, sharing and development of knowledge and the strengthening of the policy-science interface at local, national, regional and global levels.

The implementation of these policies represents a complex and ambitious challenge as they involve a wide variety of players whereas each Member State often follows specific national approaches (national action plans) for dealing with crises and are also differently organised in terms of disaster risk management capabilities. The EU framework represents a means and a real opportunity to discuss possible ways to improve coordination among the various national approaches and develop a common EU vision strengthened by a joint strategy in this field. The development of a Community of Users is, in this respect, an essential component to bring together key scientific, policy and industry actors, as well as other stakeholders (e.g. first responders, police representatives, fire fighters, civil protection units) around this common vision and strategy. This is

closely linked to the EU industrial policy 201 under the responsibility of DG GROW, the EU research policy 202 coordinated by DG R&I and involving DG HOME (Secure

Societies Programme), DG CNECT and JRC, the EU civil protection policy managed by DG ECHO, as well as the EU environmental and climate policies coordinated by DG

ENV and CLIMA respectively.

195 Regulation (EU) No 1291/2013 i of the European Parliament and of the Council of 11 December 2013 establishing

Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No

1982/2006/EC, L 347/104, 20.12.2013

196 COM(2009) 273 final i and COM(2014) 247 final i

197 Doc. 8109/08 of 18 April 2008

198 Regulation 347/2013 i and Decision 661/2010

199 COM(2012) 793 final i

200 Directive 2012/18 i/EU

201 COM (2010) 2020 final i

202 COM (2011) 152 final i

P.6.4. Lessons learnt/Areas for improvement

Information from desk research and consultation supports the view that the Secure Societies programme is generally coherent with other EU interventions which have similar objectives. Nevertheless, it would appear that some areas of research do fall under the remit of multiple Commission DGs (e.g. drone research, cybersecurity research) and that there is therefore a possible risk of duplication. Feedback from the

Commission indicates that during the evaluation process, consideration is given to whether project proposals are coherent, whether they are complementary and whether there would be any duplication of work.

There are several national funding programmes that provide grants for security research that may be complementary to the actions of the Secure Societies programme. While stakeholders have identified that national and EU strategies for security research are generally well aligned, it has been noted that there is insufficient coordination between EU and national funding programmes and that this can give rise to unnecessary duplication of efforts and missed benefits in terms of collaboration.

In terms of the coherence of the Secure Societies programme with other international initiatives, stakeholders noted that there are there are special challenges facing Europe – e.g. the migrant situation and border management – which may not be obvious to all countries to solve.

P.7. EU A DDED V ALUE

P.7.1. Horizon 2020 projects demonstrating EU Added Value

"SafeCloud 203 - Secure and Resilient Cloud Architecture" – Call Horizon 2020-DS-2014

SafeCloud is an Innovation Action, with a budget of EUR 3,298,987.50, including EUR 2,150,810.00 of EU contribution, which started on 1 September 2015 and will end on 31 August 2018 (36 months).

SafeCloud addresses the privacy, integrity, and security of offsite data storage and associated processing tasks that arise from the use of cloud infrastructures. This project, based on the principles of partitioning and entanglement, is re-architecting cloud infrastructures, to ensure that data transmission, storage, and processing can be partitioned in multiple administrative domains that are unlikely to collude (considering privacy by design); entangled with inter-dependencies that make it impossible for any of the domains to tamper with its integrity. After the first year, the project already produced high quality scientific and technological results, namely by developing mechanisms for secure storage, secure communications and privacy preservation, which are considered at the cutting edge of cloud computing.

SafeCloud is expected to have a high socio-economic impact, considering the wide range of economic sectors that can adopt and benefit from the project results (reinforcing the EU priority of a Digital Single Market), and also through increasing citizens’ trust in cloud computing infrastructures and applications.

203 http://www.safecloud-project.eu/

"Trillion 204 - TRusted, CItizen - LEA coILaboratIon over sOcial Networks" - Call

Horizon 2020-FCT-14-2014

Trillion is a Research and Innovation Action, with a budget of EUR 4,263,407.50, including EUR 4,263,407.50 of EU contribution, which started on 1 September 2015 and will end on 31 August 2018 (36 months).

Trillion focus on promoting a trustful collaboration between citizens and law enforcement agencies. This overarching goal is being pursuit by implementing bidirectional communication channels over social networks in order to enhance the discovery of relevant information for public security and early identification of emerging risks. The project is developing a fully-fledged platform for cooperation between citizens and LEA, including mobile applications for real-time collaboration, and trust-building mechanisms to motivate cooperation.

After one year, the project already presented some scientific and technological achievements, namely in what regards the use of a wide range of geo-location technologies, including those that enhance incident reporting in confined spaces, to enable LEAs to operate in a more efficient, content- and context-aware manner.

"ResiStand - Increasing disaster Resilience by establishing a sustainable process to support Standardisation of technologies and services" – Call Horizon 2020-DRS-2015

ResiStand is a Coordination and Support Action, with a budget of EUR 1,962,553.75, including EUR 1,962,553.75 of EU contribution, which started on 1 May 2016 and will end on 30 April 2018 (24 months).

ResiStand focus on improving the crisis management and disaster resilience capabilities of the European Union and individual Member States through standardisation. After analysing the drivers, constraints and expectations of the main stakeholder communities involved in this domain, ResiStand defined a prioritised roadmap for future standards clearly identifying the necessary standardisation deliverables to be achieved in the short and mid- to long term. The project is expected to deliver a pre-standardisation process that supports the development of standards, and facilitates collaboration between national, European and international entities dealing with disaster resilience.

P.7.2. Other issues related to EU Added Value

Most stakeholders that participated in the consultation indicated that, without EU funding, their project would not have gone ahead. It has been indicated that funding for security research at national level is decreasing and that funding from most other sources (e.g. at national level and from the private sector) is only available in relatively small amounts.

Most project coordinators that participated in the online survey have agreed that the Secure Societies programme delivers better value for money, improved research, outcomes, better trans-national and cross-sector coordination and integration of research and innovation efforts and is better placed to address externalities than Member States or regions acting alone. It is therefore unsurprising that the overwhelming majority of stakeholders believe the European Commission should continue funding for research and

204 http://trillion-project.eng.it/#Project

innovation in the area of the Secure Societies programme and that the level of funding available should either remain the same or be increased. When asked to indicate the most likely consequences of stopping or withdrawing funding for the Secure Societies programme, one stakeholder noted that this would jeopardize the sustainability of ongoing large European research and innovation projects, and would act as a disincentive to the collaboration between the various stakeholders involved in the innovation chain (including industry, academia, research organizations and end-users). The stakeholder noted that it would also have consequences in terms of industrial competitiveness, on the quality and quantity of research and it would undermine the European innovation ecosystem developed by the previous framework programs.

The importance of EU security research funding was also highlighted in discussions with other stakeholders such as discussions with those involved in managing national security research programmes. It was confirmed that only a few Member States fund a civil security research programme and that such programmes, where they exist at all, lack a transnational dimension. Since some types of security research projects necessitate a transnational approach if the research results are to be useful, this points to the ongoing strong relevance of EU funding. One end-user involved in a Conflict Prevention project viewed this international dimension as vital in ensuring, for example, that all Member States were using the same methods, curricula and standards when training their police officers, army etc. Within the European context, much peacebuilding and conflict prevention work is done by teams made up of representatives from many Member States. The interviewee therefore viewed it as strategically vital that this type of work be carried out at the European rather than the national level.

Several project coordinators and beneficiaries identified that their project would not have gone ahead without EU support and that withdrawing EU funding would mean that projects would stop. It is therefore possible to conclude that without EU funding, there would be a significant reduction in research in the security domain. One small PRC from Spain has noted, for example, that if funding for the Secure Societies programme were stopped or withdraw, “Security would not get enough attention in EU and terrorism attacks in the last few years show that this should not happen.”

The Horizon 2020 funding has market corrective value, pushing stakeholders to work together and engaging stakeholders (e.g. end-users and practitioners) that might not otherwise be involved. For example, one Commission official that participated in the consultation explained that the police are not naturally engaged in research on the technologies they use, but that the Secure Societies programme helps in this respect. Multiple stakeholders have identified that stopping EU funding would lead to a fragmentation of the EU security market and a lack of coordination and cooperation between EU actors, as illustrated by the following quotes from the online survey:

“Several global activities in the area of cooperation between different stakeholders will be stopped or reduced”

“I would hamper the trans-national research and innovation efforts.”

“reduced cooperation between organisations who have participated in the projects”

“Lack of scientific study on one of the main issue of our time (security) and lack of interactions between universities and stakeholders”

“fragmentation of EU security market”

“Lack of coordination among EU actors”

Stakeholders have identified that stopping EU funding would mean that Europe would fall behind in terms of protecting its critical infrastructure and its citizens. It has also been suggested that the competitiveness of the European security sector would be harmed and the EU might become a less prominent player on the international market. For example, it was suggested that companies would go to the USA and Asia to find funding and cooperation within the industry.

Finally, it has been noted that there would be fewer job opportunities for young researchers and experts working in the field of security.

Consultees were asked to provide examples of technologies, products, data or other outputs from Horizon 2020 Secure Societies research that would not have been developed without EU support. One stakeholder noted that the following would have been developed to a lesser degree (or not at all in some cases) without support from the Horizon 2020 Secure Societies programme:

 Sensors for maritime security  Mission scenario builders/simulators for disaster management  Systems and protocols for secure mobile radio communications  Secure and resilient software architectures  Decision support systems  Systems of cyber intelligence and support to investigations

It was noted however that it is still too early to give many examples of outputs from Horizon 2020 studies as most have only started very recently.

In order to verify the online survey findings, other research methods have been used, including secondary research to assess recent survey findings, and interviews with participants in Horizon 2020 Secure Societies.

A major online survey was undertaken in June-August 2016 as part of a study to assess the Union added value of Horizon 2020 by PPMI, on behalf of DG RTD. The survey elicited 963 responses, of which 31 responses were from participants in Secure Societies (from a total of 96 projects identified in CORDIS). Among the overall findings across Horizon 2020 were that:

 Horizon 2020 funding has been a key factor enabling participants in European

research projects to attract greater financial resources than would have otherwise been available through national and regional R&I programmes. Compared with the EU-15, scope and scale effects were slightly higher among projects

coordinated by beneficiaries from the EU-13.

 In terms of added value compared with national/regional R&I programmes,

Horizon 2020 has increased research capacities, increased the quality of the research, promoted valuable scientific and innovation outputs and results, and in

some cases, led to commercial exploitation.

 Horizon 2020 has a clear additionality and added value by supporting many

research projects that would not have otherwise been implemented or would have suffered substantial modifications, in scope and/or in terms of the expertise

available to contribute to the research.

In relation to the specific findings in respect of Secure Societies, 55.3% of respondents stated that the project would not have gone ahead, whilst 33.4% said that the project would have gone ahead with significant modifications. Of those indicating that the project would have gone ahead, ten respondents provided further detail as to why this was the case. Eight respondents stated that the scope of the project would have changed (i.e. the scope, number of areas addressed, or the ambition of the objectives), ten respondents stated that the timeframe for project implementation would have changed and six indicated that the composition of the consortium in terms of partners would have changed.

A relatively high proportion of project coordinators under Secure Societies stated that their research capacities would have decreased if their project had been funded by national/regional funds instead of Horizon 2020. For instance, 59.1% stated that this would be the case relating to "Understanding and knowledge in existing areas" and 58.7% “Understanding and knowledge in new areas”. EU funding was seen as less crucial for participants in SC7 in relation to "Access to infrastructure and equipment" (since less than half - 48.7% - thought that capacities would have decreased without EU funding).

In relation to human resources, had they received national/regional funding instead of Horizon 2020 funding, there were interesting findings in relation to the percentage of respondents that thought their capacities would have decreased. For instance, 76.5% stated that their relationships and networks with other partners would have decreased (although this was lower than for most other societal challenges), and 56.3% stated that not obtaining EU funding would have "reduced their ability or capacity to attract researchers and other staff". However, the mobility of researchers was seen as less impacted by whether or not a participant received Horizon 2020 or national / regional funding, since only 42.7% thought this would have decreased their human resource capacity.

The responses in relation to the "Share of the project and consortium partners to whom their commercial advantage would have decreased had they received national/regional instead of Horizon 2020 funding" are also interesting from a comparative point of view between societal challenges. For instance, under secure societies, 65.3% thought that their competitive position internationally would have been affected whereas for some of the other societal challenges this was as high as 84.3% (SC5). With regard to the same question but relating to "maintaining market share in existing markets" 38.7% of respondents (SC7) thought that this would have had a negative impact compared with 52.7% (SC3) and 63.5% (SC5). Whilst variations between societal challenges may shed light on the perceived added value of projects funded under each SC, it should be recalled that the responses will inherently be determined by what level of research and innovation funding is available through national and regional funding. For security research, several Member States have well-funded national civil security programmes (e.g. France, Germany), so there are alternative sources of financing available especially among larger EU countries and in Western Europe. This may not be the case across all societal challenges therefore the variations in responses between SCs should be interpreted in this context.

P.7.3. Lessons learnt/Areas for improvement

On the basis of the information gathering during the consultation, it can be concluded that without EU funding there would be a significant reduction in research in the security domain. It has been indicated that funding for security research at national level is decreasing and that funding from most other sources (e.g. at national level and from the private sector) is only available in relatively small amounts. Unsurprisingly, therefore, the overwhelming majority of stakeholders have indicated that the European Commission should continue funding for research and innovation in the area of the Secure Societies programme and that the level of funding available should either remain the same or be increased.

Multiple stakeholders have noted that stopping EU funding would lead to a fragmentation of the EU security market and a lack of coordination and cooperation between EU actors. It has also been suggested that the competitiveness of the European security sector would be harmed, that there would be fewer job opportunities for young researchers and experts working in the field of security and that the EU might become a less prominent player on the international market. Perhaps most importantly, however, stakeholders have identified that stopping EU funding would mean that Europe would fall behind in terms of protecting its critical infrastructure and its citizens.

The threats facing the EU are not only facing a single Member State, and the Secure Societies programme plays an important role in protecting the European community as a whole. Concerns about sharing information and data mean that stakeholders can be hesitant to collaborate on sensitive issues and it has been noted that, in this regard, the

Secure Societies programme delivers added value. However, it would appear that this is not the only way in which the Secure Societies programme has delivered benefits over and above what could be achieved at Member State or regional level alone. During the online survey, more than two thirds of project coordinators indicated that the Secure Societies programme has delivered better value for money, improved research outcomes and better trans-national and cross-sector coordination and integration of research and innovation efforts compared to what could have been achieved at the Member State or regional level alone.

P.8. S UCCESS STORIES FROM PREVIOUS F RAMEWORK P ROGRAMMES

Among the many success stories from FP7 finished projects, the following three have been selected as having been particularly successful in protecting freedom and security of Europe and its citizens, in inducing socio-economic impacts and in providing EU Added Value:

"MOBILEPASS 205 - A secure, modular and distributed mobile border control solution for

European land border crossing points" – Call FP7-SEC-2013-1

MOBILEPASS is a Collaborative project (Small or medium-scale focused research project), with a budget of EUR 4,133,598.49, including EUR 3,141,321.50 of EU contribution, which started on 1 May 2014 and ended on 31 December 2016 (32 months).

MOBILEPASS focused on creating mobile equipment (based on biometric technologies) for border control authorities. The equipment developed enables authorities to perform contactless fingerprint acquisition, encompassing the whole chain from fingerprint data obtained from passports up to contactless verification. This innovative solution also has significant added value, as the border checks can be executed in a more comfortable, fast and secure way. This is particularly pertinent in the view of the proposed European

205 http://mobilepass-project.eu/

Travel Information and Authorisation System (ETIAS) and the recent agreement between the European Parliament and the Council on the Commission's proposal to introduce mandatory systematic checks of all travellers, including EU citizens, against relevant databases when crossing the EU's external borders.

Furthermore, the project was active on standardisation within the CEN TC 224 WG18 (Personal identification, electronic signature and cards and their related systems and operations) and one of the activities of this WG was initiated by MobilePass (on Personal identification – Border and Law enforcement identity check - Minimum Information Display).

"SAVEMED 206 - Microstructure secured and self-verifying medicines" – Call FP7-SEC-

2010-1

SAVEMED is a Collaborative project (generic), with a budget of EUR 4,278,114.80, including EUR 3,144,724.50 of EU contribution, which started on 1 April 2011 and ended on 31 March 2014 (36 months).

SAVEMED focused on developing anti-counterfeiting and track & trace technologies to prevent the illegal re-import and re-packing of medicines (pharmaceutical tablets, injection moulded pharma caps and laminated sterile pouches). The technologies developed allow protecting, tracking and trace data on the pills, ensuring that the medicine's content always match the packaging. Moreover, their replication by criminal organisations cannot be done in an easy, quick and economic way.

While developing these innovative solutions, the consortium considered the traditional production processes used by the pharmaceutical industry in order to ensure an easier adoption by the industry. Furthermore, the consortium had conversations with different pharmaceutical companies and European governments (e.g. Romania) to implement some of the proposed solutions and adopt some of the recommendations delivered by the project.

"ICARUS 207 - Integrated Components for Assisted Rescue and Unmanned Search

operations" – Call FP7-SEC-2011-1

ICARUS is a Collaborative project (Large-scale Integrating Project), with a budget of EUR 17,306,992.64, including EUR 12,584,933.45 of EU contribution, which started on 1 February 2012 and ended on 31 January 2016 (48 months).

ICARUS had as overarching goal to reduce the total cost of a major crisis event. This was accomplished by developing a comprehensive and integrated set of unmanned search and rescue (SAR) tools that can equip first responders and support crisis managers in their mission.

Based on existing robotic systems, ICARUS adapted three aerial robotic systems, two ground robots and three types of marine vehicles that operate in a coordinated manner for SAR operations. The vehicles are equipped with human detection sensors and the information retrieved by the robots feeds a command and control infrastructure which interfaces with a command and control system (GDACS) already used by the end-users. Moreover, the project demonstrated the value added of using complementary types of

206 https://www.savemed.org/

207 http://www.fp7-icarus.eu/

unmanned vehicles, as a mean to reduce operational costs and potential loss of human lives.

A cornerstone of the project was the strong involvement of the end-users since its inception. This significantly enhanced the outputs of the project, which present a high technical progress beyond the state of the art, validated in large scale integrated demonstrations (land and sea).

Some of the technologies are already being commercialized, such as the power-saving stereo camera sensor installed in one of the UAV that enables 3D reconstruction in realtime and in indoor conditions. The achievements of the project were acknowledged in high level fora, such as the UN World Conference on Disaster Risk Reduction and the

EU Civil Protection Forum.

P.9. L ESSONS LEARNT /C ONCLUSIONS

P.9.1. Relevance

It is clear that the original objectives of the Secure Societies programme still correspond to the needs and problems facing the EU, with this being confirmed through both the literature review and the consultation. The concerns addressed by the Secure Societies programme appear to align well with the concerns of EU stakeholders (emergency services, public and private security services, operators of / companies with critical infrastructure, disaster relief and crisis management organisations, policymakers and regulators, EU research community, EU industry, EU SMEs) and citizens and, by and large, it is anticipated that Secure Societies projects will provide benefits to these various stakeholder groups.

Most stakeholders that participated in the consultation indicated that, without EU funding, their project would not have gone ahead. It has been indicated that funding for security research at national level is decreasing and that funding from most other sources (e.g. at national level and from the private sector) is only available in relatively small amounts.

P.9.2. Effectiveness

A barrier to the full effectiveness of the programme lies in the significant differences in terms of success rates across the various call topics, which may cause deviations between the expected and the actual topic coverage and therefore lead to an unbalanced project portfolio, where some areas of research may be over represented and others neglected.

For instance some topics were very popular and attracted a higher number of proposals than expected with respect to the programme size. For instance in the same call 'Disaster Resilient Societies', the topics DRS-07-2014 and DRS-01-2015 received massive numbers of proposals, leading to respectively to 5 and 3 grants, whereas some other topics did not attract applicants and led to 'orphan topics', without any proposal retained for funding, e.g. DRS-5-2014 and DRS-15-2015. The topic DRS-5-2014 had to be rewritten and reintroduced in the Work Programme 2016-2017, causing a two year delay in addressing a critical need, namely paving the way for a pre-commercial procurement for situation awareness system for civil protection authorities in the EU.

Such a situation was particularly acute for the Work Programme 2014-2015 and has been mitigated in the Work Programme 2016-2017, with the introduction of dedicated budgets for a majority of topics.

As a consequence of the above, there are parallel grants which are funded under the same topic and are expected to deliver similar results, in line with their respective topic requirements. Whereas complementary approaches can be beneficial and respond to a variety of stakeholders needs, the competition between projects to reach out to the main stakeholders e.g. at the EU level can be largely counterproductive and have an adverse impact.

The lack of clearly defined and contractually binding mechanism to make sure such "parallel" grants are coordinating their effort could be perceived as a barrier to ensure full effectiveness of the programme.

Actions to take advantage of synergies between projects can be implemented, but remain too dependent on the good will of the project coordinators and the project officers, and represent an additional project monitoring task which is not acknowledged in the current state of play.

Projects are very often expressing the need for an ad-hoc post project support, when successful results are being delivered. There is currently no support mechanism which can directly be granted to projects which have achieved a very high TRL or very promising results in order to bring them to actual exploitation and implementation.

In regards to consultation, most stakeholders have agreed that the Secure Societies programme promotes excellence in scientific and technological research and that it also plays an important role in supporting “open science” in the field of security.

Nevertheless, it would appear that the Horizon 2020 Secure Societies programme has, to date, produced relatively few publications and intellectual property applications – although, to some extent, this is not unexpected given the limited amount of time that has elapsed since the beginning of the programme.

Low success rates for proposals are a key source of dissatisfaction among stakeholders and have a disincentive effect on the research and technology community. Where proposals are successful, it would appear that the amount of funding allocated to projects is generally satisfactory to achieve the projects’ objectives; however, a large share of project coordinators (35%) have also indicated that better research outcomes could be achieved with a larger budget.

To date, over EUR 45 million in funding has been leveraged alongside the EUR 399 million contributed by the European Commission. Most (88%) has come from the private sector.

The analysis of the CORDA data shows that the average time to grant for the Horizon 2020 Secure Societies programme has improved considerably since FP7 and is now just

213 days across Secure Societies as a whole. This compares to an average time to grant of 546 days for FP7 projects that are still ongoing or that have finished since the 31 December 2014. Although only 58% of Horizon 2020 Secure Societies projects have achieved the Commission’s target of 240 days, 90% of Horizon 2020 Secure Societies projects have managed to achieve a time to grant of 250 days or less. It has been noted during the consultation that delays in receiving funds from the Commission may be problematic for SMEs.

Most project coordinators (75%) have indicated that their project will achieve its aims in full. Most (90%) have also indicated that end-users are somewhat to very likely to use the research results/outputs. There are, nevertheless, some issues that have been identified as potentially posing a barrier to greater uptake by end-users, with the most prominent including: confidentiality arrangements, resource/financial constraints, the need for further research/development and a lack of knowledge/trust among end-users. It has been identified that one of the main gaps in the programme is that, at the end of the project, there is insufficient financial, technical, policy and project support to implement project results and take innovations to market.

It has been identified that global competition in the security industry is very strong, mainly emanating from the US and China. Most project coordinators that participated in the online survey agreed that the programme is helping to position Europe on the global map of science and technology in the field of security research and that it has contributed to strengthening the competitive position of the European security industry.

Most also agreed (strongly or somewhat) that the Secure Societies programme has contributed to the diffusion of innovation in the European economy (63%) and to better trans-national and cross-sector coordination and integration of research and innovation efforts (56%).

Two thirds of project coordinators that responded to the online survey have agreed (either strongly or somewhat) that the Secure Societies programme has contributed to increasing the security of Europe’s citizens.

Approximately half of the projects with signed grants under Horizon 2020 Secure Societies have a female coordinator, which would suggest that the programme is performing well in terms of gender equality. However, separate data on the gender of the

LEAR contact for each of the signed grants paint a slightly different picture. Where available, these data show that 68% of the LEAR contacts for signed grants are male and

32% are female. Although the proportion of female LEAR contacts has increased since FP7, where the vast majority (83%) of LEAR contacts for participations in Security projects were male (17% female), the balance between males and females under Horizon 2020 is not yet even. CORDA data show that 14.9% of the grants signed under Horizon 2020 Secure Societies have taken into account the gender dimension.

Stakeholders have explained that submitting a proposal is very difficult and that, although the participant portal has improved access to information for new players, it is still an expert task to know how to bid. It has been identified that there tend to be established consortia and that this can make it difficult for newcomers to participate. Identifying additional sources of funding has also been identified as a source of difficulty by a large proportion (28%) of project coordinators.

P.9.3. Efficiency

The results of the online survey show that there is a high degree of satisfaction among project coordinators with most aspects of the Secure Societies project cycle. The amount of information available about Horizon 2020 Secure Societies Actions and Calls for Proposals and information about the application process were both judged to be somewhat/very satisfactory by the majority of survey participants. The amount and nature of financial support available was also judged to be somewhat/very satisfactory by more than half of the survey participants.

Areas of the project cycle that were identified as being less satisfactory by survey participants include the commercialisation of project results and the commission’s IT tools, which were both identified as somewhat/very dissatisfactory by 17% of project coordinators. Other areas of dissatisfaction include the time to grant and time allowed for submitting proposals, which were each ranked as somewhat/very dissatisfactory by 15% of respondents.

P.9.4. Coherence

Information from desk research and consultation supports the view that the Secure Societies programme is generally coherent with other EU interventions which have similar objectives. Nevertheless, it would appear that some areas of research do fall under the remit of multiple Commission DGs (e.g. drone research, cybersecurity research) and that there is therefore a possible risk of duplication. Feedback from the Commission indicates that during the evaluation process, consideration is given to whether project proposals are coherent, whether they are complementary and whether there would be any duplication of work.

There are several national funding programmes that provide grants for security research that may be complementary to the actions of the Secure Societies programme. While stakeholders have identified that national and EU strategies for security research are generally well aligned, it has been noted that there is insufficient coordination between EU and national funding programmes and that this can give rise to unnecessary duplication of efforts and missed benefits in terms of collaboration.

In terms of the coherence of the Secure Societies programme with other international initiatives, stakeholders noted that there are there are special challenges facing Europe – e.g. the migrant situation and border management – which may not be obvious to all countries to solve.

P.9.5. EU Added Value

On the basis of the information gathering during the consultation, it can be concluded that without EU funding there would be a significant reduction in research in the security domain. It has been indicated that funding for security research at national level is decreasing and that funding from most other sources (e.g. at national level and from the private sector) is only available in relatively small amounts. Unsurprisingly, therefore, the overwhelming majority of stakeholders have indicated that the European Commission should continue funding for research and innovation in the area of the Secure Societies programme and that the level of funding available should either remain the same or be increased.

Multiple stakeholders have noted that stopping EU funding would lead to a fragmentation of the EU security market and a lack of coordination and cooperation between EU actors. It has also been suggested that the competitiveness of the European security sector would be harmed, that there would be fewer job opportunities for young researchers and experts working in the field of security and that the EU might become a less prominent player on the international market. Perhaps most importantly, however, stakeholders have identified that stopping EU funding would mean that Europe would fall behind in terms of protecting its critical infrastructure and its citizens.

The threats facing the EU are not only facing a single Member State, and the Secure Societies programme plays an important role in protecting the European community as a whole. Concerns about sharing information and data mean that stakeholders can be hesitant to collaborate on sensitive issues and it has been noted that, in this regard, the

Secure Societies programme delivers added value. However, it would appear that this is not the only way in which the Secure Societies programme has delivered benefits over and above what could be achieved at Member State or regional level alone. During the online survey, more than two thirds of project coordinators indicated that the Secure Societies programme has delivered better value for money, improved research outcomes and better trans-national and cross-sector coordination and integration of research and innovation efforts compared to what could have been achieved at the Member State or regional level alone.

• Q. 

  S PREADING EXCELLENCE AND WIDENING PARTICIPATION

  Q.1. I NTRODUCTION

  Q.1.1. Context

Despite efforts at national and European level, disparities in terms of research and innovation performance persist among EU Member States. Especially in advanced economies like Europe's, scaling up and improving investment in research and innovation is an essential pathway to economic growth and competitiveness. Increasing the R&I performance of low performing Member States and integrating their unexploited potential into the European Research Area and single market will maximise the impact of

R&I investment, for Europe as a whole and for each Member State concerned.

According to an analysis by the Commission 208 , some of the main causes of low

participation to EU Framework Programmes of certain countries were:

 Insufficient R&D investments in those countries  Lack of synergies between certain countries’ national research systems and EU

research

 Lagging system learning effects and access to existing networks  Differential wage levels between countries  Insufficient and ineffective information, communication advice and training

Additionally, the High Level Expert Group on the Ex-post evaluation of FP7 concurred that "some of most important reasons for the comparably lower share and lower success

rates of the EU ‐13 organisations are information and language barriers; lack of

professional contacts and research networks; lack of leading Universities and Research organisations leaders in proposal matters; limited understanding of FP7; weak training in preparing successful proposals; insufficient motivation to participate in FP7; lack of

practice in project management; little experience in cross ‐country cooperation;

generally low focus on R&D in policy and in business; few options for exploitation of research results at the national level."

In order to address the above challenges of R&D investments, efficiency of national research and innovation systems and networking, Horizon 2020 introduces specific measures for spreading excellence and widening participation through engaging organisations of those countries which could commit more towards the EU research and innovation effort. The Horizon 2020 legal texts refer to "low performing RDI Member States and regions". Based on the will of the legislator it has been decided that for the specific objective of Spreading Excellence and Widening Participation (that is for

Teaming, Twinning and ERA Chairs actions) specific eligibility conditions 209 apply to

ensure a targeted approach toward Member States and Associated Countries with low

208 Commission analysis of September 2011, at the request of the Polish Presidency, see

http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%2014728%202011%20INIT This has been confirmed by other studies, analysis and public discussions, for instance the FP7 MIRRIS project http://www.mirris.eu/ .

209 Based on the criterion of the composite indicator of research excellence (capped at the 70% of the EU average). A

decision was taken to consider the results of the composite indicators as fixed for the entire Horizon 2020 period.

performance in research and innovation. The "Widening" Member States 210 currently

eligible for support using the above mentioned criterion are those who joined the EU after 2004 (Bulgaria, Croatia, Cyprus, Czech Republic, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Romania, Slovakia and Slovenia) plus Portugal and Luxembourg. This implies that for the above three main actions, only entities from the Widening countries specified above, are allowed to submit (in the case of ERA chairs) or coordinate (Teaming, Twinning) a project proposal. Additionally, internationally leading institutions can participate to the programme as partner.

This document presents an interim evaluation of the SEWP programme as a whole, based on multiple and separate assessments of the actions implemented until now. Twinning and ERA Chairs' approved projects have undergone an interim evaluation study performed by an external contractor (IETEC 2016) with the aim of providing DG RTD with results focused on the progress and achievements of the projects in the current stage of implementation according to the five evaluation criteria of relevance, effectiveness, efficiency, coherence, and EU added value. The methodology applied in this assessment is based on three sources: Desk research, a questionnaire survey and qualitative interviews. For Teaming, as the action is still in its first implementation phase, with only

Phase 1 projects funded until now, an internal assessment by DG RTD has been performed based on a short survey sent to 31 project coordinators (29 responses) and feedback from the Commission project officers based also on the project reviews. An internal assessment has been performed for the Horizon 2020 Policy Support Facility and for the COST programme. For the latter the programme midterm review will take place in 2017.

A study supporting the interim evaluation has collected information and data and provided evidence on the development of synergies at the level of strategies, programmes and project implementation. A major study funded by DG REGIO is also on-going on the co-ordination and harmonisation of ESI Funds, including with other EU policies.

Q.1.2. Objectives and intervention logic

The specific objectives of «Spreading excellence and widening participation» (SEWP)

set out in Part IV contributing to the Horizon 2020 general objectives 211 are to "to fully

exploit the potential of Europe's talent pool and to ensure that the benefits of an innovation-led economy are both maximised and widely distributed across the Union in accordance with the principle of excellence".

The specific objectives of SWEP are thus to unlock excellence in low-performing RDI regions and Member States and associated countries; to widen participation of these countries in Horizon 2020 and to contribute to the achievement of the European Research Area. Therefore, it supports actions aimed at strengthening the institutional, scientific and networking capacities of centres of excellence located in low performing regions and

210 The following Associated Countries (subject to valid association agreements with Horizon 2020) are also eligible

for support: Albania, Armenia, Bosnia and Herzegovina, Faroe Islands, Georgia, Former Yugoslav Republic of Macedonia, Moldova, Montenegro, Serbia, Tunisia, Turkey and Ukraine.

211 Horizon 2020 overall objective is to « build a society and an economy based on knowledge and innovation across

the Union by leveraging additional research, development and innovation funding and by contributing to attaining research and development targets, including the target of 3 % of GDP for research and development across the Union by 2020. It shall thereby support the implementation of the Europe 2020 strategy and other Union policies, as well as the achievement and functioning of the European Research Area (ERA). ».

Member States, on the basis of partnerships with internationally leading institutions and researchers.

In a complementary way, synergies with the European Structural and Investment (ESIF) Funds are sought for, firstly to ensure the sustainable integration of the beneficiary institutions into the national research landscapes, secondly to increase impact and quality of investments in low performing countries and regions in terms of R&I.

The lines of activities of «Spreading excellence and widening participation » are those which «shall help close the research and innovation divide in Europe by promoting synergies with the European Structural and Investment Funds (ESI Funds) and also by specific measures to unlock excellence in low performing research, development and innovation (RDI) regions, thereby widening participation in Horizon 2020 and contributing to the realisation of the ERA».

The three core Widening actions are the following:

 Twinning that aims towards significantly strengthening a defined field of research

in an emerging institution in a less R&D performing Member State through linking this institution with at least two internationally-leading counterparts in

Europe. Activities like short term staff exchanges, expert visits and short-term onsite or virtual training; workshops; conference attendance; dissemination and outreach are supported.

 ERA Chairs aims to bring outstanding researchers to universities and other

research institutions that have high potential for research excellence. On their side, institutions are expected to mobilise support from different funding sources, including from the ESIF, to invest in facilities and infrastructures and commit to

institutional change and a broader support to innovation.

 Teaming that focuses on the creation of new or updating of existing centres of

excellence in less R&D performing Member States through a "teaming" process with an advanced institute. The programme aims to develop in two steps, where in a first step funding is provided to develop a business plan for the future centre and in a second step the most successful first step proposals compete for further financial support for the first stages of the implementation phase of the future

centre.

All Widening actions are bottom-up and therefore all topics are eligible, the only condition required by the work programme is a broad alignment with the national/regional smart specialisation strategy (a requirement for 'Teaming' and a desirable asset for 'Twinning' and 'ERA Chairs').

Additionally, other Widening related actions included in SEWP work programmes 2014-2015 and 2016-2017 and complementary to Teaming, Twinning and ERA Chairs, aim to contribute in various ways to increase the efficiency and improve performance of national research and innovation ecosystems in Widening countries, tackling issues which are particularly relevant for these countries such as information on EU Widening actions (NCP Wide Net), structural R&I reforms (through the PSF), support to international networking (through COST) and dissemination of innovative solutions and new governance concepts to support urban transformation (JPI Urban Europe). Specifically:

 The Commission supports a dedicated network of Widening National Contact

Points (NCPs) through the NCP WIDE_NET project. This project promotes activities mainly related to the SEWP calls (Teaming, Twinning and ERA Chairs) in the Member States and builds the skills of the NCPs through the organisation

of brokerage events, workshops and conferences.

 The Horizon 2020 Policy Support Facility (PSF) provides tailor-made services at

the request of Member States and countries associated to Horizon 2020, aiming to support them in the design, implementation and evaluation of research and innovation (R&I) policy reforms. PSF in the context of SEWP aims at better informing the formulation and optimisation of research and innovation policies in low performing Member States and regions aiming to make them more competitive at European level by offering expert advice to public authorities. The PSF was launched in March 2015. The Horizon 2020 legal base foresaw funding of up to EUR 20 million from the "Spreading excellence and widening participation" and "Europe in a changing world - Challenge 6" parts of Horizon

2020.

 COST (Cooperation in Science and Technology) is a European intergovernmental

framework to support the networking of nationally funded research activities. It provides means to jointly develop ideas and new initiatives across all fields in

science and technology, including social sciences and humanities, through pan European networking of nationally funded research. Particular emphasis is placed on the COST “inclusiveness” countries ensuring the aim of widening participation. Additionally, researchers from Near Neighbour Countries (i.e. the non-COST members Albania, Algeria, Armenia, Azerbaijan, Belarus, Egypt, Georgia, Jordan, Lebanon, Libya, Moldova, Morocco, the Palestinian Authority, Russia, Syria, Tunisia and Ukraine) and International Partner Countries can also participate in a COST Action on the basis of ascertained mutual benefit.

 The JPI Urban Europe 212 is a transnational research and innovation initiative

with the following goals: to anticipate future urban characteristics and find innovative solutions; provide R&I input for evidence-based policies; develop and demonstrate new technologies; investigate new governance concepts and support urban transformation processes. The initiative addresses the complexity and cross-sectorial interdependence of urban development and it hence promotes

inter- and trans-disciplinarity.

Additionally, SEWP is the Horizon 2020 part particularly favourable to maximising synergies between Horizon 2020 and the European Structural and Investment Funds (ESIF) as, in its own nature; it includes requirements that encourage them. The purpose of developing synergies between Horizon 2020 and ESIF is to ensure that research and innovation investments of Horizon 2020 and investments in related fields of ESIF together lead to more, better and durable impacts on innovation, competitiveness, jobs and growth in line with Europe 2020 objectives. The development of synergies has been gaining strength in the current programming period 2014-2020 as both Horizon 2020 and

the Common Provisions Regulation of ESIF 213 include for the first time a legal mandate

to maximise synergies, not only between these two instruments, but also with other programmes, such as COSME, Erasmus+ and Connecting Europe Facility. Regarding the

212 In November 2016, the JPI Urban Europe consists of 13 full member countries: Austria, Belgium, Cyprus,

Denmark, Finland, France, Germany, Italy, Netherlands, Norway, Slovenia, Sweden and the United Kingdom. The observers are Latvia, Poland, Portugal, Romania, Spain and Turkey. The JPI has also established specific cooperation relations with non-European countries such as Argentina, Australia, Brazil, China, Japan, Mexico, Qatar, South

Africa, Switzerland and the USA.

213 The 'Common Strategic Framework' (annex 1 of the Common Provisions Regulation of ESIF) sets out "the

strategic guiding principles in order to achieve an integrated development approach using the ESI Funds coordinated with other Union instruments and policies, in line with the policy objectives and headline targets of the Union strategy for smart, sustainable and inclusive growth and, where appropriate, the flagship initiatives, while taking into account the key territorial challenges and specific national, regional and local contexts".

research and innovation-related funding volume under ESIF, Thematic Objective 1 -

'Research & Innovation' amounts to EUR 43.7 billion 214 . This will in particular focus on

business innovation support and improvement of national and regional innovation ecosystems.

Maximizing synergies is also part of the mandates of two Commissioners, Carlos Moedas, Commissioner for Science, Research and Innovation, and Corina Creţu, Commissioner for Regional Policy. DG RTD and DG REGIO have implemented a number of activities that aim at fulfilling this policy ambition. For instance, the promotion of synergies includes as a policy action the Seal of Excellence initiative that is a practical manifestation of synergies. It is a quality label awarded by the European Commission to excellent research and innovation project proposals that were submitted and positively evaluated under the SME Instrument calls of Horizon 2020, but could not be funded under available call budgets. The Seal of Excellence initiative was officially launched by Commissioners Moedas and Creţu on 12 October 2015, responding to the specific mandate they both received from President Juncker, "to maximise synergies between different funds" in order to enhance competitiveness, jobs and growth. The purpose of the Seal of Excellence is twofold: to help SMEs in obtaining alternative funding for their project proposals and, at the same time, provide the alternative funding bodies interested in investing in R&I with information on the 'ready-to-fund' high-quality project proposals, already evaluated by Horizon 2020. Recently, Commissioner Moedas announced that the Seal will be expanded to other Horizon 2020 mono-beneficiary grants such as Teaming, ERC and Marie Sklodowska-Curie Actions.

The issue of synergies (including the Seal of Excellence) will be dealt with under the chapter Q.6.2 on External coherence with other EU funding programmes.

The comparison of the objectives of the Specific Programme «Spreading excellence and widening participation» with operational objectives of the corresponding programme in the Seventh Framework Programme for Research and Technological Development (FP7), Research Potential (REGPOT), shows a relatively low degree of continuity. In addition while the eligible participants in REGPOT were the Convergence regions, the SWEP in Horizon has moved the focus to Widening countries.

To address the identified objectives, the programme has adopted the intervention logic presented in Figure 252.

214 €41.1 billion under the European Regional Development Fund (ERDF) and €2.6 billion under the European

Agricultural Fund for Rural Development (EAFRD)

Figure 252 - Intervention logic of Spreading Excellence and Widening Participation – Horizon 2020

Source: DG RTD B5.

The intervention logic chart shows how the specific SEWP objective entails several types of actions (Teaming, Twinning, ERA, Horizon 2020 Policy Support Facility, COST, the network of Widening National Contact Points (NCPs)), which while heterogeneous in their nature, all support highly complementary activities. The activities listed are expected to contribute to the four main clusters of expected outputs, 1) Creation of scientific output (increased research excellence, institutional changes within research organisations to implement the European Research Area priorities, improved capacity to compete for competitive funding, mobile researchers both trans-national and cross-sector, world-level researcher attracted to EU; 2) Partnerships & international openness (connected resource capacities, scientific collaboration across disciplines on new, high-risk ideas, cross-country (also beyond EU) and cross-disciplinary research and innovation networks strengthened; 3) Policy outputs (supported policy making, reforms, policy dialogues, strategic planning and implementation of policy and programmes at regional and national level and 4) Well-functioning R&I ecosystem ( innovative R&I practices and mutual learning taking place in regions and MS across the whole R&I ecosystem , improved quality of information, advice, and training, sharing of research agendas supported ). Strengthening synergies of Horizon 2020 with ESIF and with the Instrument of Pre-Accession (IPA II) for Candidate Countries is expected to play a significant role in contributing to improve strategic planning and therefore to increase the impact of investments in low R&I performing countries.

The expected results are likely to materialise sometime after the end of the projects. Their level of achievement depends on the use/adoption of project outputs by partners and stakeholders involved. For SEWP these include: a strengthened R&I capacity and excellence, combined with adequate framework conditions, and a strengthened international positioning that should contribute to generate jobs growth and competitiveness of participating entities.

The expected impacts include: (a) World-class excellence in science in Widening countries; b) Better trans-national and cross-sector collaboration of R&I efforts; (c) Improved capability to compete for international research funding; (d). Socio-economic cohesion of Europe; (e) More effective national and regional research ecosystems; (f) Regional, national and EU policy frameworks and programmes, more research, innovation and SME supportive. This latter impact is sought and supported also by the

ESIF and for candidate countries by the IPA programme.

Q.2. I MPLEMENTATION STATE OF PLAY

Q.2.1. Overview of programme inputs and activities

The three core actions of Widening are designed by DG RTD and implemented by the Research Executive Agency (REA) which is in charge of evaluation of proposals, grant preparation, and project monitoring. The implementation and project follow-up of ERA Chair's pilot projects funded under FP7 is instead under the Commission's direct responsibility (DG.RTD). The other Widening actions NCP Wide-net, JPI and COST are managed by dedicated organisations. PSF is managed by the Commission and some services are outsourced.

The Teaming, Twinning and ERA Chairs budget has been allocated through 4 closed calls for proposals. A brief summary table of the results can be found below.

Table 237 - Activities and allocated share of budget dedicated to Spreading Excellence and Widening Participation for the programming period 2014-2017

Activities in the legal base Allocated share of budget (in EUR million)

Calls

2014: Teaming Phase 1 2.88% (11.85)

2014: ERA Chairs 8.17% (33.60)

2015: Twinning 16.10% (66.24)

2016-2017: Teaming Phase 2 32.82% (135)

2017: ERA Chairs 8.24% (33.91)

2017: Teaming Phase 1 2.92% (12)

2017: Twinning 4.86% (20.00)

Other Actions in EUR million

2014: Transnational Network of National Contact Points (NCP) 0.49% (2.00)

Experts (expert evaluators, experts groups, monitors) 2014-2015 0.57% (2.35)

2014-2015 COST 11.82% (48.61)

2013:WIRE Conference during Greek EU Presidency Conference 0.06% (0.25)

Expert contracts Horizon 2020 Policy Support Facility 0.12% (0.5)

(Innovation missions)

2014: WIRE Conference during Latvian EU Presidency 0.06% (0.25)

Studies 2014-2015 0.01% (0.03)

Horizontal Activities 2014-2015 (Dissemination + 0.05% (0.2)

Communication)

External Expertise 2016-2017 0.03% (0.12)

COST 2016-2017 9.86% (40.56)

Horizon 2020 Policy Support Facility 2016-2017 0.49% (2)

2016: Support to JPI Urban Europe 0.30% (1.25)

2015-WIRE VII Conference during the Netherlands Presidency 0.06% (0.25) of the Council

2016 WIRE VIII Conference in Kosice (Slovakia) 0.06% (0.25)

2016 Presidency Conference: Spreading Excellence and Crossing 0.02% (0.10) the Innovation Divide 2016-2017

Studies 2016-2017 0.01% (0.03)

Source: SEWP Work Programmes 2014-2015 and 2016-2017.

The tables below (Table 237 and Table 240) detailed information on implementation of and participation in the SEWP actions Teaming, Twinning and ERA Chairs. COST, PSF and JPI Urban Europe are not included for specific reasons. The COST networking actions are managed by a dedicated implementation structure, the COST Association and Horizon 2020 funding. As for all the JPIs, the Grant Agreement awarded for the JPI Urban Europe is a CSA that supports, inter alia, governance structures of the initiative as requested by the Council, and therefore, it is not part of the mainstream activities of the SEWP.

PSF support is managed by the Commission services. A first set of nine activities was conducted by using individual expert contracts managed directly by the Commission (total budget: 0.45 million). However, given the volume of demand and staff constraints, a Framework Contract for PSF services was signed in October 2015 (based on the Horizon 2020 Work programme 2014-2015) in order to outsource some supporting services. While the Commission remains solely responsible for the policy aspects of the PSF and its activities (including their communication and follow-up), the PSF contractor provides a ‘smart back-office’ to support the activities of the PSF with the following services: selection of experts (except for high-level panel chairs and subject to Commission approval); drafting of background documents and analysis; organisation of meetings, invitation and reimbursement of experts; communication activities.

The first set of country-specific activities under the Framework Contract includes a Peer

Review of the Ukrainian R&I system 215 and Specific Supports to Slovakia, Romania,

Slovenia Lithuania, Bulgaria and Latvia. Some of these activities, with a standard duration of six to ten months, have been already finalised and the others will be concluded by the end of 2017. Several additional countries (i.e. Poland, Tunisia, Georgia) have already requested a PSF support in the first part of 2017.

The first set of topic-specific activities under the Framework Contract, launched in 2016, includes Mutual Learning Exercises on such topics as performance-based funding of research, alignment and interoperability of national research programmes, innovation procurement or open science. The total budget of the above-mentioned eleven activities under the PSF Framework Contract is about EUR 2.3 million.

In addition a PSF Knowledge Centre was launched in the framework of the joint website of the Research and Innovation Observatory and the Horizon 2020 Policy Support Facility ‘RIO-PSF website’ (https://rio.jrc.ec.europa.eu/en/policy-support-facility) to make available country-based information on Member State R&I policies and performance (including monitoring and analysis performed in the context of the European Semester), covering good practices, relevant statistics, infographics and dedicated reports broken down by country and policy topics.

Table 238 - Key data on proposals per type of action for "Teaming phase 1, Twinning and ERA Chairs": Number of eligible and retained proposals, EC contribution requested (in million Euros) and success rates (as % of proposals submitted and as % of budget available)

Type Of Nr of Nr of EC EC Success Success Rate Action Eligible Retained Contribution Contribution Rate Funding

Proposals Proposals requested by to Retained Proposals Eligible Proposals

Proposals (EUR (EUR million) million)

CSA 790 105 800.3 109.5 13.3% 13.7%

Source: CORDA data, 18 January 2016, Success Rates by Type of Action.

SEWP follows a bottom up approach and therefore there is no topic grouping/clustering of proposals.

215 See the available PSF reports at https://rio.jrc.ec.europa.eu/en/policy-support-facility

Table 239 - Key data on signed grants per type of action for "Teaming phase 1, Twinning and ERA Chairs ": number, EC contribution, time-to-grant, projects' total costs, % of EC contribution in projects

Type Nr of EC Share of EC Nr of Share of Project Share of EC of Signed Contribution Contribution Grants Grants Total Contribution

Action Grants to Signed to Signed signed Signed Cost in in Project Grants Grants (in within within TTG Signed Total Costs (EUR Programme 8 Benchmark Grants (Signed

million) Part) months (in all (EUR Grants) (TTG) Signed million)

Grants)

CSA 112 115.4 100,0% 103 92.0% 120.0 96.2%

Source: CORDA data, 18 January 2017, Selected Projects and Signed Grants by Type of Action.

At the time of the interim evaluation, 112 grants have been signed, out of which 31 Teaming phase 1 projects are completed, 81 are ongoing, none is abandoned. The programme has so far been implemented mainly through Coordination and Support Actions (Teaming phase 1, Twinning and ERA Chairs) and Grant Agreements for Teaming phase 2.

Under Twinning, 67 projects have been approved, each with an allocation of about EUR 1,000,000. 14 ERA Chairs projects have been approved and are currently being implemented with an allocated budget approximately EUR 2,500,000 each. Additionally, in the framework of the FP7 ERA Chair pilot action, 11 projects have been selected and are currently running. Most of the Twinning projects have been active for more or less one year, whereas most of the ERA Chairs projects under Horizon 2020 started in July 2015 and under FP7 in July 2014. The Twinning projects are planned to last for three years and the ERA Chairs projects will last five years.

As for Teaming phase 1, 31 projects were selected in 2015. Each project received up to EUR 500,000 from Horizon 2020 to prepare operational plans for new Centres of Excellence or for upgrading existing ones by teaming up with high-calibre institutions from all over Europe. These projects are completed and the Teaming phase 2 call (for the 31 projects competing under the 2014 Teaming Phase 1 call) resulted in 10 proposals selected for funding. For these projects the grant agreements will be signed in February 2017 and each project will receive up to EUR 15 million for the construction of a new research unit or strengthening an existing one. Funds allocated by Horizon 2020 need to be matched with appropriate additional (normally public) funding, thus creating a large financial push for the new institutions .

Table 240 - Teaming phase 2, results of the evaluation

Country Proposals Retained for funding

1 BG 2 1

2 CY 3 1

3 CZ 3 1

4 EE 2 0

5 HU 2 2

6 LT 1 0

7 LV 1 1

8 MT 1 0

Country Proposals Retained for funding

9 PL 3 0

10 PT 4 1

11 RO 1 0

12 RS (AC) 1 1

13 SI 2 1

14 SK 4 1

Total 30 10

Source RTD B5.

Chart 1 - Thematic domains covered by Teaming phase 2

2

1

0

Source RTD B5.

Teaming phase 2 successful proposals will be implemented in 9 different countries. As a

result of the selection Hungary, Latvia, Serbia had the highest success rates. Among the approved projects, six concern upgraded of existing centre of excellence and four

projects will fund the creation of new ones.

Within the NCP_WIDE.NET project, main activities implemented so far are mentoring visits, a learning platform, brokerage events and trainings. The project is correctly

implementing the Grant Agreement with a total of approx. 20 events organised in 18

months.

The JPI Urban Europe coordinates urban-related research programmes of the

participating countries and addresses societal challenges facing urban areas in Europe

and beyond. The initiative is expected to enhance the R&I knowledge and capacities to support urban transition towards sustainability. In 2016, the JPI Urban Europe was

funded under the Widening Actions of Horizon 2020 to enlarge participation of its

member countries and actors towards the EU-13. The JPI Urban Europe is now in the

process of establishing a Stakeholder Involvement Platform that should facilitate implementation of its Strategic Research Agenda, help to reach out to new countries and

urban actors and continue the co-creative approach.

Q.2.2. Participation patterns

A total number of 112 projects have been selected so far including under Teaming phase

1, Twinning and ERA Chairs projects. Teaming phase 2 projects don't appear in the

tables below because the grant agreements are not signed yet.

Table 241 - Key data on participation per type of organisation for Teaming, Twinning, ERA Chairs: number of participants, of project coordinators, of newcomers, of participations, and EC contribution to participations (in million Euros)

LE Type Nr of Nr of Projects Nr of Nr of EC

Participants Coordinators NewComers Participations Contribution

in Signed in Signed in Signed in Signed to

Grants Grants Grants Grants Participations

in Signed

Grants (EUR

million)

HES 172 68 3 239 71.9

OTH 5 1 3 5 0.6

PRC 16 5 19 1.5

PUB 17 13 7 21 2.3

REC 91 30 4 133 39.0

SUM: 301 112 22 417 115.4

Source: CORDA data, 18 January 2017, Applicants and Applications by Type of Organisation (only

Teaming, Twinning, ERA Chairs).

Table 242 - Success rates (as % of proposals submitted, and as % of budget available) per type of organization for "Teaming, Twinning, ERA Chairs"

LE TYPE Success Rate of Success Rate of Success Rate of Funding Applicant distinct applicants Applications (Applicants)

HES 25.5% 12.1% 12.6%

OTH 10.0% 8.3% 7.7%

PRC 12.5% 12.7% 7.6%

PUB 26.7% 20.9% 21.3%

REC 19.7% 14.0% 16.5%

Sum: 22.1% 12.9% 13.7%

Source: CORDA data, 18 January 2017, Applicants and Applications by Type of Organisation (only

Teaming, Twinning, ERAChairs).

Charts 2 - Key data on participation per country for "Teaming phase 1, Twinning,

ERA Chair: number of participants, of project coordinators, of newcomers, of

participations, and EC contribution to participations (in million Euros)

S ource: CORDA data, 18 January 2017, Applicants and Applications only for Teaming phase 1, Twinning and ERA Chairs.

Charts 3 - Success rates, submitted and funded proposals, eligible and retained

proposals

Source: CORDA data, 18 January 2017, Applicants and Applications only for Teaming phase 1, Twinning and ERA Chairs.

Table 243 – EU-28, EU-13, EU-15, Associated countries, Third Countries for "Teaming phase 1, Twinning and ERA Chairs ": number of participants, of project coordinators, of newcomers, of participations, and EC contribution to participations (in million Euros)

Simplified Country Nr of Nr of Nr of Nr of EC Group distinct Projects NewComers Participations Contribution to participants Coordinators in Signed in Signed Participation in in Signed in Signed Grants Grants Signed Grants Grants Grants (EUR million)

Associated 16 5 1 18 4.0 Countries

EU-13 101 87 16 145 54.1

EU-15 183 20 5 286 57.2

Third Countries 1 1 0.0

Total 301 112 22 450 115.4

Source: CORDA data, 18 January 2017, Participants and Participations by Country group (without Adhoc).

The table above reflects a distinction between groups of countries according to the categories EU-28, EU-13, EU-15, however since SEWP' specificities and participation criteria target a group of countries, the widening countries, that do not fully correspond to EU-13 (as they also include PT and LUX and some of the associated countries), it is relevant to analyse the budget distribution per call, between Widening and non-Widening countries.

Table 244 - Budget distribution between Widening and non-Widening countries

Budget Distribution Widening Non Widening

Teaming Phase 2 111.082.892,25 € 28.755.173,75 €

Teaming Phase 1 8.205.160,00 € 5.888.968,00 €

Twinning 31.669.789,00 € 34.612.125,00 €

ERA Chairs 34.024.659,00 € 0

Source DG RTD.

Table 245 - Success rates (as % of proposals submitted, and as % of budget available) per group of country for "Spreading Excellence and Widening Participation"

GROUP Success Rate of Success Rate of Success Rate of Funding

Applicants Applications (Applicants)

Associated Countries 10.8% 6.8% 6.1%

EU-13 20.4% 12.7% 13.8%

EU-15 25.5% 13.7% 14.9%

Third Countries 7.1% 6.7% 0.0%

Total 22.1% 12.9% 13.7% Source: CORDA data, 18 January 2017, Applicants and Applications by Country groups (without Ad-hoc).

Q.2.2.1. Participation per type of organisation

The selected proposals represent a total of 417 participations, mobilising 301 distinct participants. As shown in the Table 240, most of these were higher education and research organisations, with private commercial organisations and others such as government authorities and private non-profit organisations accounting for a much smaller share of participants. Research organisations and higher education institutions typically participate in projects more often than private commercial organisations and other organisations.

Public organisations are the applicant categories with the highest success rates, followed by research and higher education organisations and private commercial organisations.

Q.2.2.2. Attraction of new participants / newcomers

There are 22 newcomers in a total of 301 participants. 19 of these newcomers are out of the 117 participants from Widening countries (16%). The numbers of newcomers per country varies between 3 and 0. Interesting to note that, except Germany, all other countries with newcomers are Widening Countries. Amongst those, Slovakia has the highest number with 3 newcomers. There are no newcomers in Estonia, Latvia, Luxembourg and Lithuania as well as in all EU-15 countries.

Q.2.2.3. Geographical participation patterns

All 28 EU Member States and Associated countries participate in projects funded under SEWP, however to ensure a targeted approach toward the Widening countries , only entities from the Widening countries, are allowed to submit (in the case of ERA chairs) or coordinate (Teaming, Twinning) a project proposal. This explains the distribution of coordinators in the funded proposals. Portugal is the country which has the highest number of project coordinators (19) followed by Estonia (13) and Poland (12).This is reflected in the EC contribution where those countries remain on the top three. Following these three country, Cyprus and Romania have a good performances in terms of project coordination and EC contribution. The Widening country which receives the least contribution is Lithuania. Taking all participants, including the countries of the advanced partners, the country with the highest participation and distinct participants is Germany followed by UK. The first Widening Country is Portugal in third position. In contrast, Lithuania, Ireland and Latvia are the countries that participate the least with less than 5 participants and/or participations each.

Estonia, Malta and Portugal, have the highest success rates in funding. Estonia and Malta also count the highest success rate on distinct applicants and are in the top 3 for success rate of applications. The least successful in terms of funding rate are three Widening Countries: Slovakia, Slovenia and Lithuania.

Q.2.2.4. International cooperation

A total of 15 entities from third countries applied to the programme, within 12 project proposals. 6.66% of these proposals were retained for funding, involving one third country participant from South Korea in a Twinning project.

Q.2.3. Cross-cutting issues

Performing an analysis at SEWP programme level, 40.3% (EUR 84.3 million) of the budget has been so far allocated to Sustainable development topics (the target for Horizon 2020 is at least 60%), 8.7 % (EUR 18.2 million) of the budget to Climate related topics (it should exceed 35% of the overall Horizon 2020 budget) and 7% (EUR 14.6 million) of the budget has been so far allocated to biodiversity. 10.2% (EUR 21.3 million) of the EC contribution is ICT Research and Innovation related.

In terms of promotion of socio-economic sciences and humanities, it can be observed that 12% of projects are flagged as SSH relevant receiving EUR 102 million of EC contribution.

In SEWP projects 43.6% of the workforce are women (1345 women) while 69 women are project coordinators (59.0 % of coordinators); 8 men and 11 women out of 19 (42% vs 58%) are members of the EC advisory groups.

There are no innovation actions in SEWP.

Q.3. R ELEVANCE

Q.3.1. Is « Spreading Excellence and Widening Participation » tackling the right issues?

Q.3.1.1. The relevance of «Spreading Excellence and Widening

Participation» given the challenges to address

The 2011 Commission's analysis on the low participation and budget allocation of organisations from a set of countries, coinciding with the Widening countries, led to the introduction of SEWP. The problems identified were insufficient national R&D investments, lack of synergies between certain countries’ national research systems and EU research, lagging system learning effects and access to existing networks, differential wage levels between countries as well as insufficient and ineffective information, communication advice and training.

The SEWP actions aim to address these problems each with its own approach: Twinning facilitating networking; Teaming focussing on excellence and institution building and change; ERA Chairs by attracting talents and encouraging the use of ERA principles; PSF enabling national reforms; COST connecting research capacities and interdisciplinary research; JPI through national programmes alignment and inspiring innovative practices; NCP and other conferences and events by enhancing the quality level information.

The importance of the existence of SEWP is confirmed by the recent data on participation pattern, funding allocation, success rates, etc. for FP7 and Horizon 2020 as shown in the table below:

Table 246 - Performance of EU 13 versus EU 15 (EU Funding, success rate, participation, EU funding per researchers)

Horizon 2020

FP7, EU-13 EU-13 EU-15 EU28 Overall

Share of EC contribution 4.2% 4.4% 88.5% 92.9% 100%

Annual EC Contribution per researcher FTE (in 1,321 1,271 3,808 3,475 n.a

EUR)

EC Contribution per EUR million spent on R&D N/A 67,524 63,277 63,429 n.a

(public and private, GERD)

Share of participations 7.9% 8.5% 82.6% 91.1% 100%

Success rate of applications 18.0% 11.1% 14.4% 14.0% 14.1%

Source: European Commission, cut-off data 1 January 2017

The Table 246 shows that the situation highlighted in FP7 still persist in Horizon 2020. The share of the funding received by organisations in EU-13 countries in Horizon 2020 is in fact relatively low (4.3% in 2014 and 4.7% in 2015).

In addition, the recent debate at the 'Crossing the innovation divide' (SECID) conference

(Brussels, November, 2016) – proceeding will be soon published by SLORD 216 office,

organiser of the event during the SK presidency- has also confirmed that the main problems identifies in 2011 still persist. The lack of national investment, need for institutional reform and opening up the established networks for new researchers from the widening countries are still challenging. While it was recognised that widening actions target rightly these problems, it is a shared responsibility to create the conditions to maximise the impact at all levels.

The impact of the spending on research and innovation is related to the functioning of the national and regional Research and Innovation systems. They vary widely across Europe and one of the first challenges for countries with low levels of excellence is to introduce reforms to their systems. Adequate reforms require good diagnosis of the system and then a process to reach agreement between the different actors, expertise and persistence to implement reforms. The role of the Horizon 2020 Policy Support Facility, in this context is crucial and countries like Bulgaria, Romania, Hungary, Slovakia and Malta have already benefited from this support. In a complementary way Teaming and ERA Chairs, besides building excellence in research, also contribute to building sustainable institutional capacity and change in the target countries’ institutions. This goal calls for improvements in the leadership, management and governance of institutions, which are often hindered by national legislation and capacity issues.

In addition, the strong demand for SEWP funding as expressed by the oversubscription to the calls, provides evidence of the relevance of the programme to the stakeholders.

However, it should be highlighted that some further data needs to be taken in consideration, and in some EU-13 countries, the EU Framework Programme plays a very important role in total national RTD expenditure (GERD): Horizon 2020 funding represents 25.9% for Cyprus, ~10% for Malta and ~5% for Romania, compared to 1.5%

of Germany and France's national GERD 217 . Moreover, EU contribution normalised by

GERD shows that EU-13 countries now get relatively similar amounts to EU-15.

216 SLORD > Slovak Liaison Office for Research and Development http://slord.sk/

217 Data from Annual Monitoring Report 2015

It should also be noted that Widening countries (as defined by the composite indicator) are affected by the above identified problems to different extents and intensity, showing that the group is not a homogeneous block and that the dichotomy EU-15 versus EU-13 is a strong simplification of the reality (a detailed analysis on geographical coverage is in the efficiency chapter).

The IETEC study has been looking also at the perception of the Widening actions beneficiaries. Results show that the ERA Chair's objectives are perceived as corresponding to the needs of the institutions. 93% of the coordinators and ERA Chairs holders find that the objectives of the ERA Chair action correspond to their needs to a very large extent or to a large extent. This finding is supported by the qualitative interviews in which all the interviewed ERA Chairs project coordinators and Chair holders express the opinion that the actions respond to the needs of either the specific department, overall institution or both. This is demonstrated by their declarations on the results already obtained (see chapter 5 on Effectiveness). Among the ERA Chair holders and coordinators, 88% found that there are relevant costs not covered by the action. In particular most interviewees highlighted that the ERA Chairs grant does not cover the cost of research e.g. equipment and infrastructure, leaving the institutions applying for research funding from other sources, in particular national grants but also Structural Funds. For some of the projects this has been and still is a problem to overcome. While they will have the expertise of talented researchers, the equipment or infrastructure for these talents to work on may be lacking. For instance, one ERA Chair project coordinator mentioned that their ERA Chair had to spend time in making research applications in search of research funds before being able to do research. Some of the interviewees suggested that a small amount of the ERA Chair grant could cover the research costs at the beginning of the projects to avoid this problem. However this is not a widespread problem, as most of the hosting institutions fund their research from their own resources and ERA Chairs do also accept to have to apply separately for funding and activities.

Also for the Twinning actions, according to the interim evaluation (IETEC study, 2016) 96% of the coordinators find the objectives of the Twinning actions corresponding to the needs of their institutions to a very high degree/high degree. Also the qualitative interviews with the Twinning project coordinators confirm this result. However, 57% of the Twinning coordinators responding, found that there are relevant costs that should be covered by the action and are not, in particular the cost of research equipment.

The Horizon 2020 Policy Support Facility (PSF) was set up to provide operational support to Member States in the design, implementation and evaluation of research and innovation policy reforms. In order to enhance its relevance, Member States were actively involved in design of the PSF. In October 2014 a dedicated ERAC workshop was organised to assess the rationale for the PSF and countries' specific needs with respect to the PSF. Its outcome was fully taken into account in designing the PSF services. Member States remain actively engaged in the monitoring of the PSF through the European Research Area and Innovation Committee (ERAC). The PSF is operating as an on-demand tool which directly responds to countries' requests: each PSF project is tailored to the specific country's needs and the country's high-level political commitment to the PSF work is an ex ante condition for the PSF to undertake its activity. The Commission's role is thus to assess demand, to ensure this demand is matched with the most relevant expertise, and to guarantee the policy relevance, quality and impact of the PSF activities. PSF exercises involve comprehensive interactions with a large range of national authorities and stakeholders, ensuring therefore political visibility and due follow-up at the national level. The Member States appreciate very much the possibility to access in a flexible way high-level independent expertise leading to practical advice on concrete policy issues. The increasing demand for PSF services proves that Member States value the policy recommendations received through the PSF from leading experts and practitioners.

For COST the relevance is confirmed by the increasing participation from Widening countries and from the commitment the COST Member States to invest 50% of the EC grant at the benefit of the research communities in Widening countries. This commitment is based on the proven COST inclusiveness policy and the particular expertise of COST to integrate researchers from smaller and/or peripheral countries into European science networks.

The relevance of the programme to address the identified problems is also very much provided by its requirement (for its core action) to be aligned with the Smart Specialisation strategy of the hosting country. This paves the way to fully exploitation of existing resources for R&D – including ESOF- in a synergistic way, which is very much needed in the Widening countries to catch up in the innovation divide. In particular, as through ESIF support can be provided to research infrastructures and research centres; promoting business R&I, technological and applied research and key enabling technologies, as well as networking and development of clusters, synergies with Horizon 2020 have proved to be relevant in several contexts where they have been operationalised. A concrete example of synergies between ESIF and Horizon 2020 is the ELI - Extreme Light Infrastructure (distributed) project, located in Czech Republic, Hungary and Romania, that is supported by these countries under their ESIF resources

complementing the European Strategy Forum on Research Infrastructures (ESFRI). 218

Strengthened synergies of Horizon 2020 with ESIF and with the Instrument of Pre Accession (IPA II) for Candidate Countries are a way to increase the impact of investments in low R&I performing countries.

The development of the JPI Urban Europe Strategic Research and Innovation Agenda (SRIA) was the result of a co-creative process, inviting main urban stakeholder groups - cities, business, society and research - on local, national and European level - to inform the SRIA of their specific priorities and reflect the strategy against the requirements at different levels. The SRIA defines research and innovation priorities of the initiative until 2020. It responds to the pressing need for ambitious, sustained and genuinely inter- and transdisciplinary research and innovation that will radically improve our understanding of the social, economic and environment sustainability of urban areas. It will help the transition of Europe's cities to a future that maximises their sustainability, resilience and their liveability. In doing so, the SRIA aims to bridge the silos of urban-related R&I activities with an integrated approach, linking national and European funding and connecting science with innovation and policy as well as with business and civil society. The box below provides evidence of the implementation state of play.

Q.3.1.2. The relevance of «Spreading Excellence and Widening Participation» to address European objectives

The objectives of the SEWP were set in a different political context (ERA, EU2020 strategy). In this context SEWP was formulated to contribute to the ERA objectives by improving researchers' mobility, supporting the modernisation of national science

218 See publication EU funds working together for jobs & growth at http://bookshop.europa.eu/en/eu-funds-working

href="http://bookshop.europa.eu/en/eu-funds-working-together-for-jobs-growth-pbKI0116339/">together-for-jobs-growth-pbKI0116339/

systems and national reform agendas, unlocking career prospects of young researchers and improving the gender balance. In addition, it contributes to the realisation of the Innovation Union by fostering collaborative research and knowledge transfer and strengthening the science base of Europe.

In view of the more recent political agenda SEWP's relevance is very high with reference to the Commission's 3Os Strategy (Open Innovation, Open Science, Open to the World), which lays out a vision for R&I for the future that is actually based on practices that are already widely evident in parts of the R&I. With reference to the Widening countries SEWP contribute to Open Science with its focus on strengthening R&I capacities, to Open Innovation with its attention to improvement of framework conditions and to growth and jobs and finally to Open to the World with its support to improved international positioning. In particular, Open innovation is about involving far more actors in the innovation process and for this Europe needs to create the right eco-system, increase investments and bring also regions into the knowledge economy. Considering the role of these actions to boost synergies with ESIF, they also contribute to Open Innovation by maximizing the impact of every euro spent on R&I whatever the source of funding. Along the same line, the Seal of excellence (SoE) expands the range of Horizon 2020 funding by opening up opportunities for alternative funding to excellent projects evaluated under the Horizon 2020 SME scheme but not financed due to budgetary constraints.

Additionally, the PSF is also one of the flagship initiatives developed under Commissioner Moedas' Open Innovation, Open Science, Open to the World agenda. The PSF is a key aspect contributing to improving the regulatory and legislative environment for R&I, which is a priority of the Open Innovation pillar. A first mutual learning event specifically dedicated to some key aspects of "Open Science" will be launched early 2017 and it will be followed by others. Finally, the PSF is "Open to world" since its support is also dedicated to associated countries (such as Moldova, Ukraine, Georgia, Armenia and Tunisia).

In this regard also COST significantly contributes to the outreach of Horizon 2020 through its near neighbourhood and international cooperation policy. The main motivation of non-EU COST partners is network-driven as revealed by a recent impact study undertaken by COST. Major benefits for them are network building, paving the way for joint research projects as well as exchange possibilities for their PhD students. COST is today often the first collaborative European scheme for New Neighbour Countries (NNC) and for some International Partner Countries (IPC) partners, as the case of the newer EU Member States 10-15 years ago. It is an “entry point” to European

collaborative research. 219

Q.3.2. Flexibility to adapt to new scientific and socio-economic developments

All Widening actions are bottom-up therefore proposers are free to come-up with actions in all fields they consider relevant to strengthen their socio-economic context. The only condition required by the work programme is a broad alignment with the national/regional smart specialisation strategy (a requirement for 'Teaming' and desirable for 'Twinning' and 'ERA Chairs'). The latter is based on the identification of strategic

219 Final Report on COST Targeted Impact Assessment 2016, done by Fraunhofer ISI and Austrian Institute of

Technology, Deliverable D9.3 of SGA 681 463

areas for intervention based both on the analysis of the strengths and potential of the economy and on an Entrepreneurial Discovery Process (EDP) with wide stakeholder involvement.

Q.3.3. Addressing specific stakeholder needs

The process of defining priorities in the SEWP work programme is based on a broad consultation of stakeholders. This ensures that each programme addresses the relevant and issues and corresponds to the possibly changing needs of the stakeholder community. Several stakeholders' events have been held to ensure the involvement of stakeholders in the drafting of the SEWP work programme.

Q.3.4. Other issues related to relevance

The characteristics of the innovation divide between EU-13 and EU-15 appear to be gradually changing, with some of the newer Member States increasing their performance substantially. Therefore, the current composite indicator needs to reflect these changes in order to define correctly the countries to be targeted with the Widening actions.

Q.3.5. Lessons learnt/Areas for improvement

The lack of national investment, need for institutional reform and opening up the established networks for new researchers from the widening countries are still persisting challenges for the widening countries.

In addition, the strong demand for funding as expressed by the oversubscription to the calls provides evidence for the relevance of the programme.

However, it should be noted that Widening countries (as defined by the composite indicator) are affected by these problems to different extents and in various intensity, showing that the group is not homogeneous. This also entails that the dichotomy EU-13 versus EU-15 is a strong simplification of the reality.

Q.4. E FFECTIVENESS

Q.4.1. Short-term outputs from the programme

SEWP actions are at a very early stage of implementation, therefore the results at this stage are quite limited. However, some interesting outputs from the programme have been already materialised, especially in relation to new partnerships created, increased visibility and attractiveness, mobilisation of complementary resources. In view of the heterogeneity of the SEWP actions, it is useful to look at each of them to assess the current and expected outputs.

Q.4.1.1. Creation of scientific output

The majority of SEWP' actions are aimed at strengthening the R&I capacity and excellence, concretely by improving the research capacity, supporting excellence-based recruitment, supporting mobility of research, connecting research capacity, improve the capacity to mobilise research funding. In terms of publications, of course this takes time and while it is possible already to identify some results for ERA chairs and Twinning, for Teaming it is very early.

In terms of publications, the survey (which includes 11 FP7 project and 14 Horizon 2020) shows that 25% of the ERA Chairs projects have not yet published any peer reviewed publications, while 31% have published 1-4 peer reviewed publications. The Twinning survey shows that 43% of the projects have had 1-5 publications peer reviewed.

This positive result for both actions gives indication about how the projects are contributing to improve the scientific performance of the institutions involved and the progress being made towards the expected output of increasing the scientific excellence.

ERA Chairs projects point to recruiting the Chair Holder and establishing research groups, as the main achievements of the ERA Chairs projects to this date. In some of the projects there have been some delay in recruiting the Chair holders and consequently in the recruitment of the research group members and preparation of grants and publications has only recently started. In the projects in which the Chair holder was recruited according to the original time schedule (17 out of 25), the processes of preparing grant applications and submitting papers for peer review have started.

The survey shows that all of the Chair holders were selected through an international call. One of the interviewed ERA Chair project coordinators highlighted the open call recruitment as an important result and the most transparent recruitment in the history of the hosting institution. While another noted how they have to follow the national recruitment procedure, they have strived to make the recruitment as transparent as possible and published the position on EURAXESS. However, two of the interviewed ERA Chair project coordinator stated that the institution followed Western standards of recruitment before receiving the ERA Chairs grant and therefore do not see this as an accomplishment in itself. This shows that also the ERA Chairs hosting institutions have different degrees of openness to and acceptance of the ERA principles.

The majority of the ERA Chairs survey respondents have not observed any particular unintended or unexpected effects from their participation, respectively 72% and 78%. The interviewees however highlighted the problems created by the differences between the countries and the institutions with relation to ways of doing research, national legislations, cumbersome administrative procedures, or a different culture at the institutions that led to delays in the recruitment process of a replacement PhD.

The survey shows that a minority (13%) of the projects has experienced unintended effects so far, and only 5% have experienced unexpected effects.

Looking at the extent to which the planned objectives of the ERA Chairs action have been achieved (see table below) 75% of the ERA Chair project coordinators and Chair holders find that the projects have to a high degree (53%) resulted in increased attractiveness for international excellent researchers or to a very high degree (22%). The majority of the respondents find that the institution has become more capable to compete for international funding (50% to a high degree and 25% to a very high degree). Furthermore, the survey shows that the majority find increased research excellence in the field of the chair holder, with 44% finding this to a very high degree and 44% to a high degree. A majority of the respondents (60%) perceive the transparency in the recruitment procedures to have increased, where 23% to a neither high nor low degree believe the transparency has increased and 13% find this to a low degree. This reflects how some of the institutions perceived their recruitment procedures to be transparent before receiving the ERA Chairs grant.

40% of the respondents find that gender equality has increased in a neither high nor low degree. As mentioned previously, the gender of the Chair holder was not highlighted as a main reason to choose the Chair holder which indicates that gender is given less importance than other indicators.

Teaming phase 1 coordinators reported that the project was a unique opportunity for learning by doing on: process of preparation of a business plan, including thorough financial analysis, the research strategy; connecting research output with innovation performance; development of sound technical and administrative documentation such as the quality management system; establishing and implementing a CoE; integrating technological and regulatory aspects, both at national and international level. Additionally, several coordinators considered a significant learning experience the process of putting together a large and complex project with a long-term vision and well-thought-out tasks and task forces, creating and managing a large, complementary and well-balanced consortium. Their overall experience in writing and managing EU- funded projects was also enriching.

Collaboration with the advanced partners helped the coordinators to get acquainted with best research and administrative practices (management and funding of leading EU research and innovation institutions, creating inspirational highly competitive environment) and understand the importance of collaboration as well as involvement of policy-makers and all other relevant stakeholders for achieving scientific progress and overcoming the existing barriers and cooperation with industry through their involvement from the very beginning of the project. Finally, several coordinators mentioned the new insights that they gained into their research areas and into opportunities for cross-sectorial interaction (utilization and application from/to other research areas).While there is not yet a 'scientific output' as such, the phase 1 of Teaming resulted in 31 business plans for the development of a centre of excellence. This exercises to which all partners contributed, entailed long-term R&I strategies and possibly it will be used by the participants even in the cases where the proposal was not selected for Phase 2 funding, due to the tough competition.

COST also contributes to expected outputs of achieving a better connection of research capacities through networking and leadership opportunities for new talents and thereby contributing to strengthen and build up excellent S&T communities; secondly it addresses the issue of strengthening the trans-disciplinary research networking as well as new approaches and topics and identifying early warning signals of unforeseen societal problems. Three open calls were made since 2014 and 66 excellent proposals were selected and approved using a novel submission, evaluation and selection system under the supervision of a high level scientific committee that replaces the former domain committees. Currently, there are about 300 networking actions run by more than 45000 researchers.

International Cooperation continued to be implemented on the basis of mutual benefit. Montenegro (Widening country) formally became a COST Member in May 2015.

Q.4.1.2. Partnerships and international openness

Teaming, Twinning, ERA Chairs, COST, JPI Urban Europe are all contributing to enhance partnership and international openness of the people and institutions involved.

Under ERA Chair the majority of the projects report that partnerships have been established, 75% of the ERA Chair projects having made two or three new partnerships. Furthermore, 47% of the ERA Chairs projects have submitted 1-5 new research proposals, whereas 35% have submitted between 6 and 10.

One objective of the ERA Chairs grant is to internationalize the research institutions. Therefore, the nationalities of the Chair holders were analysed (IETEC, 2016). Most of the ERA Chairs holders' nationalities are different from the country of the hosting institution. Thus, the objective of establishing international networks with high profile researchers is met in the majority of the projects. Three Chair holders are from the same countries as the hosting institutions. A majority of the ERA Chair holders are from Western European countries, and five are from Asian countries e.g. India, Pakistan and South Korea. One Chair holder is from USA and one is from Australia. The interviews showed that the Chair holder's network has made networks and collaboration with other high profile researchers as well as international institutions possible. These collaborative networks are expected to last after the end of the projects. A similar conclusion can be drawn from the survey. A large majority expects that the research institution will experience increased networking capability with world-class institutions.

The Twinning survey shows that the number of newly established partnerships varies between one and five, where 37% of the Twinning projects have established one new partnership and 17% have established two. Furthermore, 25% of the projects have submitted one new research proposal and 17% have submitted two such proposals. In view of the current early stage of implementation such performance can be satisfactory.

Some Twinning project coordinators emphasised the learning effects of managing a Horizon 2020 project, for the first time in the role of project coordinator. Some have found that it is a challenge to initiate and formulate the collaboration with the partnering institutions and to combine research strategies. However, they find that these obstacles can be overcome or already have been. The Twinning project institutions experience international researchers coming to conferences at the institutions. Before the Twinning project, they would not have come. Through the contacts with the collaborative institutions, the Twinning institutions can establish contacts with international institutions and researchers outside of the projects. Looking to the results of the survey, a similar conclusion can be drawn. The majority of the respondents expect the Twinning support to ensure that the institution's networking capabilities with high profile research institutions will increase to a very high (48%) or a high (50%) degree.

For Teaming, it is clear that already as a first positive outcome one can consider the creation of new partnerships or strengthening of existing ones. While most of the projects (25 out of 31) were based on some form of existing partnership – with six coordinators basing their 'Teaming' project on existing partnerships in Horizon 2020, FP7, FP6, EUREKA, EUROSTARS or the European Innovation Partnership on Active and Healthy Ageing Reference Sites (MIA) – it is interesting to note that for 19 projects the partnership was partially new - based on existing scientific links that had not necessarily evolved into joint project applications or bringing together former collaborating partners (Horizon 2020, COST) and new ones and for 6 out of the 29 projects the partnership was new.

In addition, Teaming also led institutions to take a coordination role they had never covered before: while for most of the Teaming projects (23 out of 31) all partners have had previous experience in Horizon 2020 (incl. a successful ERA Chair project – the coordinator of MIA), as well as FP7, EIP on Healthy Aging Reference Sites (MIA), EIT Health KIC (MIA), EIT ICT KIC (EE-IT), ERA-NET, etc. (and most of the advanced partners clearly have substantial previous experience in the Framework programmes), it interesting to note that for three coordinators, the 'Teaming' project is the first one in Horizon 2020, in which they act as coordinator. The coordinators of 4 projects appear not to have any previous experience at all. Teaming therefore brings valuable experience for the coordinators of the low-performing countries: they learnt how to put together a big and complex project with a long-term vision and manage a large consortium and they also got familiar with EU best practices in the management and funding of leading research and innovation institutions. Nearly 45% of the coordinators who replied to the survey clearly stated that the partnership was balanced.

SEWP contributes 50% of the EC support to COST in order to boost participation from Widening countries. To monitor progress on this objective COST has adopted a list of Inclusiveness Target Countries (ITC), which corresponds to the Horizon 2020 Widening countries, and has committed to spend 50% of its budget for the benefit of the research communities in those countries. Such objective however has not been fully attained by the end of 2016 when the contribution was at the level of 40%. For this purpose a specific task force was launched in July 2016 with participation of European Commission DG RTD Unit B5 to boost participation and fully fulfil the KPI. The Task Force came up with a package of actions for Widening countries to be implemented in 2017 including: 1) minimum condition for the inclusion of those countries already at proposal stage (based on a fixed ratio); 2) the obligation to fill at least one key position of the management committee (chair, vice chair, working group leaders); 3) a new conference grant for young researchers, 4) the development of a mentoring scheme for improving payment modalities. Closer collaborations with other ERA initiatives such as Joint Programming Initiatives (JPI), ERA-NETs could be developed. Links between the thematic parts of Horizon 2020 and COST are systematically improving.

Q.4.1.3. Well-functioning R&I ecosystem

Under SEWP several actions contribute to improving the functioning of the R&I ecosystem as a whole and to the improve collaboration among institutions and policy makers. As to the support for sharing of research agendas, the JPI Urban Europe

launched its Strategic Research and Innovation Agenda (SRIA) 220 and related

implementation plan in 2015 as a result of a co-creative process mobilising the main urban stakeholder groups (cities, business, society and research) on local, national and European level. The SRIA aims to bridge the silos of urban-related R&I activities with an integrated approach, linking national and European funding and connecting science with innovation and policy as well as with business and civil society. At this stage of the implementation of all the Joint Programming Initiatives, the agreed progress indicator for the JPIs is the following: Strategic Research and Innovation Agendas and related implementation plans are in place. As for the overall effectiveness of the JPI Urban Europe, the initiative has thus achieved this milestone (as described above).

With regard to raising the quality of information, advice and training, the NCP WIDE.NET - a network of NCPs for the Widening actions - has received support from SEWP and facilitates transnational co-operation between NCPs. The network organises regular annual meetings and training days and also involves the Commission services in its events. As an outcome of these actions, the administrative and operational capacity of the Widening NCPs is strengthened and the flow of information between them and the

220 SRIA - Transition towards Sustainable and Liveable Urban Futures; http://jpi-urbaneurope.eu

Horizon 2020 implementation bodies is ensured. Within 18 months the project organised many events: 3 mentoring visits, 3 trainings and 4 workshops. An assessment on the impact of the NCPs is currently under preparation and will be finalised in 2017 providing information on the effectiveness of the various NCP including the Widening one.

The Horizon 2020 Policy Support Facility (PSF) offers expert advice to public authorities at national or regional level on a voluntary basis, covering the needs to access the relevant body of knowledge, benefit from the insight of international experts, use state of the art methodologies and tools, and receive tailor-made advice.

When looking at the progress of the policy on synergies between Horizon 2020 and ESIF, the framework for synergies is now in place in the regulatory documents of Horizon 2020 and ESIF. The respective regulations of both Horizon 2020 and ESIF already provide for the harmonisation of some rules and procedures. The Commission (DG REGIO) has set up a Task Force on Simplification that also explores Horizon 2020 approaches as synergy inspiration for ESIF as Horizon 2020 comprises major simplifications compared to its predecessor programmes.

The policy commitment to maximise synergies between Horizon 2020 and ESIF

stimulates new R&I practices across regions and Member States 221 . In that regard,

national and regional authorities have a central role in the transition towards greater synergies, including the design of new alternative funding schemes for implementing the Seal of Excellence initiative. The exchange of knowledge is essential for the promotion of synergies, and this process will be assisted, inter alia, by the Knowledge Exchange Platform (KEP) established between DG RTD and the Committee of the Regions in 2015. The overall aims of the platform include activities targeted at maximising synergies between Horizon 2020 ESIF at regional level.

Furthermore, numerous conferences and events organised in connection with SEWP contribute to the overall knowledge sharing. In that regard, the Week of Innovative Regions in Europe (WIRE) Presidency conference is a key activity. Each year WIRE brings together stakeholders, such as policy makers, enterprises, public institutions and European research centres, to discuss the latest issues relating to research and innovation. Similarly, the annual SEWP Presidency conference provides a high level platform for knowledge sharing in the area of the Widening actions.

Finally, the JPI Urban Europe – which was awarded a EUR 1.5 million grant under the SEWP in 2016 to enlarge participation of its member countries and actors towards the EU-13 – is currently in the process of establishing a Stakeholder Involvement Platform that will help to reach out to new countries and urban actors. The platform will support experimentation with different kinds of R&I tools, test ways to co-operate and mobilise different actors and give reflections on urban practices. A forum will be held each year inviting stakeholders from national networks and urban practitioners to join the debate.

Q.4.1.4. Policy and institutional reform outputs

Looking at the effectiveness of the Horizon 2020 Policy Support Facility, each activity is

concluded with a report 222 including a set of recommendations together with supporting

221 Examples of new practices in synergies are provided in this publication: http://bookshop.europa.eu/en/eu-funds

href="http://bookshop.europa.eu/en/eu-funds-working-together-for-jobs-growth-pbKI0116339/">working-together-for-jobs-growth-pbKI0116339/

222 For the already published reports please consult https://rio.jrc.ec.europa.eu/en/policy-support-facility

evidence and analysis, which is transmitted to the host country (for country-specific activities) or to the group of participating countries (for topic-specific MLEs). In parallel the reports are made publicly available on the RIO-PSF website and disseminated to ERAC for mutual learning purposes.

The reforms proposed by the PSF support better and more impactful public investments in research and innovation and the creation of a more attractive environment for innovation at national level. The PSF responds quickly to demands by Member States and helps overcome policy bottlenecks identified by countries, such as complex or ineffective funding procedures for scientific research or the lack of strong innovation eco-systems or connectivity between the science produced in a country and its industrial tissue. The PSF furnishes a strong evidence base to tailor policy reforms to national circumstances. Increased demand from countries led to a rich pipeline of 26 PSF activities in 2015-2017. This allows the PSF to capitalise on the experiences gained and to benefit from a rich set of lessons from policy design to evaluation.

As examples, the recommendations from the PSF Peer Review carried out for Bulgaria 223

in 2015 formed a key input for its recent national strategy: a Science Agenda for

Bulgaria. The Peer Review recommendations issued for Moldova in 2016 224 advised the

government to establish a new Ministry for Science and Education which is being set up. One of the recent PSF mutual learning exercises allowed several countries to investigate different practices on the administration and monitoring of R&D tax incentives, a topic which so far had not been discussed at such operational level in any other forum. Finally,

the PSF knowledge centre 225 provides policy makers and researchers with structured

information and data on R&I performance and policies through a single repository.

The operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms. This is attested, for example, by the sets of policy actions which are being enacted at the national level in Bulgaria as a consequence of the PSF Peer Review, like the recently adopted (in 2016) renewed Science Agenda of Bulgaria which pays attention to the recommendations put forward by the Peer Review.

Even is still at its first stage of implementation, also Teaming is already contributing to institutional change and reform. As pointed out by some coordinators, innovative practices became necessary during the implementation of Teaming phase 1. For example new legal acts regulating the co-financing of the future Centre of Excellence brought up the need for simplification of the tax system for financial transfer and infrastructure exchanges in Poland. Clarification of legal issues related to the Statute of the coordinator and its approval by the Government was needed for one of the projects and in two other projects the institute regulations had to be modified or clarified with a view to accommodating the future CoE. In Hungary the Teaming action prompted an amendment of the Act on National Higher Education to enable universities to become owners of economic legal entities, which was previously not possible. One project led to a

governmental decision to join EMBL 226 as prospect Member State in 2014. The cofinancing

 of the future CoE was ensured through a bill of the Ministry of Education

223 https://rio.jrc.ec.europa.eu/en/country-analysis/Bulgaria/country-report

224 https://rio.jrc.ec.europa.eu/en/library/horizon-2020-policy-support-facility-peer-review-moldovan-research-andinnovation-system

225 https://rio.jrc.ec.europa.eu/en/policy-support-facility

226 European Molecular Biology Laboratory, http://embl.org/index.php

(Estonia). The same project led to ongoing reforms at the university under the new management.

Considering the ambitious role assigned to the Teaming projects and the Centre of Excellence that should be generated, and the impact also a local level, a specific call requirement refers to consistency with the smart specialisation strategy priorities and to mobilizing complementary funding, possible ESIF, through a sound and string commitment by the funding authority, to be used in a synergistic way.

Q.4.2. Expected longer-term results from the programme

At this stage of programme implementation it is not possible to present evidence-based longer-term results of the programme. Nevertheless, some indications on the results to be expected can be made based on expected effects as perceived by the beneficiaries

When analysing the expected effects of the ERA Chairs projects, four effects are prioritized by about 70% of the projects (IETEC, 2016): Establishing partnerships with high profile international institutions, increasing attractiveness for high profile international researchers, increasing excellence in the field of the Chair holder and improving their ability to compete for international research funding . In a complementary way, when looking at the expected improvement of the level of research excellence in scientific and technological research at the institutions, a large majority of the coordinators and ERA Chair holders expects the level to rise to a very high degree (50%) or a high degree (47%).

The most expected effect of the institutions’ participation in the Twinning projects (93% of the respondents) is the establishment of partnerships with international, excellent research institutions. 78% expect an increased research excellence in the fields covered by the Twinning partners and about 50% expect increased attractiveness and capability of the institution to compete for research funding. Neither recruitment transparency nor gender equality is seen as particularly increased, and the majority finds these to have increased to a neither high nor low degree.

Considering at the expected level of excellence in scientific and technological research at the institutions, a large majority of the respondents expect the level to rise to very large extent (34%) or a large extent (60%).

As regards the Horizon 2020 Policy Support Facility, the mobilisation of the host/participating country during the whole PSF process, starting with the commitment and background preparation and concluding with the country's involvement in the discussions on the final report and recommendations is not only an output of the PSF process but also a key condition for ensuring the country's readiness for change and endorsement of the final recommendations through concrete reforms. In rolling out these reforms, which could be considered as the PSF long-term result, the host country can continue to call upon the PSF for support. The recurrent feedback on the PSF work received from national policy-makers and stakeholders has shown that the operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms. This is proven, for example, by the sets of policy actions which are being enacted at the national level in Bulgaria as a consequence of the PSF Peer Review, like the recently adopted (in 2016) renewed Science Agenda of Bulgaria which pays attention to the recommendations put forward by the Peer Review. In addition, to be able to proceed further with the reforms in R&I, Bulgaria requested a follow-up specific support for developing a performance-based funding system and a model for the evaluation of the Research institutions' performance.

Q.4.3. Progress towards attaining the specific objectives

The specific objective of SEWP is "to fully exploit the potential of Europe's talent pool and to ensure that the benefits of an innovation-led economy are both maximised and widely distributed across the Union in accordance with the principle of excellence".

This translates in a set of objectives, namely to unlock excellence in low-performing RDI regions and Member States and associated countries; to widen participation of these countries in Horizon 2020 and to contribute to the achievement of the European Research Area. As detailed in the logical framework in Section Q.1 the main expected outputs are related to the strengthened institutional, scientific and networking capacities of centres of excellence located in low performing regions and Member States, on the basis of partnerships with internationally leading institutions and researchers. Together with the output on R&I policy framework and support to strategic planning and implementation.

It is early for assessing the achievement of the outputs triggered by SEWP and therefore the progress towards the specific objectives. At this stage, it is only possible to analyse the portfolio of running and already approved SEWP projects and in particular what they have currently achieved already at this early stage of implementation (as mentioned in the effectiveness part). Based on the information collected, by extrapolation, one can assume that the projects will achieve their targets therefore leading to the overall project portfolio to achieve its specific objectives. In the analysis, it should be kept in mind that the current ongoing projects (Teaming phase 1, Twinning and ERA Chairs) represent only 14% of the total available budget for the SEWP. The Teaming phase 2 projects, which have been approved but don't appear yet in the financial reporting because the grant agreements have not been signed yet, will allocate 17% of the SEWP budget as illustrated in the 13 below (10 projects of max. 15 million Euro each) and therefore represent a significant investment for the institutions and the countries which have managed to have a project approved.

With regards to Teaming phase 1, Twinning and ERA Chairs, 112 projects contribute to the SEWP's objectives in the 19 Widening countries listed in the table below. The number of projects currently under implementation varies among countries with Portugal, Estonia, Poland and Cyprus as most successful in terms of participation. In a complementary way the objective of strengthening framework conditions, which is pursued primarily by the PSF, has so far provided and is providing support to the countries listed in the last column of the table below. Together with Teaming, Twinning and ERA Chairs, COST programme plays a role in improving the international positioning of the stakeholder in each country with a different number of participations. The yellow column shows the number of ongoing projects in Widening countries within the COST programme.

Table 247 - Countries participation to SEWP's actions

TEAMING phase 1, TWINNING, ERA CHAIRS COST Teaming phase 2 PSF

Country Nr of Nr of Nr of EC Nr of Projects

distinct Projects Particip Contributio participatio Teaming

particip Coordi ations n to ns in phase 2

ants in nators in Participatio running approved PSF

Signed in Signed n in Signed COST (Grants not yet activities

Grants Signed Grants Grants actions (as signed, EC

Grants (EUR of January contribution

million) 2017) max of 15 mil.

Euro)

Bulgaria 8 4 8 1.8 165 1 completed

Croatia 9 5 10 4.5 232

Cyprus 7 10 14 7.7 125 1

Czech Republic 6 8 9 3.3 200 1

Estonia 6 13 17 13.7 158

Hungary 10 6 13 1.9 200 2 completed

Latvia 4 4 4 1.2 104 1 completed

Lithuania 3 1 3 0.3 161 completed

Luxembourg 3 1 6 0.9 70

Malta 5 3 5 1.0 134 completed

Moldova

(Republic of) 1 1 1 0.5 NA completed

Poland 14 12 16 9.5 250 planned

Portugal 22 19 34 17.4 268 1

Romania 11 11 12 5.8 201 ongoing

Serbia 5 3 5 1.2 229 1

Slovakia 8 6 8 1.6 157 1 completed

Slovenia 10 4 12 1.8 214 1 ongoing

Turkey 1 1 1 0.5 194

Ukraine 1 1 0.1 NA completed

Montenegro 44

Bosnia and

Herzegovina 128

Source: DG RTD B5.

The table provides an overview of the actions in each country. It is right to assume that the combination of different SEWP actions in an integrated manner will enhance the R&I excellence and capacity in the different country, once a critical mass is reached. This assumption should be assessed in the upcoming monitoring and evaluation exercises.

The countries which are most successful in Widening actions such as Portugal, Estonia and Cyprus have not yet requested the support of PSF. Estonia however will not have a Teaming phase 2 project.

Bulgaria, Czech Republic, Hungary, Latvia, Slovakia and Slovenia belong to the group of countries which currently combine the Widening actions with PSF and will also benefit of significant investments for Teaming 2. Poland, which has been relatively successful so far, has also requested the PSF support, however institutions in this country have not managed to succeed in Teaming 2. It is worth recalling that the Teaming phase 2 projects will firstly mobilise significant investments (up to 15M each), and secondly the beneficiaries have committed to leverage complementary ESIF funding.

Therefore, besides an analysis based on the single actions, an effective coordination is required at programme level to ensure that the multifaceted effects of SEWP actions are operationalised at participants and country level. Cross-checking the effectiveness of interventions on a country basis, besides at the level of the actions, would allow firstly to analyse the progress made in each country during the SEWP implementation and secondly to assess the progress on expected results at the level of the multiple SEWP objectives (scientific output, policy reform and institutional change, networking and improved capacity to mobilise funding, adequate R&I ecosystem).

The following map (Figure 2) shows the combination of the EC contribution in signed grants for the Widening actions currently under implementation with the PSF activities.

Figure 253 - Intersecting between Widening actions and PSF

SourceEuropean Commission, DG RTD B5.

Q.4.4. Progress towards the overall Horizon 2020 objectives

Q.4.4.1. Fostering excellent science in scientific and technological research

The current or expected results from SEWP contribute to fostering excellent science in scientific and technological research in the Widening countries. Scientific excellence as an output can be measured through the number of publications, especially in high-impact journals. This is indeed the Key performance indicator for Twinning and ERA Chairs. However, at this stage of the programme implementation, it is too early to assess credibly to what extent funded projects could actually deliver excellent scientific and technological results. The survey has indicated that projects are currently working on new collaborations and new publications which are the two main expected results mentioned by the beneficiaries.

Box 25 - Contribution to the achievement and functioning of the ERA

SEWP contributes to several key ERA objectives. The PSF, through the expert advice provided by renowned experts and policy practitioners, helps increase the effectiveness of the national research systems of the Widening countries by triggering the implementation of national R&I reforms and the formulation and optimisation of national research and innovation policies. The Teaming action also aims to improve the national research and innovation ecosystems by bringing in the valuable knowledge and experience of advanced research institutions.

The Widening actions (Teaming, Twinning and ERA Chairs) and COST Intergovernmental framework enhance the transnational cooperation and knowledge transfer, specifically supporting the integration of institutions and researchers from the Widening countries. In this way they address the low level of involvement and participation of these countries in the established international scientific networks, which has been a major drawback for the improvement of their overall R&I performance.

The ERA Chairs action has increased the attractiveness of institutions in the Widening countries for excellent international researchers and through the open and merit-based recruitment for the high profile position of the ERA Chair holders has reinforced the open labour market for researchers. One of the main tasks of the ERA Chairs action is to foster compliance with the ERA priorities (gender balance, open recruitment, peer review) at institutional level.

Q.4.4.2. Boosting innovation, industrial leadership, growth, competitiveness and job creation

Data are currently not available.

Q.4.4.3. Addressing the major societal challenges

Analysing the percentage of EU financial contribution that is climate-related 4 project have indicated 100% climate relevance, 34 with a relevance at 40% and 60 projects reported a 0 climate relevance.

Q.4.4.4. Science with and for society

Within SEWP, total number of projects is 118, out of these 15 are marked as SSH relevant, 74 as non SSH relevant and for 29 the information is missing. Share of signed grant SSH relevant is 16.9%.

Looking at the percentage of Responsible Research and Innovation projects where citizens, Civil Society Organisations (CSOs) and other societal actors contribute to the co-creation of scientific agendas and scientific contents out of 118 total projects, 29 are marked as relevant, 52 as not relevant and for 37 the information is missing. Share of signed grant RRI relevant is 35.8%.

Regarding the gender dimension in research and innovation content, 42.7% (50) of EC funded projects have included a sex and/ or gender analysis as part of their research or innovation activities.

Q.4.5. Early success stories

Twinning

 SUPREME 227

The transition from fossil fuels to renewable and sustainable energy sources has become the European Union’s top developmental priority, with low-performing countries in Central Europe facing the most urgent need. As the region’s largest country, Poland’s continuing economic progress has not come without significant costs; due to its history in electricity production, in 2009 it had the highest rate of production by coal of any EU Member State. This makes Poland Europe’s third largest polluter in terms of damage to society, home to six of Europe’s 30 most damaging power-plants, and to be among Europe’s worst for public exposure to harmful pollution. At the same time it is experiencing rises in domestic electricity demand twice the EU average. This makes Poland the most urgent nation in the EU with regards to the need for immediate conversion to renewable energy systems and resources. However, unlike traditional power facilities, energy produced by RES often produces unpredictable and variable outputs related to weather, season, and geographical location. While Polish research now has expertise in many of the technologies needed for energy transition, it lacks critical knowledge in modelling, planning, integrating, and managing large scale renewable energy systems in a flexible and effective manner. The SUPREME project twins one of Poland’s best energy research centres, the Instytut Maszyn Przeplywowych Im Roberta Szewalskiego Polskiej Akademii Nauk with needed expertise in Denmark (Aalborg University), the Netherlands (University Twente), and Austria (the European Sustainable Energy Innovation Alliance). Focusing on needed knowledge transfer in integrating energy technologies, the project’s well-formulated mix of extended staff exchanges, joint work, Summer Schools, and other events will create a long-lasting and effective partnership that will have a very significant impact on Poland’s energy systems infrastructure.

EU Contribution: EUR 1.047.551

Start date: 01/11/2015

227 http://cordis.europa.eu/project/rcn/200260_en.html

 LINK 228

The LINK (Linking Excellence in Biomedical knowledge and Computational Intelligence Research for personalized management of CVD within PHC) project address today’s PHC (personalised healthcare) systems that miss adequate integration of clinical evidence and knowledge from holistic clinical practice and biomedical research required to support truly holistic management of chronic diseases and their co-morbidities. Current PHC systems are designed using the “one fits all” principal lacking a truly personalization by capturing and adapting to the patients’ phenotype (e.g., by linking systems medicine and the virtual physiological patient to tele-monitoring data) and individualized treatment or context needs. Data processing is at the core of PHS where acquired data is turned into meaning and action. In order to pave the way from personal to personalised systems, PHC require intelligent algorithms to treat and correct data obtained from uncontrolled conditions, to efficiently integrate multimodal and multiscale data, to be self-adapting (moving from population-based to patient-specific adaptations) and interpretable, and to integrate clinical and biomedical evidence at their genesis. LINK aims at linking competences in intelligent processing in order to create a research ecosystem to address two central scientific and technical challenges for PHC deployment: (1) infusion of clinical evidence biomedical knowledge in PHC solutions and (2) moving PHC solutions from personal to personalized services, i.e., services adapted to the specific user needs and characteristics. The project is led from Coimbra, in cooperation with Valencia (Universitat Politechnica de Valencia) and Milan (Politecnico di Milano).

EU Contribution: EUR 1.010.590

Start date: 01/01/2016

ERA Chairs

 CEITEC 229

The ERA Chair project (part of the FP7 pilot) aims at supporting the on-going structural shift in the culture of the scientific community of Masaryk University by engaging a world-class scientific leader capable of inspiring positive change. Masaryk University has already been deeply involved in this process as the largest contributing member of the Central European Institute of Technology (CEITEC).

CEITEC is a scientific centre in the fields of the life sciences and advanced materials and technologies with the aim of establishing itself as a recognized centre for basic as well as applied research. The CEITEC consortium includes the most prominent universities and research institutes in Brno in the Czech Republic. Thanks to EU Structural Funds, CEITEC is currently in a very rapid stage of development. One of the challenges facing the organization is to maximize the high potential that CEITEC has at its disposal through cross-disciplinary collaboration.

The recruitment of the ERA Chair has been accomplished and a new research infrastructure, financed from the EU Structural Funds has been completed in 2015. Currently the set-up of a critical mass of progressive scientists inspired and influenced by the ERA Chair has the potential to bring in sustained research funding for further development of the research capacity. The high-impact research outputs of the newly established excellent research group that will be made public will serve as a key motivating factor for other groups.

228 http://cordis.europa.eu/project/rcn/199966_en.html

229 http://cordis.europa.eu/result/rcn/170378_en.html

Through the regular exposure of high potential researchers to a broad selection of international grant application avenues and working with scientists to hone their grant writing skills, the opportunities to access grant funding, will increase.

The project is also contributing to a better integration in the European Research Area thanks to aligning to the European Charter for Researchers and Code of Conduct for the Recruitment of Researchers or respecting the Principles of Innovative Doctoral Training.

The economic impact will be most significantly realized with the creation of new technologically demanding jobs. The training of these new graduates as outlined in the project would come closer to the demands of the market than is currently the case. Current involvement of the ERA Chair into the regular Regional Innovation Strategy meetings will also help to ensure linkage between academia and industry in the region. EU Contribution: EUR EUR 2.246.401

Start date: 01/06/2014

 PSF Peer Review in Bulgaria

The recurrent feedback on the PSF work received from national policy-makers and stakeholders has shown that the operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms. This is attested, for example, by the sets of policy actions which are being enacted at the national level in Bulgaria as a consequence of the PSF Peer Review, like the recently adopted (in 2016) renewed Science Agenda of Bulgaria which pays attention to the recommendations put forward by the Peer Review. In addition, to be able to proceed further with the reforms in R&I, Bulgaria requested a follow-up specific support for developing a performance-based funding system and a model for the evaluation of the Research institutions' performance.

Another example of a promising follow-up to some Peer Review recommendations can be identified in Moldova where the new Ministry for Science and Education is under the development as advised by the PSF experts in 2016.

Q.4.6. Lessons learnt/Areas for improvement

ERA Chair and Twinning projects already resulted in a substantially increased of the attractiveness of the institution for international excellent researchers and in an improved capability of the institution to compete for international funding. Additionally for ERA Chair the research excellence in the field of the chair holder. ERA Chairs projects are improving the institutions recruitment procedures that have become more transparent after receiving the ERA Chairs grant. On a more operational level, the majority of Twinning and ERA Chairs co-ordinators consider the ineligibility of equipment and consumable costs as an obstacle for the effectiveness and speed of the actions.

COST actions demonstrated effectiveness in including excellent researchers from widening countries with a steadily increasing participation rate. Some more efforts need to be undertaken to move the financial contribution at the benefit of the Widening countries from the current 40% to 50%.

The recurrent feedback on the PSF work received from national policy-makers and stakeholders has shown that the operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms. The operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms. This is attested, for example, by the sets of policy actions which are being enacted at the national level in Bulgaria as a consequence of the PSF. However, it is the country’s responsibility to ensure the followup to the PSF activity and to implement the PSF experts' recommendations through concrete reforms and therefore a more systematic follow-up (on demand and in the context of ERAC or cross-cutting workshops) and exploitation of the PSF results would be useful.

Looking at the programme level, the effectiveness of SEWP integrated approach to the multifaceted dimensions of the innovation divide (scientific output, partnerships and international openness, policy output and effective R&I ecosystem) requires effective programme coordination in implementation, monitoring and evaluation.

Q.5. E FFICIENCY

Q.5.1. Budgetary resources

Table 248 – EC contributions, number of signed grands for SWEP actions

SEWP Actions Nr of Signed Grants EC Contribution to Signed

Grants (EUR million)

COST 3 90.1

SEDIC Conference 1 0.1

Twinnng 67 67.3

JPI-UrbanEurope 1 1.5

Teaming phase 1 31 14.1

ERA Chair 14 34.0

Widening NCP 1 2.0

Total 118 209.1

Source: CORDA data, 18 January 2017, Selected Projects and Signed Grants by Type of Action.

Resources spent in SEWP are in line with plans and work programme. However, the data are incomplete since the PSF budget is not included.

Q.5.2. Programme's attractiveness

Q.5.2.1. Mobilisation of stakeholders

Consultation of stakeholders has been an integral part of SEWP. Inputs were collected during a Stakeholder consultation with selected external stakeholders. A dedicated lunch debate took place on 17 June 2016 where representatives of various relevant associations gave their views and suggestions for designing the next Work Programme.

When looking at the implementation state of play of Teaming phase 1, Twinning and ERA Chairs the following data describe the current status. With reference to Time-togrant (TTG), by 24 November 2016, the percentage of projects (Teaming, Twinning and ERA Chairs) signed within this eight month period was 92.0%, the average time-to-grant being 238 days.

Looking at the oversubscription rate per type of instrument, there is only one instrument in SEWP calls CSA (Coordination and Support Actions).There have been 790 eligible proposals with a success rate of 13.3% and 13.7% in terms of EU funding. As 2016 calls (Teaming Phase 2) have not yet concluded the grant signing phase, the success rate remains the same as 2014-2015. The oversubscription rate is 7.17.

Analysing the success rates per type/country of applicant, Public Bodies (excluding research and education - PUB) and Higher or secondary education (HES) remain the main beneficiaries of SEWP with 26.7% and 25.6% respectively. Others (OTH) and industry participation (private for profit organisations - PRC) are the lowest represented participants with 12% each. This data is not surprising considering that the actions do not foresee a specific involvement of private sector. Main beneficiaries are research institutions, universities and academia.

A survey of Horizon 2020 National Contact Points 230 was run end 2016 to explore their

knowledge, awareness and assessment of synergies. Overall, the preliminary analysis of the NCP survey shows that the engagement of NCPs in the development of synergies between Horizon 2020 and ESIF seems rather limited, even if EU-13 NCPs are in general more active and more positive than EU-15 NCPs. The level of NCPs' activity to develop, promote, disseminate and implement synergies as well as their knowledge of synergies still seems to be in the early stages of development. In order to better track synergies the Commission (DGs RTD & REGIO) is exploring possibilities for setting up a comprehensive monitoring system for synergies that would complement and/or substitute the current specific surveys addressed to Horizon 2020 and ESIF communities.

Dissemination and Communication activities

Following the launch of Horizon 2020 three Info Days to present the Widening calls were organised in Brussels in 2014 and 2015. Additionally, a stakeholder's lunch was held in Brussels to raise visibility and awareness among key-target audiences and several presentations on SEWP were given in Brussels. High level meetings took place with Member State representatives in Germany (September 2013) and Poland (March 2014 and April 2015) focused on Teaming action.

A brokerage event was organised by the NCP_WIDE_NET (Widening NCP) in the framework of the Week of Innovative Regions Conference in Riga (2015) and in Eindhoven (2016).

NCP_WIDE.NET project's activities aimed to contribute to an improved and professionalised NCP service across Europe, thereby helping simplify access to Horizon 2020 calls, lowering the entry barriers for newcomers, and raising the average quality of proposals submitted, a more consistent level of NCP support services across Europe and an increased application levels as well as improved quality of proposals particularly from newcomers. The project has contributed to raise the quality of information advice and training.

The project has established trans-national collaboration of NCPs in the area of Spreading Excellence and Widening Participation (SEWP). The project has organised tasks (events, workshops and trainings etc.) specifically for EU-13 NCPs. Within 18 months the project organised many events: 3 mentoring visits, 3 trainings and 4 workshops.

230 The on-line survey undertaken as part of a study supporting the interim evakuation was sent to 840 NCPs and the

response date was 27.4%, i.e. 230 NCPs responded

The project is focussed on sector-specific questions and one of the major objectives of the network is to develop high quality services provided to Spreading Excellence and Widening Participation beneficiaries i.e. regional stakeholders participating in Teaming, Twinning, ERA Chairs and COST activities. The assistance provided by NCP_WIDE.NET goes beyond typical NCP services since the network has to promote cooperation between regional/national authorities and research community and their trans-national cooperation, and guide the complex ecosystem of regional vs. global innovation centres.

Events of the project were attended by hundreds of attendees and it should be beneficial to Horizon 2020 applicants. All these events aimed at increasing quality of services offered to applicants by NCPs – this goal is being achieved by systematic capacity building of NCPs relying on possibility of exchange of knowledge and experience, enhancing knowledge, knowing best practices due to the participation in mentoring visits and trainings offered under WP2, as well as relevant publications presenting expertise on

Spreading Excellence and Widening Participation activates 231 .

The Week of Innovative Regions in Europe Conferences (WIRE) which is organised on a yearly base in the framework of the Presidency of the Council, is among the main European policy fora for debating and inspiring new policies and practices in the R&I in the context of regional development. Since 2014 WIRE conferences have been gathering on an annual basis stakeholders from the whole R&I ecosystem: policy makers, regional and national authorities, academia and universities representatives, cluster organisations and innovative companies as well as all other stakeholders supporting the growth of innovativeness from a regional and urban perspective. Topics related to SWEP are often dealt with within the conference.

Brainport Eindhoven Region (Netherlands) hosted the 7th edition of the European Commission’s Week of Innovative Regions in Europe (WIRE 2016). More than 350 people attended the 3-day conference in the high-tech city of Eindhoven. Several field visits were organised, showcasing the key elements of success for a region previously in decline, and now in the forefront of technological innovation at global level. The conference stressed the important role of regional innovation ecosystems as a cornerstone of the innovation process. It highlighted the role of place-based innovation that emerges as the key tool for economic growth in a globalised knowledge economy.

The WIRE VIII conference is due to take place in Kosice (Slovakia) on 28-30 June 2017. The conference, entitled ‘Creative, social and trusted regional innovation in digital era” has got three objectives. It will contribute to maximising and understanding the societal impact of research, innovation and regional policy in digitalisation on regions and people. It also aims to support R&I ecosystem stakeholders (quadruple helix) in using the benefits from digitalisation and enhance trusted initiatives and policies for effective innovative and socially responsible regional development. It will also contribute to the creation of the future in Horizon 2020 and regional R&I.

The international conference “Spreading Excellence and Crossing the Innovation Divide” took place on 23rd of November 2016 in Brussels and was attended by 200 guests. The overall objective of the conference was to provide a platform for stakeholders to have an in-depth discussion on the role of excellence in European research and innovation as well as opportunities presented through Horizon 2020 and ESIF for

231 Review/ Evaluation of all 16 NCP projects is in process and conclusions will be available in 1Q2017.

crossing the innovation gap between countries, regions, research institutions, universities, and enterprises. The event served as a forum to assess results of Teaming, ERA chairs and Twinning projects in order to point out the main reasons for these disparities, and provided opportunities creating for future knowledge sharing opportunity.

All national and regional calls for proposals and events related to the Seal of Excellence

are published on DG RTD dedicated website 232 . The Seal of Excellence website received

863 visits in November 2016, of which 675 were unique visitors. This means that both the visits and the unique visitors have doubled compared to June 2016. November also counted 474 returning visits and 1815 page views. Page views have tripled since June 2016.

Q.5.2.2. Geographical dimension

Portugal, Estonia and Cyprus have a good performance in terms of application and EC contribution. In this group only Estonia has not yet managed to attract a Teaming phase 2 investments.

The second group showing a discrete performance includes Malta, Latvia, Romania, Hungary, Czech Republic, Slovakia, Slovenia and two associated countries, Croatia and Serbia. Poland is in-between the two groups with a good performance in the widening actions except for Teaming phase 2. Most of these second group countries have also requested the support of the PSF, with the exception of Czech Republic, Serbia and Croatia. PSF's support deals with innovative entrepreneurship in Romania; the internationalisation of the science base and its cooperation with business in Slovenia; and with funding system for public research, including its governance and organisational aspects in Latvia.

Bulgaria, Lithuania, Luxembourg and the remaining associated countries belong to a third group of countries which are the least successful. However, Bulgaria's success in Teaming phase 2 will lead to an increase in the EC contribution for the country once the grants will be signed. PSF support in Bulgaria includes a Peer Review of the Bulgarian R&I system followed by a specific support on performance-based funding and evaluation of public research institutions. PSF's support to Lithuania deals with science-business links and attraction of innovation-related foreign investments.

The overall performance of the Widening countries in Horizon 2020 for the first two years of the programme implementation shows that some EU-13 countries are, in spite of overall lower Horizon 2020 contribution, outperforming the EU-15 average. E.g. Slovenia, Cyprus and Estonia outperform the EU-15 averages, taking into account the size of the population, the number of researchers and national investments in R&D. Conversely, Polen, Lithuania and Bulgaria (exept for R&D expenditure) are underperformin the EU-13 average.

232 https://ec.europa.eu/research/regions/index.cfm?pg=soe

Table 249 – Horizon 2020 contribution normalised by inhabitant, researcher and R&I investment nationally

Source: European Commission

Q.5.3. Cost-benefit analysis

The majority of the interviewed ERA Chair project coordinators and chair holders see the projects as being cost-effective as they have followed the budget that was prepared in the applications. 59% of the survey respondents find that the funds allocated to the projects are adequate to achieve the planned effects to a high degree.

Some comments were made by the interviewees on factors hindering cost efficiency. One of these was paperwork and national bureaucracy e.g. in relation to recruiting foreign PhD students to research teams, or in the approval of a new course by the national ministry of education. Even though most of the interviewees find the projects to be very cost-efficient external factors such as the national bureaucracy do not contribute to cost efficiency. Many ERA Chairs project coordinators and Chair holders also pointed out that the ERA Chair recruitment enables the institutions to offer the salaries needed to attract high profile researchers. However, some also claimed that when the projects end, the institutions can probably not offer the same salary level and expect that researchers will not be attracted to the hosting institutions due to low salaries.

The majority of the interviewed Twinning project coordinators all find that they are in a condition to achieve good results at low costs. The projects follow the budget that was prepared in the applications. A similar conclusion can be drawn from the survey results. A majority find that the grants for the Twinning action are adequate to achieve the expected effects to a high (50%) or very high (18%) degree. Some interviewees find that the hosting institution being located in low cost countries enables them to achieve many results at low cost. On the other hand, one Twinning coordinator points out that the staff exchange visits are costly as the living costs in the countries of the collaborating partners are high.

Two Twinning project coordinators mentioned that understanding the definitions of eligible travel costs in the Twinning call is difficult. This concerns e.g. eligible travel costs for the research partners or the daily benefits given to the PhD students. The project coordinators found this to be a challenge, because the eligible costs were defined somewhat vaguely in the call and there had been disagreements between the institutions on defining specific eligible travel costs in the projects. The project coordinators have had negotiations with respectively the research partners and the head of the institution to find solutions to these obstacles.

Teaming project coordinators consulted mentioned that the potential barriers to the success of the action lie in the difficult balance between feasibility of objectives as perceived by the project partners and the perceived requirements of the call for a critical mass of expertise and scientific achievement and between broader and more specific objectives; the need to define the most important aspects of excellence and of a definition of a Centre of Excellence (focus on scientific excellence or market relevance and commercialisation; focussed on specific research discipline or covering a broader area), its organisational structure and independency and the applicable legal requirements and necessary start-up funds for the setting up of a new legal entity; the low level of remuneration of researchers in the less-performing RTD countries.

The concept of a Centre of Excellence with a view to costs to be covered by the action proved also to be problematic for some coordinators who raised the issue of the need to consider funding the infrastructure and research activities. The inert thinking of some institutions in the low-performing RTD countries could also be a barrier to the successful implementation and overall impact of the action. According to Teaming project coordinators, the requirements for national funding should be strengthened (e.g. contract among the consortium and the national funding authority already at proposal stage). Clear indication should be provided on whether the support from the regional authorities has the same weight as the support provided by the national authorities. The motivation of the advanced partners to participate in the action and the need for them to see clearly the benefits from such an involvement should also be considered.

Looking at the Teaming action, coordinators responding to the survey (Assessment made by DG RTD, 2017) commented that potential beneficiaries need to receive clear guidance on the concept of a Centre of Excellence in terms of objectives, structure and legal form, scientific and financial sustainability (including eligible costs under the action). Indications on possible organisational models of the Centre of Excellence and share of the budget devoted to the management and administration of the Centre of Excellence were considered necessary. Clear guidance on the scope of the Centre of Excellence (specific or broader research area) and the required mix of scientific excellence and innovation/market prospects, was also suggested.

As the Teaming action addresses both the upgrade of existing Centres of Excellence (with track record of achievements and implemented projects) and the establishment of completely new Centres of Excellence, a distinction between the two might be relevant already at the evaluation stage. Regarding the support from the national or regional authorities, coordinators claimed that it should have been better defined and possibly formalised at an early stage. As regards the evaluation, the coherence between the evaluations at the two stages is important and the continuity of the process should be ensured through communication among evaluators, project officers and reviewers.

Q.5.4. Lessons learnt/Areas for improvement

Some efforts have been deployed to reach out the stakeholders in the Widening countries, however, to fulfil the specific aim of widening participation in the targeted countries more integrated efforts could be made to inform potential beneficiaries, especially in the countries which show a discrete or low capacity to be successful in research funding.

The stakeholder's feedback provides relevant suggestions on how to improve the simplifications of the call's participations and the cost eligibility.

Strengthening the role of the Widening NCP on one hand and reinforcing synergetic actions with the Horizon 2020 Policy Support Facility and the pilot project Stairway to Excellence (covering EU 13) and Lagging regions (covering EU 28) that the JRC implements on behalf of DG Regional and Urban Policy would certainly bring to a stronger mobilisation of Widening stakeholders.

Project coordinators mentioned that the definitions of eligible travel costs in the Twinning call needs to be made clearer. Teaming's coordinators commented that potential beneficiaries need to receive clear guidance on the concept of a Centre of Excellence in terms of objectives, structure and legal form. National bureaucracy e.g. in relation to recruiting foreign PhD students to research teams, or in the approval of a new course by the national ministry of education, is mentioned by participants as one important factor hindering cost efficiency.

Q.6. C OHERENCE

Q.6.1. Internal coherence

Q.6.1.1. Internal coherence of the actions implemented for Spreading Excellence and Widening Participation

The internal coherence of the SEWP programme is ensured by the complementary character of the different actions as they address different needs and scales. Applicants may opt for long term partnerships with advanced partners aiming at the development of new or modernisation of existing Centres of Excellence. Alternatively, they may head for a twinning partnership with less financial and institutional commitment but driven by the need to develop a specific scientific domain. Excellent individual scientists and their teams have the potential to become game changers in the institutions in the less research performing countries and boost institutional reforms. The ERA Chair scheme provides the right intervention point in this regard without requiring deeper institutional

partnerships. Far ranging, large scale but less binding networks supported by the COST

Framework further complement this portfolio by connecting pockets of excellence.

Q.6.1.2. Internal coherence with other Horizon 2020 intervention areas

One of the SEWP's objectives is institution building and institution networking to

maximize and enhance the R&I performance of the beneficiaries in low

performing/Widening countries to be able to better compete for competitive funding. The SEWP actions should therefore lead to increased participation of these countries in the

future also to the other Horizon 2020 actions, in the topic relevant to the field of

expertise. SEWP has therefore adopted a bottom-up approach which allows projects in any R&I area of interest to the proposer.. SEWP is in this way highly complementary to

the other intervention areas as it allows projects in any topics area that will then fit into

other Horizon 2020 calls. In addition, as SEWP does not cover the cost of research and

therefore participants do apply to other funding sources for this purpose.

This is further confirmed by the perception of the beneficiaries. The IETEC's survey

shows that 88% of them have already received funding from other Horizon 2020 sources

such as ERC and Marie Curie grants together with Research Infrastructure. Societal Challenge, FET and LEIT are to a minor extent included in the complementary funding

received by the ERA Chairs beneficiaries. Regarding other regional, national or

international funding, many of these funds cover the cost of research.

Chart 4 - Funding received by the ERA Chairs institutions

n=30. Source: IETEC Twinning and ERA Chairs survey, 2016.

The ERA Chair beneficiaries perceive low overlaps with other funded activities. As

earlier mentioned, some find it challenging and time consuming to apply for funding for

research equipment. However, none of the interviewees received any funding they perceived as overlapping with the ERA Chair projects. Many of the project coordinators

mention that their research costs and infrastructure are funded at national level.

Therefore, they find it complementary that the ERA Chairs grant covers the recruitment of researchers. The survey results on the extent of overlap between the ERA Chairs

support and other Horizon 2020, EU/regional/national activities show that a majority of

the ERA Chair coordinators and Chair holders (59%) find that there are only few overlaps. A rather small minority find that there are overlaps to a high degree (13%).

Chart 5 - Overlaps between ERA Chairs and other interventions

n=32 ; Source: COWI Twinning and ERA Chairs survey, 2016

As illustrated by the figure below, a similar result is found when asking about the perceived coherence with other activities funded at EU, regional, national level with

objectives related to the ERA Chairs support. The majority of the ERA Chairs

coordinators and Chair holders perceive the ERA Chairs grant to be complementary with other actions, 47% to a high degree and 13% to a very high degree. From the chart below

and the chart above, it can be concluded that the ERA Chairs support is complementary

with other activities funded within both Horizon 2020, other EU, regional or national

levels including activities with objectives related to the ERA Chairs support.

Chart 6 - Coherence and complementarity with other EU interventions

n=32, Source: COWI Twinning and ERA Chairs survey, 2016

Of the Twinning institutions, 91% have already received funding from other EU sources.

The figure below shows the distribution of other funding sources. Other regional,

national or international funding include among others funding for research.

Chart 7 - Funding received by the Twinning institutions

n=38, Source: COWI Twinning and ERA Chairs survey, 2016

Overall, the beneficiaries perceive the Twinning projects to be complementary with other

funded activities. Likewise to ERA Chairs, some of the coordinators find it challenging and time-consuming to apply for funding for research equipment, none receives any

funding that overlaps with the Twinning grant. At many of the supported institutions,

research costs and infrastructure are funded at national level. Therefore, they find it complementary that the Twinning grant covers the research and networking activities,

which are not covered by national funds.

A large majority of the Twinning coordinators believe to a high (50%) and very high

(33%) degree that the Twinning support is coherent and complementary with the objectives of Horizon 2020 or other activities funded EU, regional or national level. A

similar result is found on overlaps, however, the respondents are somewhat spread. Few

find the other funded activities to be overlapping with the Twinning intervention (13%), whereas the majority does not see the funded actions to be overlapping to a low (21%) or

a very low (18%) degree. 28% of the respondents find the actions to be neither

overlapping nor complementary and 21% do not know.

The coherence of the Horizon 2020 Policy Support Facility with the EU economic policy is ensured as the analyses developed in the context of the European Semester, in

particular the identification of the bottlenecks to the contribution of R&I to growth and

job creation 233 , constitute the basis for the PSF activities. Moreover, there are important

synergies with the EU Regional policies although the PSF support is not requested by regions but by countries. In particular, the PSF support can help to ensure a more efficient use of ESIF funding for R&I, in the context of smart specialisation. Regional and Urban Policy's support to regions and countries via the S3Platform, experts and peer-to-peer learning and help to ensure a more efficient use of ESIF funding for R&I, in the context of smart specialisation. The PSF draws on analysis and information collected by the S3 platform and the Research and Innovation Observatory, but also on information from other sources such as the Innovation Policy Platform of the OECD and the World Bank. In this context it is important to mention that there is a close cooperation of the Commission services with the OECD on the development of the economic evidence base to underpin R&I reforms.

In addition, links and synergies have been explored between the PSF and the Structural Reforms Support Service (SRSS) of the Secretariat General since those two services are complementary Commission instruments to support national reforms. The core business of the SRSS is however different. The SRSS supports the building up of administrative capacity in the EU Member States in order to attain more effective institutions, stronger governance frameworks and efficient public administrations. The PSF provides expertise and operational support to Member States for the design, implementation and evaluation of structural reforms of the national research and innovation systems. Strengthening the ties between the activities of PSF and SRSS will also allow for strengthened synergetic work to address the challenges identified for each Member State in the context of the European Semester.

Q.6.2. External coherence

Q.6.2.1. Coherence with other EU funding programmes/initiatives

The Widening actions are coherent and complementary with other policy strands of the EU notably Cohesion Policy. The design of the programme already entails synergies with Cohesion Policy in particular for Teaming where applicants are obliged to ensure appropriate co-financing for building centres of excellence from the European Structural and Investment Funds (ESIF) or other sources. This encourages the use of parallel funding from different sources for the same objective, maximizing the investments made on R&I from both sides. Beyond the mere financial dimension, the programme is well aligned with the overall objectives of Cohesion Policy notably to help less R&I developed European countries and regions in order to catch up and to reduce the economic, social and territorial disparities that still exist in the EU.

When analysing the complementary between SEWP and ESIF, it is important to note that both programme have a large and open coverage of the different R&I dimensions. For the ESIF the selection of specific type of intervention can vary among regions and Member States, according to the strategic choices made in each territorial context. However the overlapping between the two programme is not possible due to the different geographical coverage of the two instruments, i.e. the ESIF fund national or regional interventions, while Widening is based on trans-national partnerships and networking and hence helps to pool resources and achieve synergies at the European scale.

233 http://ec.europa.eu/europe2020/pdf/themes/2016/research_innovation_201605.pdf

Especially the Teaming action has attracted a lot of attention at political level, with submitted proposals either coordinated or supported financially by national or regional authorities. In several countries (e.g. Poland), national competitions were held by relevant Ministries in order to identify the best proposals for the EU-wide competition – a first in the history of Framework Programmes. Equally, because of the link (Teaming in particular) with regions' Smart Specialisation Strategies for Research and Innovation, some countries have taken the initiative to link the actions with their Operational Programmes in ESIF (e.g. Poland, Czech Republic).

To provide support for the implementation of synergies, the Commission has produced guidance to the relevant authorities through a Staff Working Document (SWD (2014)205 final) and annexes ‘Establishing Synergies between European Structural and Investment Funds, Horizon 2020 and other research, innovation and competitivenessrelated

 Union programmes’ 234 . The guidance document contains explanations on the

basic rules and principles for achieving synergies and combining the different funds, and contains recommendations to the relevant actors as well as to the Commission on how to facilitate synergies. In 2016, DG RTD jointly with DG REGIO published a publication ‘EU funds working together for jobs & growth - Examples of synergies between the framework programmes for research and innovation (Horizon 2020) and the European

Structural and Investment Funds (ESIF)’ 235 with many real-world examples of synergies

at strategic, programming and project implementation levels and coming mostly from the previous programming period (2007-2013). The publication also highlights initiatives with a high potential for synergies during the current programming period (2014-2020). DG RTD together with DG REGIO has actively communicated and exchanged information on the development of synergies at inter-institutional meetings, for example at different configurations of the Council working groups and at the European Parliament; organised conferences and seminars where sessions were dedicated to

synergies 236 ; delivered training on synergies to various external audiences, inter alia, to Horizon 2020 NCPs 237 , and to the staff of the Commission and Executive Agencies.

At strategic and programming level, DG RTD supported DG REGIO in the cohesion programming process in 2012-2014 and ensured that the programming documents include references to synergies. This resulted in an outcome where all smart growthrelated Operational Programmes contain references to synergies. In a short period of time, smart specialisation strategies have also taken their place as an ideal framework for synergy action. Smart specialisation strategies are the ex-ante conditionality underpinning R&I funding through ESIF. Supported by very significant assistance from the Commission's services, the now over 120 existing Smart Specialisation Strategies at national or regional levels identify priority R&I areas and activities, and many of the identified smart specialisation priorities allow a matching with Horizon 2020 themes and thus offer opportunities for synergies. The Thematic Smart Specialisation Platforms (TSSPs) are a new concept launched by the Commission that allows regions and countries with related smart specialisation priorities to come together in a cooperative manner. They in particular team-up innovation stakeholders to develop a pipeline of projects along value chains and exploiting related Smart Specialisation priorities. The

234 http://ec.europa.eu/regional_policy/sources/docgener/guides/synergy/synergies_en.pdf ; also available at the

Horizon 2020 Participant Portal http://ec.europa.eu/research/participants/portal/desktop/en/opportunities/other/index.html

235 http://bookshop.europa.eu/en/eu-funds-working-together-for-jobs-growth-pbKI0116339

236 For example, annual Presidency events; 'Week of Innovative Regions in Europe'; workshops at the Week of

European Regions and Cities

237 For example, NCP WIDE.NET http://www.ncpwidenet.eu/

TSSPs have already been established in the domains of energy, industrial modernisation and agri-food. DG RTD is actively involved in the development of the TSSPs as they provide potential for implementing synergies.

Synergies have been stimulated in Horizon 2020 Work Programmes and related calls 238 ,

by raising awareness of future applicants of the opportunities offered by a synergetic use of funds in the upstream or downstream activities linked to their project. Some topics encouraging applicants, once their project is retained for funding based on the Horizon 2020 selection criteria, to search in the course of the project for complementarity, coherence and synergies with other funding, including ESIF in the context of smart specialisation at national and regional levels, with the purpose to facilitate the exploitation and uptake of project results. It is in this context that in some cases this

encouragement has resulted in projects that facilitate synergies. 239 As part of a study

commissioned by DG RTD to collect information and data on the development of synergies, a sample of Horizon 2020 Thematic Work Programmes has been reviewed

from the perspective of synergies 240 . The results show that most of the Work

Programmes reviewed refer to the links with ESIF. However, the level of synergy specification is varied; in some cases, there is guidance in the main text of the Work Programme and in other cases, the references to ESIF and synergies are more limited.

At project implementation level, the emergence of 'upstream synergies' is noticeable, through actions that build R&I capacities of actors aimed at participating in Horizon 2020, for example through research infrastructures, science parks, training, networking actions. Teaming phase 2 is instead a concrete example of the 'parallel' synergy approach, since regional and national authorities commit to invest significant resources (normally ESIF) complementary to the Horion2020 contribution in setting up Centres of Excellence. Teaming proposals must clearly identify alignment and complementarity with the national or regional Smart Specialisation Strategies of the Member State or region from which the applicant is coming. In the other two Widening actions - Twinning and ERA Chairs - interactions with national/regional authorities and other stakeholders to coordinate actions and mobilise support, e.g. from ESIF, are desirable in the proposals, and the proposals are encouraged to identify alignment and complementarity with Smart Specialisation Strategies.

Synergy cases

Examples of upstream synergies through actions that build R&I capacities of actors aimed at participating in Horizon 2020 (for example through research infrastructures, science parks, training, networking actions)

The ELI - Extreme Light Infrastructure is an example of the upstream synergies located in Czech Republic, Hungary and Romania. It is supported by these Member States under their ESIF

238 Specific guidance is available for the Commission services responsible for the preparation of Horizon 2020 Work

Programmes on the encouragement of synergies in the Work Programmes (a section on synergies in an internal guidance document)

239 For example, in the Horizon 2020 LEIT/KETs part, the projects EU-GREAT (http://eu-great.com/) and

SYNAMERA (http://www.synamera.eu/), and in the Horizon 2020 SC1 Health the projects DanuBalt

(http://danubalt.eu/ ) and RegHealth-RI (http://rhing-net.eu/reghealth-ri/)

240 The main challenge is that encouragement of synergies upfront in Work Programmes must not create misleading

expectations, as the evaluation system of Horizon 2020 cannot 'reward' synergies (with higher scores). The reasons for this are two-fold: the excellence criterion of Horizon 2020 cannot be undermined, and many potential Horizon 2020 beneficiaries are located in the Member States and regions with low, and sometimes negligible, ESIF funding available for R&I, and therefore, rewarding synergies in the evaluation system would result in an unequal treatment of such beneficiaries.

programmes complementing the resources received under the European Strategy Forum on Research Infrastructures, ESFRI. The European Spallation Source (ESS) is also part of the ESFRI and receives support for its construction costs from ESIF. In the current programming period, around EUR20 million of ESIF in Sweden will be allocated to this research infrastructure. Also some ESS partner countries such as Estonia and Czech Republic are applying the new provision in the Cohesion Policy regulation (Article 70) that allows regions to spend part of their ERDF allocations (up to 15%) in other regions, even abroad, and thus these countries invest from ESIF in the construction of the ESS.

The PLATFORM initiative, funded by Horizon 2020 and coordinated by Technalia, the Basque Region in Spain, aims to develop open access pilot lines for the industrial production of specialised materials for composite applications in sectors such as aerospace and automotive. During the technology development phase, a predecessor initiative received support from the Structural Funds and was involved in two FP7 projects that developed methods to manufacture multifunctional composites. The core PLATFORM group is composed of 12 partners (public and private organisations) coming from Spain, Poland, Belgium, Greece, Italy and United Kingdom. The consortium comprises five RTOs, three large enterprises and four SMEs that represent the whole value chain for the development of the new nano-enabled products production facilities. PLATFORM will specifically provide SMEs with open access to these facilities for the purposes of direct product purchase, product development, collaborative research as well as training. The PLATFORM initiative will continue to combine different funding sources as part of its activities to provide services to existing pilot projects that cover innovation and commercial activities.

Examples of the initiatives targeting the wide range of synergy-types (upstream, downstream, parallel/complementary)

Clean Sky 2, the Public-Private Partnership between the European Commission and the European aeronautics industry, encourages synergies with ESIF by allowing complementary activities to be proposed by the applicants to Clean Sky Joint Undertaking's (CSJU) calls and by amplifying the scope, adding parallel activities or continuing CSJU co-funded project/activities through ESIF in synergy with the Clean Sky 2 Programme and its technology roadmap. In that regard, the initiative has launched an action plan on synergies in close cooperation with interested Member States and regions. Memoranda of Understanding with 12 MS/regions have been signed. This type of cooperation strengthens the research and innovation capacity and the European dimension of regions specialised in aeronautics; identifies complementary areas of technical cooperation and achieves a leverage effect from synergies between ESIF and the Clean Sky 2 funding. A dedicated 'Clean Sky 2 Synergy Label' has also been developed. It functions as an incentive and 'guarantee of success' for MS/Regions to invest in projects and support actions as well as infrastructures in favour of well performed/on-going actions of the initiative. The Bio Based Industries and the ECSEL JTIs/Joint Undertakings are also very actively seeking for synergies with ESIF and other JTIs/JUs are exploring their opportunities.

Example of the 'downstream' synergies, where national ESIF Managing Authorities/Research and Innovation Authorities/Agencies favour actions that capitalise on existing (on-going or finished) FP7/Horizon 2020 R&I actions aimed at market up-take /commercialisation

The FP7 project TIGER (Transit via innovative gateway concepts solving European intermodal rail needs) supported the development of competitive European rail transport and co-modal freight logistics chains. TIGER DEMO , the follow-up project, aimed at taking the pilots developed by TIGER forward into a full-scale demonstration for subsequent market uptake and commercial exploitation. One of these demonstration sites is the City of Genoa in Italy, where the construction of the ‘Genoa Fast Corridor’ was partially funded by ESIF of the Liguria region.

Examples of synergies in the coordinated parallel actions that complement FP/Horizon 2020 projects

In the past years, the Central European Institute of Technology CEITEC in the South Moravia Region (Czech Republic) has implemented its strategy and activities with funding coming from multiple sources, including Horizon 2020 and ESIF. Another example is the Danube:Future cooperation initiative that contributes to the EU Macro-regional Strategy for the Danube Region by developing interdisciplinary research and education in the area, in particular, strengthening a long-term humanities’ perspective. The initiative funds its activities from multiple sources including Horizon 2020 and ESIF and aims to have a lasting effect on research and teaching in the humanities in the region.

The Seal of Excellence is another example of concrete efforts to create synergies between different programmes. The purpose of the Seal of Excellence ('Seal') is twofold: to help SMEs in obtaining alternative funding for their project idea and, at the same time, provide the funding bodies interested in investing in R&I - including the Managing Authorities (MA) of European Structural and Investment Funds (ESIF) - with 'ready-to-fund' high-quality project proposals, already screened by Horizon 2020, they can invest their resources in. The Seal has been firstly applied to the "SME Instrument" of Horizon 2020. Until October 2016, a total of 4 613 Seal of Excellence certificates have been delivered and they have been distributed between the two phases of the SME Instrument calls as follows: 1 446 for Phase 1 and 3 167 for Phase 2. At national level, the majority of certificates have been delivered to proposals of SMEs located in Spain, Italy and United Kingdom with more than 500 certificates delivered to proposals of SMEs in each one of them, followed by Germany, France and Israel with more than 200 certificates and Finland, the Netherlands, Sweden, Denmark and Hungary with 100 or more certificates delivered. At regional level, the most certificates have been delivered to proposals of SMEs locating in Catalonia (Spain) with more than 250 certificates, followed by Lombardy (Italy), Madrid (Spain) with 150 or more certificates while Emilia-Romagna (Italy), Valencian Community (Spain), Ile-de-France (France) and Helsinki-Uusimaa (Finland) have received 100 or more certificates each. Regarding thematic call topics, the highest number of Seal were assigned in the field of ICT (up to October 2016 call cut-offs) with more than 1000 Seals delivered to Phase 1 and Phase 2 proposals followed by Nanotechnology and Biotechnology with more than 500 certificates delivered.

Upon request, the Commission provides countries/regions with aggregated information on the number of proposals per country/region that have received the Seal. So far alternative funding bodies in 20 countries/regions have requested such data as it provides an approximation on the overall volume of financial resources needed for designing and launching alternative funding schemes for the Seal proposals. It is up to the country or region to establish supporting funding schemes that are dedicated to these types of proposals and enable the provision of alternative funding, in compliance with national and EU rules.

To support alternative funding bodies in implementation of the Seal, the European Commission set up in 2015 a Community of Practice (CoP) as a forum for exchange of information, experiences and discussion of bottlenecks. It works through regular meetings and an IT platform. The membership is restricted only to managing authorities and any public or private bodies with funding power committing to implement 'Seal' friendly measures. Currently, the CoP comprises 198 members representing 25 Member States and two Associated Countries. The only missing Member States are Malta, Bulgaria and Luxemburg. 95% of the members come from public organisations, 5% works in the private sectors. 56% of the members are working for a national institution, 41% for a regional institution and the other 3% are not geographically defined. Since the launch of the Seal, four Community of Practice meetings have taken place where concrete implementation modalities have been discussed and solutions have been proposed for possible barriers. The CoP meetings are taking place in Brussels and are coorganised by DG RTD and REGIO. In 2017, the CoP will be further developed with a view of establishing a 'Club of Seal SMEs' to exchange experiences and best practices.

Overall, the reactions by alternative funding bodies in countries and regions to the Seal of Excellence initiative are very positive. An increasing number of national / regional

schemes 241 supporting SME Instrument proposals that have received the Seal have been

launched (in, Cyprus, Finland, , Hungary, Ile de France, Italy (several regions), South Moravia, Slovenia, Spain, Sweden, and Norway. Up to November 2016, Commission is aware of more than 260 Seal proposals have been funded by alternative sources or they are in the pipeline to receive funding, a number which is steadily increasing given that there are schemes currently open or under final stage of preparation. Also SME applicants who have received the Seal for their proposals have reacted very positively to this new initiative.

In order to tap into the full potential of the Seal of Excellence initiative, Seal proposals should ideally receive from other alternative funding sources the same funding intensity as is provided by Horizon 2020 and with the minimum administrative burden. However, the authorities in charge of the management of ESIF may decide for a reduced funding intensity. The main source of alternative funding for Seal proposals is ESIF, but, contrary to Horizon 2020, ESIF must comply with the State Aid rules. In order to simplify the procedure for the ESIF Managing Authorities and reduce their administrative burden in verifying the compliance of Seal proposals with the State Aid rules, the Commission

(DGs RTD, REGIO and COMP) has prepared an explanatory note. 242 Despite the

Commission's to smoothen the process, in the context of the current regulatory framework, the perception of some constituencies is that compliance with State Aid rules adds some administrative burdens.

DG REGIO has set up a Task Force on Simplification that also explored Horizon 2020 approaches as a source of inspiration for ESIF. The respective regulations already provide for the harmonisation of some rules and procedures: for example, the same eligible cost base as for Horizon 2020 can be used for ESIF projects. ESIF Managing Authorities can support operations outside their programme area within a limit of 15% of the total amount allocated at the level of the priority, when this is for the benefit of the programme area in question. This was designed for instance to allow pooling of funding for ESFRI research infrastructures. The Commission services (DGs RTD & REGIO) will continue to exchange experiences in order to find concrete solutions as regards simplification.

241 Nine (9) national and/or regional 'seal friendly' funding schemes for Phase 1 proposals are today operational in

Spain, Italy (Lombardy and Friuli-Venezia Giulia), Sweden, Norway, Czech Republic (South Moravia), Cyprus, Hungary and Finland and six (6) more are under consideration and/or preparation in Spain (Asturias), United Kingdom (Scotland), Czech Republic (national), Belgium (Brussels Capital), Poland and Slovenia. Six (5) 'seal-friendly' funding schemes for Phase 2 using ESIF have been already launched. The Italian government approved a scheme covering the 8 southern regions which was published in mid-July 2016. A second scheme was launched in Cyprus in mid-September 2016 for both phase I and II Seal proposals while France (Ile-de-France) launched in October 2016 a scheme targeting also Seal SME phase II proposals. Recently, Spain and the region of Pidemont, in Italy published calls for Seal of Excellence SME Instrument Phase 2 proposals and Slovenia for Phase 2. Six (7) additional national/regional schemes (mainly Ph2) are under consideration and/or under preparation in Italy (Lombardy), Belgium (Brussels Capital), Italy (Province of Trento), United Kingdom (Scotland), Poland and Finland.

242 Explanatory note of the Commission services on the application of State Aid Rules to national and regional funding

schemes that offer alternative support to SME Instrument project proposals with a Horizon 2020 'Seal of Excellence' http://ec.europa.eu/regional_policy/sources/docgener/guidelines/2017/application_of_state_aid_rules.pdf

Tracking of synergies is an important element for continuously monitoring and gathering information about developments in the field of synergies. Currently, a major study funded by DG REGIO is on-going on the co-ordination and harmonisation of ESI Funds and other EU policies. One central part of the study is focused on the interaction of ESI and other EU Funds, and the results of this part of the study will be available in January 2017. The Commission (DGs RTD & REGIO) will continue to run a number of specific surveys with Horizon 2020 beneficiaries and ESIF Managing Authorities using the Commission's available online tools. The Commission is however exploring possibilities for setting up a more comprehensive monitoring system for synergies.

DG RTD has also commissioned a study which collected – through means of surveys and interviews – information and data on the overall development of synergies between Horizon 2020 and ESIF. The study provided evidence, inter alia, of the following: there is a clear legal basis for synergies in place; there are overall implementation guidelines for all the institutional actors involved; there is a good general knowledge among Horizon 2020 NCPs of the opportunities provided by synergies. However, the study concluded that the overall development of synergies is considered by the stakeholders (NCPs, Horizon 2020 beneficiaries, ESIF Managing Authorities and the European Commission officials) as variable, occasional and rather based on a chance than on a more systematic process. The development of synergies is still at early stage, especially in view of the strong political support given to the policy and related expectations. The study recommended that all the stakeholders involved in the promotion of synergies should thus assume their responsibilities to improve the strategic framework, communication, coordination and support to this policy. Clearer definition of roles between institutional actors would also allow a more effective and focused support to the generation of synergies. Overall, there is a need to improve the clarity of concepts and definitions as well as objectives and expected impacts. There is a need to enhance communication, awareness raising, coordination and monitoring of this policy. More concrete support should also be provided, i.e. support that is tailored to specific needs of a Member State and/region, for the processes aimed at designing and developing synergies as well as for operational implementation of synergies, for example, by showcasing good practices.

Moreover the PSF initiatives which overlap with policy areas in the remit of other Commission services, such as DG CNECT, DG REGIO, DG GROW or EEAS, are implemented in partnership with these DGs. These joint activities significantly boost the momentum which the PSF has created around the need for Member States to reform their research and innovation eco-systems in order to increase their quality and impact on economic growth and societal progress.

Figure 254 - Internal coherence of SEWP with other Horizon 2020 specific objectives

European Research Council

EIT KICs 1,00 Future and Emerging technologies

SWAFS 0,80 MSCA

SC7 0,60 Research

0,40 infrastructures

SC6 0,20 LEIT ICT

0,00

SC5 LEIT NMBP

SC4 LEIT Space

SC3 Access to risk finance

SC2 Innovation in SMEs

SC1

Complementarities/Synergies Overlaps Gaps

Source: EC Internal survey.

The Likert's tool applied to SEWP does not represent adequately the complexity of the synergies and the internal and external coherence. With reference to the internal coherence, as mentioned at p. 50, SEWP rather than overlapping with other Horizon 2020 intervention areas, it improves the capacity of R&I stakeholders in low performing/Widening countries to compete for competitive funding, namely the capacity to apply to other interventions areas of Horizon 2020.

With reference to the external coherence, the tool depicts a distorted picture. The reason is that the scale looks at 'Complementarities/Synergies related to the specific coverage of initiatives' however does not analyse the geographical coverage of the compared programmes. In this specific case both SEWP and ESIF are bottom-up programmes, meaning that they can cover all possible domains. However, despite the thematic coverage might be similar, the overlapping between the two programme is limited due to the different geographical coverage of the two instruments, i.e. the ESIF funds mainly national or regional interventions and only a small part of the budget goes into interregional cooperation (INTERREG has a budget of around EUR 1.6 billion for R&I related cooperation, mostly for trans-national cooperation, but also for mutual-learning among policy-makers across all regions), Widening is based on transnational partnerships and networking, also helpings to pool resources and achieve synergies at the European scale, however, with stronger focus on increasing scientific excellence, networking and policy reforms. Furthermore, for the ESIF the selection of types of R&I intervention can vary among regions and Member States, according to the strategic choices made in each territorial context.

So, while the thematic coverage might be similar (this varies significantly according to regions and Member States because for ESIF they are free to choose the type of intervention that are most appropriate to their territories), the type of action supported and the geographical dimension, regional/national versus EU/international cooperation, is different and can be complementary.

Q.6.2.2. Coherence with other public support initiatives at regional, national and international level

Actions aiming to reduce the innovation gap in Europe are implemented through national programmes of some associated countries (Switzerland, Norway) or multilateral programmes (EEA grants). The experience from the implementation of these programmes (bilateral programmes of Switzerland with Bulgaria, Romania, etc.; initiatives aiming to reverse the brain drain of young researchers, etc.) is not known since there any established mechanism of coordination.

Informal exchange of information and experience with the coordinators of some of these programmes is currently under way. However, the analysis on how the actions complement each other or not, is not available and would be useful to explore if the policies are working in a coherent manner (overall policy-mix) to deliver on the objectives and to know if they are having undesired impacts that counteract/undermine/support SEWP actions.

When looking at other EU programmes, SEWP and ESIF, it is important to note that both programme have a large and open coverage of the different R&I dimensions. For the ESIF the selection of specific type of intervention can vary among regions and Member States, according to the strategic choices made in each territorial context. However there is overlapping between the two programme as they follow two different approaches, i.e. the ESIF fund national or regional interventions, while Widening is based on transnational partnerships and networking and hence helps to pool resources and achieve synergies at the European scale.

Q.6.3. Lessons learnt/Areas for improvement

The internal coherence of the SEWP programme is ensured by the complementary character of the different actions as they address different needs and scales.

When looking at the perception of the beneficiaries, ERA Chairs and Twinning coordinating research institutes confirms that they are already used to combine different funds. They have already received funding from other Horizon 2020 sources such as ERC and Marie Curie grants together with Research Infrastructure. Societal Challenge, FET and LEIT are to a minor extent included in the complementary funding received these beneficiaries. Regarding other regional, national or international funding, these funds play a crucial role in covering the cost of research. For Twinning the list of complementary funding is even longer and includes more innovation related funds.

Widening 's design already entails synergies with ESIF in particular for Teaming where applicants are obliged to ensure appropriate co-financing for building centres of excellence from the ESIF or other sources.

On the complementary between Widening actions and ESIF, both programmes have a large and open coverage of the different R&I dimensions. For the ESIF the selection of specific type of intervention can vary among regions and Member States, according to the strategic choices made in each territorial context. However, the overlapping between the two programme is limited due to the different geographical coverage of the two instruments, i.e. the main part of ESIF funds national or regional interventions with only around 1.6 billion funding trans-national and inter-regional cooperation on R&I often among policy-makers, while Widening is based on transnational partnerships and networking and helps to pool resources and achieve excellent and connected research capacity at the European scale.

The work on harmonisation of ESI Funds and other EU policies including Horizon 2020 is ongoing and will be relevant to increase synergies in implementations. At the same time the support to the implementation of synergies requires by the R&I ecosystem requires more attentions and further collaboration among Commission's services.

Q.7. EU A DDED V ALUE

Q.7.1. Horizon 2020 projects demonstrating EU Added Value

HURMUR

EC Contribution: EUR 1.014.675

Start date: 01.01.2016

The project “HURMUR: Human rights – mutually raising excellence” contributes to expand the world-class research area of human rights in Europe. The project aims at establishing the Fundamental Rights Research Centre at Tallinn University as part of a global network of scholars researching new human rights, i.e. rights related to the exercise of public authority; to new technologies; to identity and personality; to wellbeing. According to the coordinator, the Twinning call is an absolutely unprecedented instrument to decisively accelerate the development of research capabilities of academic institutions in areas important for Europe. The effects of the project are likely to have global impact.

Tallinn University Law School is developing research excellence of human rights and is already involved in global research and regional dissemination networks through specific activities of an outstanding consortium, where two other partners are premier global academic institutions in the field of human rights – the Danish Institute for Human Rights and Walther Schücking Institute of International Law (Kiel University, Germany).

Main objectives of the project are those of developing capacities to i) become a leader in the Baltic region of participating in state-of-the art research of human and fundamental rights, ii) initiate new research and development project of European magnitude exploring the changing nature of human rights in the contemporary society; iii) build bridges between Estonian/Baltic/Russian human rights research and activist communities. The project's objectives will be achieved through several activities which include the: i) organizational reform of Tallinn University’s International Research Centre of Fundamental Rights; ii) establishment and publication of a new regional peerreviewed academic journal, the East European Yearbook on Human Rights; iii) strengthened research capacities of TLU on universality of human rights (including in particular freedom of speech, freedom of religion, and identity rights), and human rights narratives and discourses, and on new human rights as well as increased publication activity in this area; iv) increased dissemination and communication of TLU research and knowledge to the public, including academia, policy makers, law makers and civil society.

Q.7.2. Other issues related to EU Added Value

There is an undisputable EU added value for the programme because none of the schemes which are based on EU partnership with excellence R&I institutions could have been implemented effectively in an isolated national approach. The programme has a structuring effect on EU research and wider innovation policy because it helps to unlock the full potential of all Member States and relevant Associated Countries. Each individual scheme is based on EU partnerships and networking and hence helps to pool resources and achieve synergies at the European scale.

The IETEC survey clearly indicates that ERA Chairs project coordinators and Chair holders believe that the projects will result in EU added value. This is perceived as resulting in an increased attractiveness for other high profile researchers to work at the institutions. 88% expect the projects to result in EU additional value to a very high degree or to a high degree. Only 3% of the respondents do not expect any EU added value from the projects.

Respectively 49% and 41% of the survey's respondents expect that the Twinning projects will in a very high or high degree result in an EU added value compared with what would be the case if the projects were funded at regional or national level. The most often mentioned added value is the networks and collaboration with other major institutions. Because of the projects being EU funded, it has been possible to establish this collaboration. The project coordinators pointed out the international aspect and mentioned that if these projects were locally funded they would run at a national level and collaboration would be with other local institutions. Furthermore, these collaborative networks are expected to last after the end of the projects.

Looking at the participation of advanced countries, currently the figures show a high degree of participation. It will be important to keep these countries motivated in order to allow that the partnership principle, which is at the basis of most widening actions, remains the key principle of this approach.

When analysing the added value of the Horizon 2020 Policy Support Facility (PSF)it is relevant to acknowledge that despite widespread agreement on the need to improve national R&I policies and funding, the actual process of undertaking these reforms is extremely complex for various reasons, and notably: the limited evidence base; a lack of in-house expertise in the administrations designing, implementing or evaluating the reforms; and a need to tailor policies to country specific circumstances. The PSF helps overcome these hurdles by permitting access to the right expertise and advice, benefitting from insight of international experts and using state-of-the-art methodologies and tools. The PSF allows matching countries' needs with the most relevant expertise and peer experiences available across the European Research Area. It provides a systematic but customised approach to the countries' requests while previously ad-hoc mechanisms needed to be put in place to deal with them. The concentration of activities under the PSF allows also a learning process in which each new activity can capitalise on the experiences gained in previous activities (for instance, in the activities recently launched, the processes in terms of meetings and reports have been adapted to take into account the experience gained in the earlier projects.) The streamlined PSF website (the PSF Knowledge Centre) where countries are able to find all of the information and data about their R&I performance and policies in a single repository has been also perceived as an added value by the national authorities.

Q.7.3. Lessons learnt/Areas for improvement

Within ERA Chairs project the EU added value is perceived as increased attractiveness for other high profile researchers to work at the institutions. Similarly the Twinning projects consider EU added value the networks and collaboration with other excellent R&I institutions in EU.

PSF's support to the reform process is extremely complex for various reasons, and notably: the limited evidence base; a lack of in-house expertise in the administrations designing, implementing or evaluating the reforms; and a need to tailor policies to country specific circumstances. The PSF helps overcome these hurdles by permitting access to the right expertise and advice, benefitting from insight of international experts and using state-of-the-art methodologies and tools.

Q.8. L ESSONS LEARNT /C ONCLUSIONS

Q.8.1. Relevance

Key findings:

 The relevance of SEWP is confirmed by recent figures that depict still a significant underrepresentation of EU-13 in successful applications to Horizon 2020 and budget allocation. Notably 8,5 % of the participants are coming from these countries while 4,5% of EU 13 budget allocation.

 Widening countries (as defined by the composite indicator) are affected by these problems to different extents and intensity, showing that the group is not a homogeneous block and that the dichotomy widening-non widening countries as it stands now is a strong simplification of the reality.

The strengths are:

 Most of the main reasons for Widening disparities (insufficient R&D investments in those countries, lack of synergies between certain countries’ national research systems and EU research, lagging system learning effects and access to existing networks, differential wage levels between countries as well as insufficient and ineffective information, communication advice and training) are tackled by the programme with the exception of the differential wage levels which is a largely a national issue.

 SEWP integrated approach to the multifaceted dimensions of the innovation divide.

The bottlenecks/weaknesses are:

 Supporting world-class excellence requires long-term commitment, and structural reforms need time, national commitment and continuity.

Q.8.2. Effectiveness

Key findings:

 ERA Chair and Twinning projects already resulted in substantially increasing the attractiveness of the institution for international excellent researchers, the capability of the institution to compete for international funding.

 Additionally for ERA Chair the research excellence in the field of the chair holder. ERA Chairs projects are improving the institutions recruitment procedures that have become more transparent after receiving the ERA Chairs grant.

 COST actions demonstrated effectiveness in including excellent researchers from Widening countries with a steadily increasing participation rate.

 The recurrent feedback on the PSF work received from national policy-makers and stakeholders has shown that the operational recommendations formulated by leading experts and policy practitioners prove valuable as catalysers of national R&I reforms.

The strengths are:

 SEWP integrated approach to the multifaceted dimensions of the innovation divide.

 Good results at the early stage of implementation with positive spill over effects (scientific output, partnerships and international openness, policy output and effective R&I ecosystem).

The bottlenecks/weaknesses are:

 SEWP integrated approach to the multifaceted dimensions of the innovation divide requires effective programme coordination in implementation, monitoring and evaluation.

 The majority of Twinning and ERA Chairs co-ordinators consider the ineligibility of equipment and consumable costs as an obstacle for the effectiveness and speed of the actions.

Q.8.3. Efficiency

Key findings:

 Smooth implementation.

The strengths are:

 Teaming leveraged significant amounts of ESIF funding, as proposers were expected to mobilise from national/regional public funds (including ESIF) at least the same amount as the Horizon 2020 requested contribution.

The bottlenecks/weaknesses are:

 Twinning project coordinators mentioned that understanding the definitions of eligible travel costs in the Twinning call is difficult.

 For ERA Chairs, bureaucracy as an obstacle to overcome, e.g. obtaining permits and visas when hiring foreign researchers to the research teams, other than the chair holder.

 Teaming coordinators would like to receive stricter guidance on the key concepts (Centre of Excellence, structure and legal form, scientific and financial sustainability, scientific scope, balance between science and innovation).

 Oversubscription of Teaming, Twinning and COST actions.

Q.8.4. Coherence

Key findings:

 Strong internal and external coherence

The strengths are:

 External coherence with other policy strands of the EU notably cohesion policy, the Innovation Union as a part of the Europe 2020 strategy and the European Research Area (ERA).

 Beneficiaries combine funds from different sources (regional, national, EU).

 Growing awareness for better including researchers from widening countries expressed by some initiatives within the thematic programmes addressing Societal Challenges (e.g. the European Research Council). In addition, the Seal of Excellence widens the range of Horizon 2020 towards innovative SMES and will be extended to excellent candidates for ERC grants and Marie Sklodowska Curie in the near future.

 The coherence of the Horizon 2020 Policy Support Facility with EU economic policy is ensured as the analyses developed in the context of the European Semester, in particular the identification of the bottlenecks to the contribution of R&I to growth and job creation, constitute the basis for the PSF activities.

The bottlenecks/weaknesses are:

 Different rules due to the diversity of regulatory frameworks (ESIF, State Aid Regulation, etc.)

 The support to the implementation of synergies by the R&I ecosystem requires more attentions and further collaboration among Commission's services.

Q.8.5. EU Added Value

Key findings:

 Undisputable EU added value for the programme because none of the schemes could have been implemented effectively in an isolated national approach.

The strengths are:

 The programme has a structuring effect on ERA integration and wider innovation policy because it helps to unlock the full potential of all Member States and relevant Associated Countries.

 Each individual instrument develops new trans-national partnerships and networks and hence helps to pool resources and achieve synergies at the European scale.

 The programme helps to unlock the full potential of the European R&I system by breaking silos and opening up formerly closed shop networks.

The bottlenecks/weaknesses are:

 The (lack) of motivation of advanced partners.

• R. 

  S CIENCE WITH AND FOR SOCIETY

  R.1. I NTRODUCTION

  R.1.1. Context

The ex-ante Impact Assessment of Horizon 2020 243 underlined that "a model that is at once

sustainable, inclusive and smart will not depend solely on S&T but also on governance and on the involvement of the citizens who will make up our society – and shape it". To this end, "A shift towards "the demand side" together with users’ (and more broadly citizens’) involvement is not only a prerequisite for more robust and flourishing technologies; it is also a prerequisite for more robust and flourishing societies".

The ex-ante Impact Assessment went on to make the case that solutions to societal challenges are not always based purely on science and technology, but require stronger interactions with citizens and consumers, not least because they are most affected by the outcomes of S&T: "they require and expect high quality health care and solutions to fatal and debilitating illnesses; they hope that science and innovation can tackle problems such as climate change, clean energy, clean transport, an ageing population; and they look to Europe’s research and innovation system to come up with new sources of jobs and higher standards of living". It went on to call for research and innovation (R&I) to focus on major challenges in society such as climate change, energy and resource efficiency, health and demographic change, and to strengthen links between the scientific community and society. More recently, and at the global level, there was similar recognition of these needs by the United Nations: "Technology, society and institutions co-evolve. Hence, technology progress requires institutional

adaptations and may be constrained by social issues" 244 .

The Horizon 2020 Decision 245 came with features designed to respond directly to these points.

For instance, the societal challenges section aims to "increase the effectiveness of research and innovation in responding to key societal challenges by supporting excellent research and innovation activities. Those activities should be implemented using a challenge-based approach which brings together resources and knowledge across different fields, technologies and disciplines".

However, the Science with and for Society (SWAFS) part of Horizon 2020 is the main programme part dealing with these issues. The Horizon 2020 Decision states that "The strength of the European science and technology system depends on its capacity to harness talent and ideas from wherever they exist. This can only be achieved if a fruitful and rich dialogue and active cooperation between science and society is developed to ensure a more responsible science and to enable the development of policies more relevant to citizens. Rapid advances in contemporary scientific research and innovation have led to a rise of important ethical, legal and social issues that affect the relationship between science and society. Improving the cooperation between science and society to enable a widening of the social and political support to science and technology in all Member States is an increasingly crucial issue… Public investment in science requires a vast social and political constituency sharing

243 http://ec.europa.eu/research/horizon2020/pdf/proposals/horizon_2020_impact_assessment_report.pdf .

244 United Nations, 2016, Global Sustainable Development Report 2016, Department of Economic and Social Affairs, New

York, July.

245 http://ec.europa.eu/research/participants/data/ref/Horizon 2020/legal_basis/sp/Horizon 2020-sp_en.pdf .

the values of science, educated and engaged in its processes and able to recognise its contributions to knowledge, society and economic progress".

The Decision also highlighted two cross-cutting issues (CCIs) of particular relevance to SWAFS. Gender is "addressed as a cross-cutting issue in order to rectify imbalances between women and men, and to integrate a gender dimension in research and innovation programming and content". Secondly, "The relationship and interaction between science and society as well as the promotion of responsible research and innovation [RRI], science education, science communication and culture shall be deepened and public confidence in science and innovation reinforced by activities of Horizon 2020 favouring the informed

engagement of and a dialogue with citizens and civil society in research and innovation" 246 .

This thematic assessment on Science with and for Society is informed by the work of the Horizon 2020 Commission expert group on the interim evaluation of Science with and for

Society and Responsible Research and Innovation in Horizon 2020 247 . It is based on analysis

of existing studies and CORDA data, analysis of all projects followed up by selected interviews with project co-ordinators, and evaluation activities carried out by the European Economic and Social Committee (EESC) on Horizon 2020 and SWAFS. The work was carried out from October 2016 to January 2017.

R.1.2. Objectives and intervention logic

The three 'overarching' objectives for SWAFS are described in the Horizon 2020 Decision:

• 1. 

  Build effective co-operation between science and society; 2. Foster the recruitment of new talent for science; 3. Pair scientific excellence with social awareness and responsibility.

Furthermore, SWAFS is expected to contribute to the general objectives of Horizon 2020 to promote job creation and economic growth, address people’s concerns about their livelihoods, safety and environment, and strengthen the EU’s global position in research, innovation and technology.

The Horizon 2020 Decision describes eight specific SWAFS lines of activities. These are:

• a. 

  Make scientific and technological careers attractive to young students, and foster sustainable interaction between schools, research institutions, industry and civil society organisations ['science careers'];

• b. 

  Promote gender equality, in particular by supporting structural changes in the organisation of research institutions and in the content and design of research activities ['gender equality'];

• c. 

  Integrate society in science and innovation issues, policies and activities in order to integrate citizens' interests and values and to increase the quality, relevance, social acceptability and sustainability of research and innovation outcomes in various fields of activity from social innovation to areas such as biotechnology and nanotechnology ['public engagement'];

246 Gender as a CCI and RRI as a CCI are both subjects of separate annexes as part of the Interim Evaluation of Horizon

2020.

247

http://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupDetail&groupID=3451&news=1&Lang=EN .

• d. 

  Encourage citizens to engage in science through formal and informal science education, and promote the diffusion of science-based activities, namely in science centres and through other appropriate channels ['science education'];

• e. 

  Develop the accessibility and the use of the results of publicly-funded research ['open access/open data'];

• f. 

  Develop the governance for the advancement of responsible research and innovation by all stakeholders (researchers, public authorities, industry and civil society organisations), which is sensitive to society needs and demands, and promote an ethics framework for research and innovation ['governance and ethics'];

• g. 

  Take due and proportional precautions in research and innovation activities by anticipating and assessing potential environmental, health and safety impacts ['due and proportional precaution'];

• h. 

  Improve knowledge on science communication in order to improve the quality and effectiveness of interactions between scientists, general media and the public ['science communication'].

Comparison between FP7's Science in Society (SiS) and Horizon's 2020 SWAFS

There is marked continuity between FP7's SiS and Horizon 2020's SWAFS, with 22 out of 31

SiS items continued in SWAFS activities. 248

Seven SiS items were discontinued in SWAFS:

 The reciprocal influence of science and culture;  The role and the image of scientists;  The provision of reliable and timely scientific information for the press and other

media;

 Training actions to bridge the gap between the media and the scientific community;  Promoting science by audio-visual means via European co-productions and the

circulation of science programmes;

 Structuring activities;  Operation of the European Research Area Board.

Two SiS items were taken up by other units in DG Directorate-General for Research & Innovation (DG-RTD) or other parts of Horizon 2020:

 Improving the use, and monitoring the impact, of scientific advice and expertise for

policy making in Europe (including risk management) and developing practical tools and schemes (e.g. electronic networks), which is to some extent taken up by a

dedicated unit dedicated to the Scientific Advice Mechanism within DG-RTD;

 Foresight in Science and Society, which is taken up by unit A6 in DG-RTD.

Two new items were added to SWAFS:

 Make scientific and technological careers attractive to young students, and foster

sustainable interaction between schools, research institutions, industry and civil society organisations;

248 Ex-post Evaluation of Science in Society in FP7 - Final Report, page 13. The Stock-tacking and meta-analysis of Science

in Society projects throughout FP6 and FP7 study also finds considerable continuity between FP6's Science and Society

(SaS) and FP7's Science in Society, and finds that focus on the relationship between science-society can be traced back as far as FP4 despite lack of a dedicated programme.

 Take due and proportional precautions in research and innovation activities by

anticipating and assessing potential environmental, health and safety impacts.

One item was adapted from SiS as a CCI for the whole of Horizon 2020:

 Responsible Research and Innovation: Encouraging societal actors (researchers,

citizens, policy makers, businesses, third sector organisations, etc.) to work together during the whole research and innovation (R&I) process in order to better align R&I

and its outcomes with the values, needs and expectations of society.

Intervention logic

To address the identified objectives, SWAFS adopted the intervention logic shown in Figure 255.

Figure 255 - SWAFS Intervention logic

1544

R.2. I MPLEMENTATION STATE OF PLAY

R.2.1. Overview of programme inputs and activities

As of 1 January 2017, the state of play is the following. The EC contribution allocated to the implementation of the calls included in SWAFS Work Programmes 2014-2016 and which were closed was EUR 139.22 million. This is 32.2% of the total expected budget allocated to SWAFS in Horizon 2020, which is EUR 432.7 million for 2014-2020. The budget was allocated through 30 topics in 12 closed calls for proposals. These are detailed in Table 250. Three projects are completed and 50 are on-going. The programme has so far been implemented through 41 (77.4%) Co-ordination and Support Actions (CSAs) and 12 (22.6%) Research and Innovation Actions (RIAs).

This means that 67.83% (EUR 293.48 million) of the SWAFS budget remains to be allocated. The budget allocation to SWAFS was reduced from EUR 462.2 million (Regulation of the Council) to EUR 432.7 million – an overall budget reduction of 6.38% as a result of adaptations to the budget as part of the European Commission’s

"Investment Plan" 249 .

SWAFS is characterised by a high level of externalisation: 44 projects (83% of total grants) are managed by the Research Executive Agency (REA). Five DG-RTD units are involved in SWAFS (B7, B2, A6, A3, 01 – See Section R.4).

Table 250 - Overview of results from topics included in closed calls for proposals

Topic code # of Project acronym(s) Total requested Average

projects EC requested EC

contribution contribution

(EUR million) (EUR million)

GARRI-1-2014 2 PROSO, FoTRRIS 3,089,312.50 1,544,656.25

GARRI-2-2015 3 PRISMA, SMART-map, COMPASS 4,723,676.88 1,574,558.96

GARRI-3-2014 1 FutureTDM 1,492,370.00 1,492,370.00

GARRI-4-2015 1 OpenUp 1,951,932.50 1,951,932.50

GARRI-5-2014 1 PRINTEGER 1,987,779.75 1,987,779.75

GARRI-6-2014 1 TRUST 2,141,173.25 2,141,173.25

GARRI-7-2014 1 SiS.net2 1,999,593.75 1,999,593.75

GARRI-8-2014 1 NCP ACADEMY 1,967,828.00 1,967,828.00

GARRI-9-2015 1 DEFORM 999,712.50 999,712.50

GARRI-10-2015 1 ENERI 1,499,000.00 1,499,000.00

GERI-1-2014 1 Hypatia 1,499,693.25 1,499,693.25

GERI-2-2014 1 GEDII 999,901.25 999,901.25

GERI-3-2015 1 EFFORTI 1,998,985.00 1,998,985.00

GERI-4-2014 3 GENERA, LIBRA, PLOTINA 7,775,895.75 2,591,965.25

GERI-4-2015 3 Baltic Gender, SAGE, EQUAL-IST 6,360,333.75 2,120,111.25

IBA-SWFS- 1 Scientix 3 3,000,000.00 3,000,000.00 SCIENTIX-2016

ISSI-1-2014 1 SPARKS 3,498,839.00 3,498,839.00

249 https://ec.europa.eu/priorities/jobs-growth-and-investment/investment-plan_en .

Topic code # of Project acronym(s) Total requested Average

projects EC requested EC

contribution contribution

(EUR million) (EUR million)

ISSI-1-2015 2 DITOs, BigPicnic 6,927,721.25 3,463,860.63

ISSI-2-2014 1 CIMULACT 3,299,701.83 3,299,701.83

ISSI-3-2015 1 Marina 2,999,943.75 2,999,943.75

ISSI-4-2015 1 ONLINE-S3 3,889,000.00 3,889,000.00

ISSI-5-2014 1 NUCLEUS 3,993,632.50 3,993,632.50

ISSI-5-2015 3 STARBIOS 2, RRI-Practice, JERRI 9,498,335.00 3,166,111.67

SCIENCE WAF 5 EUCYS 2014, EUCYS2015, 4,199,645.00 839,929.00

SOCIETY EUCYS2016, ESOF2016, SIS-RRI

SEAC-1-2014 5 CREATIONS, MultiCO, PERFORM, 8,162,145.75 1,632,429.15 SciChallenge, ER4STEM

SEAC-1-2015 5 STIMEY, EDU-ARCTIC, 11,121,945.00 2,224,389.00 STEM4youth, UMI-Sci-Ed, Marine

Mammals

SEAC-2-2014 2 EnRRICH, HEIRRI 2,998,171.63 1,499,085.82

SEAC-3-2014 1 EURAXESS TOPIII 3,559,043.00 3,559,043.00

SEAC-4-2015 1 EURAXIND 812,237.50 812,237.50

SWAFS-25-2016 1 ENABLE 497,626.00 497,626.00

Total 53 108,945,175.34 2,158,169.66

Source: CORDA data, 1 January 2017, Selected Projects and Signed Grants by Type of Action.

R.2.2. Participation patterns

R.2.2.1. Participation per type of organisation

A total of 53 projects have so far been selected. These projects represent 516 participations by 413 distinct participants. The greatest number of participants are from

HES (149), followed by REC (105) and OTH (72) 250 . In total, 11.9% of participants are

co-ordinators, with the greatest proportion of co-ordinators coming from HES (5.6%), followed by REC (3.1%); the smallest proportion of co-ordinators comes from PUB (0.5%).

On average, HES participants participate 1.4 times in SWAFS projects (i.e. almost half of HES participants are involved in more than one SWAFS project); this is followed by PUB (1.3), REC and OTH (1.2), and PRC (1.1). HES receives the largest EC contributions (EUR 0.23 million), followed by REC (EUR 0.22 million), OTH (EUR 0.21 million), PRC (EUR 0.18 million) and PUB (EUR 0.12 million).

250 HES=Higher or secondary education, PRC=Private for profit (excluding education), PUB=Public body (excluding

research and education), REC=Research organisation, OTH=Other.

R.2.2.2. Attraction of new-comers

As seen in Table 251, new-comers 251 represent 19.4% of participants: OTH makes up the

highest proportion of new-comers (6.5%), followed by PRC (5.6%), REC (4.1%), HES (1.7%) and PUB (1.5%). As seen in Table 252, 3% of participants are new-comers from

associated 252 , candidate 253 , or third countries. 254

Table 251 - Key data on participation per type of organisation for SWAFS: number of participants, project co-ordinators, new-comers, participations, and EC contribution to participations

Nr of % type Nr of % Co Nr of % New Nr of EC % of EC particip of project ordinato newcomers particip contributio contribu ants in organisa cors (of comers ations in n to tion signed tion ordinato total coin signed signed participatio grants rs in ordinato grants grants ns in signed

signed rs) grants grants (EUR million)

HES 149 36.1 24 45.3 7 1.7 204 49.3 43.8

OTH 72 17.4 8 15.1 27 6.5 83 18.7 16.6

PRC 54 13.1 4 7.5 23 5.6 57 10.6 9.4

PUB 33 8.0 2 3.8 6 1.5 42 4.9 4.4

REC 105 25.4 15 28.3 17 4.1 130 29.0 25.8

Total 413 100.0 53 100 80 19.4 516 112.5 100

Source: CORDA data, 1 January 2017, Participants and Participations by Legal Entity.

R.2.2.3. Geographical participation patterns

Organisations from all 28 EU Member States have participated in SWAFS. The United Kingdom and Germany have the highest number of participants and participations, followed by Italy and Spain. Most project co-ordinators come from EU-15 countries (46 project co-ordinators), while only four projects are co-ordinated by organisations located in EU-13 countries.

R.2.2.4. International co-operation

The largest number of participations come from the EU-15 (361, 70%), followed by EU-13 (92, 17.8%), associated countries (33, 6.4%), candidate countries (11, 2.1%), and third countries (19, 3.7%).

Two projects are co-ordinated by an organisation based in an associated country:

255

SiS.net2 (Iceland) and RRI-Practice 256 (Norway). A total of 33 non-co-ordinator

participations are from organisations based in associated countries; these cover Bosnia and Herzegovina, Switzerland, Faroe Islands, Georgia, Israel, Iceland, Moldova, Norway, and Ukraine. A total of nine participations are from organisations based in candidate countries; these cover Albania, the Former Yugoslav Republic of Macedonia, Serbia, and Turkey. A total of 19 participations are from organisations based in third countries; these

251 A "new-comer" is defined as a successful first-time applicant to Horizon 2020 who did not apply to the Seventh

Framework Programme (FP7).

252 Associated countries: BA, CH, FO, GE, IL, IS, MD, NO, UA.

253 Candidate countries: AL, MK, RS, TR.

254 Third countries: AU, BR, BY, CN, EG, IN, KE, LI, EG, US, ZA.

255 http://www.sisnetwork.eu/ .

256 https://rri.univie.ac.at/en/ .

cover Australia, Brazil, Belarus, China, Egypt, India, Kenya, Liechtenstein, Uganda, United States, and South Africa.

Table 252 - Key data on participation per group of country EU-28, EU-13, EU-15, associated countries, third countries for SWAFS: number of participants, of project co-ordinators, of new-comers, of participations, and EC contribution to participations

Country Nr of Nr of Nr of Nr of Average EC group participants project newparticipations participations contribution to in signed cocomers in signed per participations grants ordinators in grants participant in signed in signed signed grants (EUR grants grants million)

Associated 29 1 6 33 1.1 4.2 countries

Candidate 10 1 1 11 1.1 1.5 countries

EU-13 71 5 17 92 1.3 13.7

EU-15 284 46 49 361 1.3 87.4

Third 19 0 7 19 1.0 2.1 countries

Total 413 53 80 516 1.2 108.9

Source: CORDA data, 1 January 2017, Participants and Participations by Country group.

R.2.3. Cross-cutting issues

As part of efforts to promote Responsible Research and Innovation (RRI) across Horizon 2020, SWAFS fostered the co-creation of scientific agendas and scientific contents as

demonstrated by the 38 (82.6%) of projects (for those that include a flag 257 ) where

citizens, Civil Society Organisations (CSOs) and other societal actors contributed to the co-creation of scientific agendas and scientific contents. This is much higher than the Horizon 2020 average (11%). Eight (15%) projects are classified as not promoting

RRI 258 . Additionally, seven (13%) projects are currently missing RRI flags 259 . Overall,

many of the projects classified as non-RRI relevant or missing an RRI flag are NCP projects and ethics parts of the governance line, from the gender equality line, from the science education line, or for ad hoc conferences and events.

Regarding the gender dimension in R&I content, for those projects that include a flag, 41 (85.4%) of SWAFS projects take into account the gender dimension in R&I content. This is significantly higher than the average of 36.4% seen across Horizon 2020. Five (9.4%) projects are missing flags for the gender dimension; these include projects funded under GERI-4-2014, ISSI-2-2014, SEAC-1-2014, SEAC-3-2014, SCIENCE WAF SOCIETY; it seems likely that some of these projects should in fact be flagged as taking the gender

257 'Flags' are a way of signalling that a project is relevant to a policy priority. They are attributed at project level by

project officers. They are part of Horizon 2020's internal monitoring system and the data are stored in CORDA. As briefly detailed in this text (and elaborated upon elsewhere, see for instance the RRI annex on cross-cutting issues and the gender annex on cross-cutting issues) these data appear to have some margin of error.

258 These include projects funded under: GARRI-3-2014, GARRI-6-2014, GARRI-8-2014, GERI-4-2014, GERI-4-2015,

IBA-SWFS-SCIENTIX-2016, SEAC-1-2014, SEAC-3-2014.

259 These include projects funded under: GARRI-7-2014, GERI-2-2014, GERI-4-2014, SEAC-1-2014, SCIENCE WAF

SOCIETY.

dimension into account. Seven (13.2%) projects are flagged as not taking the gender dimension into account.

In SWAFS projects, 59.4% (282) of the total workforce (475) are women 260 . Most

SWAFS projects are co-ordinated by women (66.7%). This is about twice the Horizon

2020 average. The Horizon 2020 Advisory Group on Science with and for Society 261 is

composed of 15 women (53.6%) and 13 (46.4%) men. CORDA indicates that 85.4% of SWAFS projects take the gender dimension into account in R&I content; this is much higher than the Horizon 2020 average of 36.4%.

In terms of promoting the socio-economic sciences and humanities (SSH), 71.1% of SWAFS projects with flags are classified as being SSH relevant. This is the highest share in Horizon 2020 (excluding Euratom for which just one project is flagged) and significantly higher than the Horizon 2020 average of 13%.

Most SWAFS projects (44, 83%) are classified as contributing to sustainability. The EC contribution to sustainable development in SWAFS is estimated at EUR 80.4 million (73.8% of the total budget). The target is for it to exceed 60% of the EC contribution; the Horizon 2020 average is 53.3%.

One SWAFS project is classified as climate related. The EC contribution to climate in SWAFS is estimated at EUR 0.4 million (0.4% of the total budget – the target is for it to exceed 35%). This is the lowest in Horizon 2020 and significantly below the average, which is 28.2% of the EC contribution.

One SWAFS project is classified as biodiversity related. The EC contribution to biodiversity in SWAFS is EUR 0.4 million (0.4% of the total budget). This is below the average of Horizon 2020, which is 4% of the EC contribution.

Three SWAFS projects are classified as ICT related. The EC contribution to ICT-related projects in SWAFS is EUR 7.3 million (6.7% of the budget). This is below the average of Horizon 2020, which is 30% of the EC contribution.

R.3. R ELEVANCE

R.3.1. Is Science with and for Society tackling the right issues?

R.3.1.1. The relevance of Science with and for Society given the challenges to address

The need for all relevant stakeholders to work together to find solutions to economic, societal and environmental challenges by building effective co-operation between science

and society was clearly made in the Ex-ante Impact Assessment of Horizon 2020 262

(Section R.1.1). The response to this was the part of Horizon 2020 dedicated to SWAFS and RRI as a CCI through the whole of Horizon 2020.

260 Workforce includes people actively participating in and paid by the EU project. Please see the cross-cutting issue

annex on gender for discussion on the veracity of all gender-related cross-cutting issue figures.

261 http://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupDetail&groupID=3093 .

262 http://ec.europa.eu/research/horizon2020/pdf/proposals/horizon_2020_impact_assessment_report.pdf.

The 2014 Rome Declaration on RRI in Europe 263 argued that good marketing cannot be

relied upon for R&I acceptability, that diversity in R&I is "vital for enhancing creativity and improving scientific quality", and that the "early and continuous engagement of all stakeholders is essential for sustainable, desirable and acceptable innovation". Building on the Lund Declaration of 2009 (which called for emphasis on tackling societal challenges) and the Vilnius Declaration of 2013 (which underlined the need for resilient

partnerships with all relevant actors for research to serve society) 264 , the Rome

Declaration called on "European Institutions, EU Member States and their R&I Funding and Performing Organisations, business and civil society to make Responsible Research and Innovation a central objective across all relevant policies and activities, including in shaping the European Research Area and the Innovation Union". It called for greater cooperation in "science education; the definition of research agendas; the conduct of research; the access to research results; and the application of new knowledge in society – in full respect of gender equality, the gender dimension in research and ethics considerations". A recent information report produced by the European Economic and Social Committee (EESC) as part of their interim evaluation of Horizon 2020/SWAFS/RRI confirms this need, finding that 92% of stakeholder respondents targeted by a survey agreed with the proposition that civil society organisations should be involved in Horizon 2020 project consortia; 83% of respondents disagreed that only

businesses and/or academia should be involved. 265

Practices across the world continue to move in the direction of openness and collaboration, as evidenced by the rise in open access/open data practices, and local

actions such as SOLVE at the MIT (USA) 266 , the Campus de l’innovation pour la Planète 267 at the IRD (France), or the Mediterranean Living Labs (EU and non-EU Mediterranean countries). 268 The recent 3Os Strategy from the European Commissioner for Research and Innovation 269 - Open innovation, Open science, Open to the world 270 -

proposed that "many more actors will take part [in the research process] in different ways and the traditional methods of organising and rewarding research will also see

many changes". 271 One important dimension of open science is citizen science, which is

envisioned as "linked with outreach activities, science education or various forms of public engagement with science as a way to promote Responsible Research and Innovation". Giving impetus to this line of activity, citizen science was recently

recognised as an open science priority by the Council. 272

Despite these new political orientations, the expert group considered that the old ways of 'doing R&I' still prevail, resulting in a waste of R&I resources and social and technological innovations that are less likely to be adapted to the real world. Even though there is increased policy emphasis on the involvement of citizens in R&I, according to a

263 https://ec.europa.eu/research/swafs/pdf/rome_declaration_RRI_final_21_November.pdf .

264 The Lund Declaration (2009), http://www.vr.se/download/18.7dac901212646d84fd38000336/ and The Lund

Declaration (2015), http://www.vr.se/download/18.43a2830b15168a067b9dac74/1454326776513/The+Lund+Declaration+2015.pdf .

265 Information Report – Section for the Single Market, Production, and Consumption: Interim evaluation of Horizon

2020); European Economic and Social Committee. INT/807. Available at: http://webapi.eesc.europa.eu/documentsanonymous/eesc-2016-05513-00-01-ri-tra-en.docx .

266 http://solve.mit.edu/ .

267 https://www.ird.fr/toute-l-actualite/actualites/communiques-et-dossiers-de-presse/cp-2016/lancement-du-campus

href="https://www.ird.fr/toute-l-actualite/actualites/communiques-et-dossiers-de-presse/cp-2016/lancement-du-campus-de-l-innovation-pour-la-planete">de-l-innovation-pour-la-planete .

268 http://www.iemed.org/llista_activitats/the-mediterranean-as-an-open-living-lab .

269 http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm .

270 http://bookshop.europa.eu/en/open-innovation-open-science-open-to-the-world-pbKI0416263/ .

271 http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm .

272 1 http://data.consilium.europa.eu/doc/document/ST-9526-2016-INIT/en/pdf .

study of CSO participation in the last three Framework Programmes this starts from a

very low level and is something that should be improved. 273 The study revealed that Civil

Society Organisations (CSOs) have played only a marginal role in EU Framework Programmes (FPs), and have therefore had only very limited effects on network morphology, performance and research output.

Building on a body of support and good practices since 2000 274 , SWAFS aims to

improve the contribution of R&I to tackling societal challenges. It creates and consolidates links between R&I stakeholders, for instance by inviting citizens and civil society to engage more in R&I. It links various local, regional and national initiatives in Europe and encourages the exchange of good practices amongst them. SWAFS is designed with transversality and transdisciplinarity in mind; it focuses on particular societal challenges (e.g. marine mammals, digitalisation, robotics, biosciences) and on

issues that cut across them (e.g. CIMULACT 275 ). This aims to ensure focused, timely and

appropriate responses to the emerging world of R&I and continual improvements in the knowledge base that can be transferred to other parts of Horizon 2020. SWAFS aims to prevent divergent practices in R&I that would distort the single market (e.g. different ethical rules for products and services, separate employment markets, and much more variability across Europe in the uptake and acceptability of R&I).

A review by the expert group of the 53 SWAFS projects across all its lines of activity suggests that funded actions so far have focused on: 1) trying to understand the notion of RRI and barriers to its implementation/uptake, 2) creating ad hoc and largely unconnected structures out of the existing system to promote RRI, and 3) encouraging cultural rather than structural changes (i.e. in terms of how science is conducted).

According to the expert group, all eight SWAFS lines have a high level of relevance:

Science careers: The relevance of this line is still high in policy terms and in terms of the three SWAFS objectives. It rightly focuses on and addresses issues relating to international movement and the mobility of students and researchers within EU Member States and from abroad. This is an important factor in making science careers attractive, encouraging the free movement of knowledge, and encouraging excellent science.

Gender equality: The relevance of this line is still high. Improving the quality of R&I requires that the very best talents are retained. Unfortunately, the 'leaky pipeline' phenomenon suggests that that this is simply not yet the case. SWAFS' focus on gender equality (i.e. tackling the structural factors behind gender inequalities and gender bias in science, and on the gender dimension of science) is laudable. The potential outcomes of greater gender equality are better science, improved productivity, more appropriate and

gendered innovations, and more attractive science careers 276 . This will, in turn, reinforce

links between science and society and increase the societal relevance of research.

Public engagement: The relevance of this line to policy and the three SWAFS objectives remains high because it directly aims to "integrate society in science and innovation

273 Study on "Network analysis of Civil Society Organisations' participation in research framework programmes"

(Tender RTD-B6-PP-00962-2013).

274 For instance, through Science and Society (FP6), Science in Society (FP7), SWAFS (Horizon 2020), and regional

development policies covering the same period.

275 http://www.cimulact.eu/ .

276 See for instance Richard, O. C., & Miller, C. D. (2013). Considering Diversity as a Source of Competitive

Advantage in Organizations. In Q. M. Roberson (Ed.), The Oxford Handbook of Diversity and Work. Oxford: Oxford University Press.

issues", e.g. through public debates and consultations. SWAFS is highly relevant to promoting new ways to tackle societal challenges.

Science education: The relevance in this line is still high and the objectives of SWAFS are still aligned with the political direction of the EC with regard to R&I and education (e.g. emphasis on science, technology, engineering, and mathematics (STEM) and higher education). The relevance of the approach is exemplified when considering the 2015 Joint Report of the Council and the Commission on the implementation of the strategic framework for European cooperation in education and training – New priorities for

European cooperation in education and training. 277 This line is also highly relevant

considering future competitiveness and the need to ensure that the EU has a pool of welleducated

workers able to adapt rapidly to a changing environment. 278

Open access/open data: The move towards open access/open data is part of the new paradigm in R&I and this line is therefore still highly policy relevant.

Governance: This line's relevance is still high because the new and disruptive practices of R&I require changes to institutional structures and governance system – e.g. 'new ways of doing R&I'. Responsive and enabling policies in the field of governance and ethics are crucial to this. However, GARRI-8-2014 (National Contact Points for quality standards and horizontal issues) does not seem to be particularly relevant to SWAFS.

Due and proportionate precaution: This line's relevance is still high and actually increasing. Advances in R&I create distrust among citizens, as evidenced by controversies relating to climate change, nanotechnology, genetic modification, pesticide use and its impact on wildlife and human health. There has so far been no call for proposals on this line of activity in SWAFS. Because of this, it is not considered in the following sections.

Science communication: This line's relevance is still high. Science journalism has been experiencing something of a crisis in recent years yet this line has not explored this phenomenon or its implications. It would be highly relevant to rethink the meaning, purpose and methods of science communication. Because this line has no activities so far it is not considered in the following sections.

R.3.1.2. The relevance of Science with and for Society to address European objectives

Following examination of all 53 SWAFS projects, the expert group considers that SWAFS remains highly relevant to addressing European objectives. It aligns with the current political direction of the European Commission (EC) as seen in its Political

Guidelines 279 and the 10 priorities to achieve smart, sustainable and inclusive growth for Europe. 280 SWAFS' focus on sustainability is in line with one of the greatest societal

challenges today, and corresponds with other EC initiatives, e.g. the forthcoming

publication of the Corporate Social Responsibility strategy. 281

SWAFS constitutes an ideal tool for opening up R&I to new collaborations and new ways of working as part of the EC's 3Os Strategy. In addition, SWAFS supports the

277 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.C_.2015.417.01.0025.01.ENG .

278 World Economic Forum Europe 2020 Competitive Index.

279 https://ec.europa.eu/priorities/publications/president-junckers-political-guidelines_en .

280 https://ec.europa.eu/priorities/index_en .

281 https://ec.europa.eu/growth/industry/corporate-social-responsibility_en .

objectives of the European Research Area (ERA), by promoting an open labour market for researchers, gender equality and gender mainstreaming in research, and optimal

circulation, access to and transfer of scientific knowledge. 282 SWAFS responds to

advances in science and technology by focusing on promoting debates in emerging technologies and societal controversies (e.g. ICT and privacy, genetics and human enhancement).

The Expert group on evaluation methodologies for the interim and ex-post evaluations of

Horizon 2020 283 found that EU and international priorities related to SWAFS are covered

to a high degree in the Horizon 2020 establishment act and in the 2016-2017 work programme, and that they are covered to a medium degree in the 2014-2015 work programme (missing terms related to tertiary education, digital technologies, and sustainable development and lifestyles). Moreover, the needs of EU citizens are covered to a high degree in the Horizon 2020 documentation, the establishment act, and in both the 2014-2015 and 2016-2017 work programmes.

The SWAFS interim evaluation expert group considers that without clear political backing (as evidenced by dedicated funding for SWAFS), there would be a lack of positive signals coming from Europe about the need to increase public engagement in science and promote scientific careers, ensure that R&I is carried out to the highest ethical standards and in a responsible manner, and align the interests of science and society so that the outcomes of R&I are more appropriate and successful. Without SWAFS there would be much less co-ordination and contact between stakeholders across Member States and a reduction in policy developments conducive to bringing R&I closer to society. As the new paradigm of R&I continues to emerge and evolve, SWAFS provides a focal point around which actors can orientate, share good practice and expand their networks across the EU and indeed the world.

R.3.2. Flexibility to adapt to new scientific and socio-economic developments

COP21 284 and COP22 285 have elevated sustainability and climate change to major and agreed global policy priorities. 286 Recent years have also seen increasing recognition of the need to take action to protect biodiversity. 287 While SWAFS projects have focused on

sustainability in general, they have so far not focused on climate change and biodiversity to a significant extent (Section 3.3). Strong links in SWAFS to the 17 Sustainable

Development Goals (SDGs) 288 have so far not been made. Inequalities, unemployment,

health and ageing continue to present critical challenges to European social and political systems, and an area where social innovations will likely be critical.

No SWAFS-specific foresight activities have taken place, but two funded projects are making use of foresight methodologies. CIMULACT aims to engage citizens across Europe to provide input to the European Union’s R&I agenda; it treats foresight as a

282 http://ec.europa.eu/research/era/index_en.htm .

283 Expert group on evaluation methodologies for the interim and ex-post evaluations of Horizon 2020 – Applying

relevance-assessing methodologies to Horizon 2020: Executive Summary; October 2016. The keywords used for this analysis are listed in the report.

284 http://www.cop21paris.org/ .

285 http://www.cop22-morocco.com/ .

286 http://europa.eu/rapid/press-release_IP-16-3589_en.htm .

287 for instance http://ec.europa.eu/environment/nature/biodiversity/strategy/index_en.htm .

288 http://www.un.org/sustainabledevelopment/sustainable-development-goals/ .

“structured dialogue on long-term futures”. 289 The project Online-S3 aims to assist

national and regional authorities in the EU to elaborate and revise their smart specialisation agendas, in terms of policies and strategy, using a variety of tools and

methods including foresight. 290

R.3.3. Addressing specific stakeholder needs

SWAFS mainly funds CSAs, as these fit with the overarching goals of the programme (see Table 253). RIAs are also funded in order to consolidate and expand the SWAFS knowledge base. An ERA-NET Cofund has been established (SWAFS-02-2016: ERA- NET Cofund – Promoting Gender equality in Horizon 2020 and the ERA) to link national initiatives in the field of gender equality; this responds to a recommendation in the Ex-post evaluation of Science in Society which called for greater involvement of policy makers at national and EU levels, so as to "set priorities and ensure that the programme is investing in relevant actions that have the scale and reach to deliver

results of utility to policy makers" 291 .

The VOICES 292 and CIMULACT projects invited citizens to interact directly with EC

services. These projects harness the knowledge and views of citizens to help shape future Work Programmes in Horizon 2020. Two topics in 2016 invited stakeholders to reflect

on the main science and society issues that should be tackled through Horizon 2020 293 .

The two SWAFS Open Public Online Consultations (OPOCs) (Work Programme 2016-

2017 294 and Work Programme 2018-2020 295 ) and the SWAFS Advisory Group helped

identify stakeholder needs. Both OPOCs suggested strong support for continuing all eight lines of SWAFS activities. In addition, contributions highlighted citizen science and open science as important aspects for SWAFS to focus on in future Work Programmes. SWAFS is monitored by the Strategic Programme Committee of Horizon 2020 and has benefited from a specific Working Group since September 2016. An indication of the needs addressed by SWAFS is the fact that seven of the European Union's top 20

universities participates in SWAFS. 296

R.3.4. Other issues related to relevance

The concept of SWAFS/RRI is comparable to a number of other national approaches promoted outside of Horizon 2020. This is evidenced by responses to the National Contact Point/Strategic Programme Committee (NCP/SPC) survey (see Section 7.2.2 and Section 8), which suggests that SWAFS remains relevant to stakeholder needs across EU Member States. Project reviews (which take place mid-way through project implementation) suggest that some projects appear to be spending considerable effort

289 http://www.cimulact.eu/ .

290 http://www.onlines3.eu/ .

291 https://bookshop.europa.eu/en/ex-post-evaluation-of-science-in-society-in-fp7-pbKI0216492/ .

292 http://www.voicesforinnovation.eu/ .

293 SWAFS-07-2016: Training on open science in the European Research Area, and SWAFS-09-2016: Moving from

constraints to openings, from red lines to new frames in Horizon 2020. SWAFS topics can be accessed from the

Participant Portal at: https://ec.europa.eu/research/participants/portal/desktop/en/home.html .

294 http://ec.europa.eu/research/consultations/swafs/consultation_en.htm .

295 https://ec.europa.eu/research/consultations/swafs-wp2018-2020/consultation_en.htm . This received 104

contributions representing more than 6500 organisations from a wide range of stakeholder groups (e.g. industry, academia, civil society).

296 According to Times Higher Eeducation World University Rankings. See:

https://www.timeshighereducation.com/student/best-universities/best-universities-europe .

reconceptualising RRI. While renewal of and reflection upon RRI in SWAFS projects is welcome, particularly as a way of ensuring SWAFS remains as relevant as possible, these activities should not stand in the way of achieving the impacts described in the topic descriptions – unless such reflection is specifically called for.

R.3.5. Lessons learnt/Areas for improvement

SWAFS remains highly relevant to the overarching challenges facing Europe in different parts of Horizon 2020, including healthcare systems, migration, social inequalities and issues related to sustainability (e.g. climate change and biodiversity). It is also highly relevant – and of growing relevance – to the 'new paradigm' in R&I. As such, it responds well to the Ex-ante Impact Assessment of Horizon 2020, the Rome Declaration on RRI in Europe, the 3Os Strategy, and activities in Member States and further afield. This is confirmed by the SWAFS interim evaluation expert group assessment, the Expert group

on evaluation methodologies for the interim and ex-post evaluations of Horizon 2020 297 ,

and the two SWAFS OPOCs (see Section R.3.1.1).

While the relevance of SWAFS to tackling societal challenges is high, it could be improved by aligning itself more closely to the outcomes of COP21/22, the SDGs, challenges related to healthcare and social inequalities, and other overarching international agreements related to societal challenges. Nevertheless, as a transversal programme with (currently) 53 projects, it is by necessity difficult to cover all relevant areas of policy satisfactorily.

The global challenges, new policy orientations outlined in the 3Os strategy, and need for increasingly open collaboration between all parts of society, calls for much greater support for citizen science and user-led innovation in the programme. This would increase the relevance of SWAFS to stakeholders and Member State practices and policies in the field of R&I. These issues could be covered in innovative ways so that they tie in with other EU objectives, for instance by considering citizen science and userled innovation as ways to improve social inclusion and employability as well as ways to tackle R&I challenges.

R.4. E FFECTIVENESS

The effectiveness of SWAFS (i.e. its ability to effectively achieve its objectives) is limited due to the scale of the ambition, the relatively small budget allocation (less than EUR 0.45 billion over 7 years) and the range of funded activities. The expert group found that the budget appears to be out of all proportion to the stated objectives. As an

illustration, public expenditure in the EU28 on education in 2012 was EUR 672 billion 298

compared to the SWAFS budget allocation to science education in 2016 of EUR 0.006 billion. A similar situation was found in SaS and SiS – where a fragmentation of

activities and a lack of clear focus hampered ability to have impact. 299

297 http://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupDetail&groupID=3426 .

298 http://ec.europa.eu/eurostat/statistics-explained/index.php/Educational_expenditure_statistics .

299 DRAFT PRELIMINARY LIST OF FINDINGS; Stock-tacking and meta-analysis of Science in Society projects

throughout FP6 and FP7 (RTD-B6-PP-00965-2013).

R.4.1. Short-term outputs from the programme

Annex II of the Horizon 2020 Specific Programme (L 347/1037) lists the Key Performance Indicators (KPIs) for assessing the results and impacts of the specific objectives of Horizon 2020, adding that these KPIs may be refined during the implementation of Horizon 2020. None of the KPIs relate to society; instead, most relate to the number of publications or patents or to inputs to the S&T system.

The EC was invited to provide KPIs for Horizon 2020 during negotiations. The agreed SWAFS KPI was the "Number of institutional change actions promoted by the programme"; examples include the introduction of specific rules, governance arrangements or practices favouring open access, gender equality or public engagement. This KPI attests to the importance of supporting institutional changes in R&I organisations and of the need to ensure that the outcomes of SWAFS live on beyond the

lifetime of funding. 300

Although institutional change in R&I organisations appears to be a valid and relevant response to the needs outlined above (e.g. Section R.3), the SWAFS KPI does not capture all SWAFS activities. For instance, the SWAFS KPI does not cover many activities funded by RIAs, even though they may support the implementation of institutional change actions as a longer-term outcome. Even the CSAs do not always work towards institutional changes.

Three projects have been completed and 50 projects are on-going as of 1 January 2017. SWAFS is expected to contribute to 24 outputs (see intervention logic). The early stage of implementation of most projects means that many of the short-term outputs have not yet been delivered.

Educational material and curricula for RRI: Several projects should help towards this

short-term output. For instance, the FP7 SiS project RRI Tools 301 produced and compiled

a large body of training materials and tools. It involved more than 1000 members in an RRI Community of Practice and held more than 90 training events in 19 RRI Tools Hubs. Hypatia (GERI-1-2014) will also contribute, by developing, piloting and disseminating a modular toolkit of activities and guidelines for engaging teenagers in science, technology, engineering and mathematics (STEM) in a gender-inclusive way. These innovative activities will be implemented in 14 EU countries and further afield, in schools, science museums and by institutions in research and industry.

Institutional changes within research organisations to implement the European Research Area priorities: Six projects funded by GERI-4-2014 and GERI-4-2015 of the gender equality line (GENERA, LIBRA, PLOTINA, Baltic Gender, SAGE, EQUAL- IST) will contribute to this, as should two projects funded under GARRI-1-2014

(FOTRRIS 302 and PROSO 303 ).

Gender Equality Plans in RPO and RFO: Six projects are developing, implementing, and monitoring GEPs (GERI-4-2014-2015). Between them, they involve 59 partners; these are mainly RPOs but a number of RFOs are also involved (HES 29, REC 23, OTH

300 See https://ec.europa.eu/programmes/horizon2020/en/news/horizon-2020-indicators-assessing-results-and-impact

href="https://ec.europa.eu/programmes/horizon2020/en/news/horizon-2020-indicators-assessing-results-and-impact-horizon">horizon for full details of all Horizon 2020 KPIs and indicators for monitoring Horizon 2020 cross-cutting issues.

301 http://www.rri-tools.eu/ .

302 http://fotrris-Horizon 2020.eu/ .

303 http://www.proso-project.eu/ .

2, PRC 5). These efforts should benefit from existing pan-European platforms dedicated

to the issue of gender in R&I, e.g. GenPORT 304 , as well as from other projects that work

to develop methodologies to monitor and assess gender equality in R&I, such as

EFFORTI. 305

Networks of developers, providers and users of solutions involved in co-creation (value chain): SWAFS has the potential to be a major facilitator and contributor to this short-term output. However, because of a lack of involvement of all parts of the quadruple helix, and the low quality involvement of citizens and for-profits (see Efficiency section) SWAFS is not contributing to this short-term output as much as it could.

Publications: Three publications have been produced, all of which are provided as open

access. Two publications were funded under the topic GARRI-3-2014 306 , and one publication was funded under SEAC-1-2014. 307 Two are characterised by international

co-authorship (publications with authors from at least two institutions from different countries).

Project websites: 39 SWAFS projects currently have a dedicated website. 308 Many

appear to be making strides to develop their websites as user-friendly tools – not just static repositories. This is important from the perspective of turning SWAFS projects into

effective science communication and outreach tools. 309

Policy outputs: These have generally not been produced by projects yet. One exception is CIMULACT, which has already produced a significant body of material to input into

Horizon 2020. 310 As projects progress more policy outputs can be expected.

Overall, the short-term results of the programme are in line with expectations. As projects progress more publications and policy-related outputs can be expected. In addition, more institutional changes and gender equality plans will be implemented.

R.4.2. Expected longer-term results from the programme

SWAFS is expected to contribute to 33 longer-term results (see intervention logic) across all three parts of the 3Os strategy: open innovation, open science, and open to the world. Many results could be attributed to more than one of the 3Os, but they are categorised below according to where they are likely to make their main contribution.

Open Innovation

Solutions brought closer to market (increase in technology readiness level (TRL)): SWAFS can and should contribute to this longer-term result. For instance, the project SPARKS involves 7 PRCs in the consortia, alongside other organisation types, to show Europeans that they can get involved in science and that various stakeholders share responsibility for scientific research and innovation. As noted in Section R.5.2.1,

304 http://www.genderportal.eu/ .

305 http://cordis.europa.eu/project/rcn/203534_en.html .

306 See http://www.ssrn.com/abstract=2608513 and http://ebooks.iospress.nl/publication/42896 .

307 See http://conference.pixel-online.net/NPSE/files/npse/ed0005/FP/2393-ESM1531-FP-NPSE5.pdf .

308 As of 1 January 2017.

309 See for example Hypatia ( http://www.expecteverything.eu/ ) and RRI Tools ( http://www.rri-tools.eu/ ).

310 See, for instance, http://www.cimulact.eu/publications-2/ .

SWAFS does better than many other parts of Horizon 2020 at involving a balanced set of participant types, including PRC. Nevertheless, more PRCs (including social entrepreneurs) and OTH should be involved in funded activities to make a stronger contribution to this.

More responsible research and innovation: Almost all SWAFS projects contribute directly to RRI (see Sections R.2.3 and R.5.2.3). Nevertheless, the relatively low participation in PRC could hold back parts relating to innovation.

Reinforced innovation potential of European research infrastructures: Gender equality should increase the innovation potential of European research infrastructures. Institutional changes funded under the governance line should also contribute to this longer-term result.

Better R&I integration: All SWAFS lines should contribute to this, particularly those relating to RRI institutional change (governance) and public engagement. For instance, PRISMA, which focuses on nanotechnology, synthetic biology, the Internet of Things and self-driving or automated cars, should help by transforming the relationship between users, suppliers, consumers and other stakeholders. Greater involvement of for-profits in SWAFS would increase the effectiveness of contributing to this longer-term result.

Stronger pan-European collaboration across disciplines, sectors, value chains and technology levels: Several SWAFS projects are transdisciplinary and work on issues that cut across Horizon 2020 and disciplines. For instance, in the public engagement line MARINA aims to engage all stakeholders involved in the marine field; other examples in this line include BigPicNic (which focuses on food security), SPARKS (which focuses on technological shifts in health and medicine). Examples can also be found in the science education lines, such as ER4STEM (educational robotics for STEM), EDU- ARCTIC (natural science and polar research), and UMI-Sci-Ed (Ubiquitous Computing, Mobile Computing and the Internet of Things).

Improvement of societal awareness, understanding and engagement to tackle societal challenges through R&I: Most SWAFS projects focus on sustainable development (Section R.2.3). Several projects focus on one or more particular aspect of sustainability or environmental outcomes. Projects such as CIMULACT help citizens shape the R&I agenda.

Better societal acceptability of innovative solutions: This should be supported by the science education, public engagement and (indirectly) the governance line. Projects funded under the science communication and due and proportionate precaution would also contribute.

Open Science

Improved attractiveness of researchers' careers across the EU: The science careers line should contribute. For instance, in 2016 the EURAXESS portal was completely revamped to embed the policy objectives of R&I; researchers, entrepreneurs and the business sector can now access a wealth of information related to career development and mobility. Efforts to develop, implement and monitor GEPs in research institutions under the gender equality line should also increase the attractiveness of research careers in the longer term.

Strengthened human potential in R&D in business and academia (including gender balance) across European countries: The science careers (e.g. projects funded under SEAC-3-2014, SEAC-4-2014), gender equality (e.g. projects funded under GERI-1- 2014, GERI-4-2014, and GERI-4-2015) and science education lines (e.g. projects funded under SEAC-1-2014, SEAC-1-2015, SEAC-2-2014) should contribute to this longerterm result.

Reputation and excellence of Europe in scientific and technological research (modernisation of research institutions, vitality of the research environment and quality of research outputs in both basic and applied research): SWAFS is at the forefront of these issues. RIA and CSA actions should help contribute to this longer-term result. Evidence suggests that improved gender equality leads to improved scientific

excellence 311 .

Accelerated open data sharing and use in Europe: The open access/open data line (particularly GARRI-4-2015) will support this, as will actions to promote RRI (particularly institutional changes in the GARRI lines).

Improved science education for all citizens: The science education line focuses on this longer-term result. The public engagement line could also play a part.

Improved and professionalised NCP service, helping simplify access to Horizon 2020 calls: SWAFS funds two NCP actions, one of which is dedicated to SWAFS and the other to quality standards and horizontal issues for the whole of Horizon 2020.

Reinforced research integrity and ethics standards: Ethics projects funded under the governance line (e.g. projects funded under GARRI-4-2014, GARRI-6-2014, GARRI-9- 2015, GARRI-10-2015) support this longer-term result. Actions in the governance line to support RRI institutional changes should also contribute (e.g. projects funded under GARRI-1-2014, GARRI-2-2015).

More effective promotion of gender equality and the gender dimension in research and innovation content: GEPs play an indispensable part in contributing to this longerterm result in projects funded under the gender equality line (6 different projects involving 59 partners).

Responsible R&I principles embedded in EU higher education institutions: Institutional changes funded under the governance line work directly towards this longerterm result.

Open to the World

Enhanced position and role of the EU R&I in the international R&I arena: This could be an important outcome of SWAFS and the relatively high levels of international co-operation in SWAFS projects should help (Section R.2.2.3). However, the low budget given to SWAFS does not appear to give the political 'clout' required to contribute greatly to this longer-term result.

311 See for instance: Campbell, L. G., Mehtani, S., Dozier, M. E., & Rinehart, J. (2013). Gender-heterogeneous

working groups produce higher quality science. PloS One, 8(10), e79147. doi:10.1371/journal.pone.0079147. See also Cheruvelil, K. S., Soranno, P. A., Weathers, K. C., Hanson, P. C., Goring, S. J., Filstrup, C. T., & Read, E. K. (2014). Creating and maintaining high-performing collaborative research teams: the importance of diversity and interpersonal skills. Frontiers in Ecology and the Environment, 12(1), 31–38. doi:10.1890/130001.

Global challenges are opportunities to innovate: SWAFS should be able to contribute by more greatly involving citizens and end-users in the R&I process from across the

world. TRUST 312 , for instance, could play a part in the field of ethics. The involvement

of organisations from the EU and further afield should open up opportunities to innovate and spread innovations to the EU. SWAFS already does better than many other parts of Horizon 2020 at involving organisations from outside the EU (see Section R.2.2.3 and R.5.2.2).

Overall, SWAFS' progress towards longer-term results is in line with expectations. Increasing the involvement of OTH and PRC (public for profit) organisations including social entrepreneurs in project consortia, could contribute to increasing the longer-term results relating to open innovation.

R.4.3. Progress towards attaining the specific objectives

At the moment it is not possible to assess progress towards the SWAFS KPI ('number of institutional changes') as this will be reported by project co-ordinators when projects end. In addition, the chain of activities working towards the three SWAFS objectives is difficult, if not impossible, to evaluate given the extremely broad nature of the programme and KPIs that do not measure progress towards them. However the expert group analysed the effectiveness of the programme according to each of the eight lines of activity based on the projects selected so far:

Science careers: This line consisting of 3 topics (3 projects) appears to be effective at making scientific careers more attractive and supporting students and scientist in their

scientific careers. For instance, EuroScience Open Forum (ESOF) 313 competes with its US counterpart American Association for the Advancement of Science (AAAS) 314 and

attracts thousands of delegates to several days of lively debate. Euraxess (EURAXESS

outreach to Industry) 315 , Euraxind (EURAXESS for Industry) 316 and Enable (European Academy for Biomedical Science) 317 aim to exchange knowledge between science and

innovation stakeholders, building on actions funded under FP7. However, these actions do not appear to involve civil society/citizens and SMEs and it is not clear how this line contributes to the SWAFS KPI.

Gender equality: This line focuses in particular on institutional change actions, which

appears to be an effective approach to the issue 318 . However, it does not appear to be

effective in terms of the small number of organisations it reaches compared to the number 'in the field' and the sheer scale of gender inequalities across the EU. In addition, the effectiveness of this line can only really be considered in relation to sustainability – and the sustainability of the actions beyond the lifetime of funding is not clear. The effectiveness of the actions could also be hampered by lack of disciplinary links between partners and different national operating contexts. This line appears to work effectively towards the SWAFS KPI, as six out of nine funded projects work towards institutional change.

312 http://trust-project.eu/.

313 See http://esof2014.org/ and https://www.esof.eu/ .

314 http://meetings.aaas.org/ .

315 http://cordis.europa.eu/programme/rcn/665137_es.html .

316 http://cordis.europa.eu/project/rcn/203166_en.html .

317 http://cordis.europa.eu/project/rcn/205424_en.html .

318 https://ec.europa.eu/research/science-society/document_library/pdf_06/structural-changes-final-report_en.pdf .

Public engagement: This line, consisting of 5 topics (10 projects) involves citizens/CSOs in discussing R&I issues. Efforts are made to bring the results into policymaking processes within the EC, which helps increase the potential effectiveness and sustainability of the funded actions. Nonetheless, this line's effectiveness is limited by the kinds of activities it funds: mostly outreach, debates and discussion. While this may appeal to a certain demographic, it does not involve citizens and scientists jointly in 'doing' R&I, which would likely engage a completely different demographic. Just two

projects, MARINA 319 and BigPicnic 320 , encourage collaboration in actually 'doing' R&I.

It is not clear how this line contributes to the SWAFS KPI.

Furthermore, the skewed participation of social actors limits this line's effectiveness. There are a total of 174 participations in the public engagement line, but this is dominated by HES (60), followed by REC (40), OTH (36), PRC (23) and PUB (15). The quality of the participation could be an additional limiting factor – for instance, forprofits mostly seem to be involved as suppliers of services and consultancies.

The public engagement line is likely to have an impact on the formulation and development of topics in other parts of Horizon 2020 and in the successor programme (e.g. through CIMULACT and through SWAFS-09-2016). While this is no small achievement in itself, the relatively low level of funding allocated to this line raises questions about the natural limits to its effectiveness. Moreover, and to some extent connected to the budget, wider impact on Member States seems unrealistic.

Science education: The effectiveness of this line, consisting of 2 topics and 4 grants to identified beneficiaries (16 total projects), appears to be very low; it cannot be expected to have sustained impact on the teaching of science across all 28 Member States given the 18.5 million euro allocation, organisational inertia within education systems, and the sheer scale of the challenge. Instead, this line is likely to see small, incremental and rather isolated improvements in the teaching of science education. There are 154 total participations in this line, dominated by HES (68), followed by OTH and REC (28 each), PRC (16) and PUB (14). As such, this line does appear to have been moderately effective at mobilising a more balanced set of participants. It is not clear how this line contributes to the SWAFS KPI.

Open access/open data: The effectiveness of this line, consisting of 2 topics and 2 projects, appears to be in line with expectations. The two projects should provide useful input to EU policy making. There are a total of 18 participations in this line, dominated by 9 REC, followed by 5 HES, 3 PRC, 1 PUB. As such, the line has not mobilised any OTH-type organisations. It is not clear how this line contributes to the SWAFS KPI.

Governance: The effectiveness in this line, consisting of 4 topics and 1 grant to an identified beneficiary (total of 12 projects) appears to be moderate. Not all topics aim for institutional change; instead, many appear to focus internally on developing understandings of RRI rather than provoking change in the current structures related to R&I. For those topics that do work towards governance changes, there is a danger that too much focus is put on cultural aspects of governance/institutions. While undoubtedly important, they are just one part of the picture. In addition, and in some cases connected to the internal focus of projects, the low representation of citizens/civil society in these

319 http://www.marinaproject.eu/ .

320 http://cordis.europa.eu/project/rcn/203174_en.html .

projects poses a big question of effectiveness: out of 104 total participations in this line, most are from HES (34), followed by REC (29), OTH (18), PUB (15) and PRC (8).

Overall, progress is in line with expectations. However, greater progress would likely be achieved if all topics specified SMART objectives, if all lines worked towards the SWAFS KPI, and if a better representation of all parts of society (particularly PRC and OTH) were included in the funded actions. Connected to this, institutional changes are not clearly defined for all of the eight lines. While the gender equality line has rather clear definitions and methods, what an institutional change is in the public engagement line (for instance) is less clear. Moreover, the relatively low budget, the limited lifetime of funding, and the fact that just a handful of projects are funded per topic, which spreads resources rather thinly, means that the institutional changes that are implemented need to be sustainable to be considered effective.

R.4.4. Progress towards the overall Horizon 2020 objectives

R.4.4.1. Fostering excellent science in scientific and technological research

The expected results of SWAFS-funded projects will contribute to fostering excellence in scientific and technological research through:

 Boosting formal and informal science education (e.g. HEIRRI, UMI-SCI-ED 321 , DITOs 322 , Hypatia);

 Attracting young people to science (e.g. DITOs, Hypatia, ENABLE 323 , EDU- ARCTIC 324 , PERFORM 325 , EURAXESS TOP III 326 );

 Promoting the mobility of researchers (e.g. Euraxess, RESAVER 327 , MultiCO 328 );  Promoting ethics and research integrity (e.g. ENERI 329 , DEFORM 330 ,

PRINTEGER 331 , TRUST);  Promoting open access to publications and research data (e.g. FutureTDM,

OpenUp);

 Supporting citizen science (e.g. BigPicnic, DITOs);  Promoting and supporting gender equality in R&I content (Hypatia, GENERA 332 , GEDII 333 , LIBRA 334 , PLOTINA 335 , Baltic Gender 336 , EFFORTI, SAGE 337 , EQUAL-IST 338 ).

321 http://umi-sci-ed.eu/ .

322 http://togetherscience.eu/ .

323 http://enablenetwork.eu/ .

324 http://eduarctic.eu/ .

325 http://www.perform-research.eu/ .

326 https://topiii.eu/ .

327 "Other actions", SWAFS Work Programme 2016-2017. See: http://www.resaver.eu/ .

328 https://multicouk.wordpress.com/ .

329 http://cordis.europa.eu/project/rcn/204323_en.html .

330 http://cordis.europa.eu/project/rcn/203532_en.html .

331 http://printeger.eu/ .

332 http://genera-project.com/ .

333 https://www.gedii.eu/ .

334 http://www.eu-libra.eu/ .

335 http://www.plotina.eu/ .

336 http://cordis.europa.eu/project/rcn/203533_en.html .

337 http://cordis.europa.eu/project/rcn/203535_en.html .

338 https://equal-ist.eu/ .

Despite the relatively modest amount of funding available, the programme can expect to have important leverage effects in terms of fostering excellent science in scientific and technological research. To enable this, it will be important that projects continuously feed into and interact with policy-making processes at all relevant levels (EU, national, regional, etc.).

R.4.4.2. Boosting innovation, industrial leadership, growth, competitiveness and job creation

SWAFS contributes by funding projects that explore societal concerns in emerging

technologies (e.g. synthetic biology in SMART-map 339 ; nanotechnology, synthetic biology, the Internet of Things and self-driving or automated cars in PRISMA 340 ; healthcare, nanotechnology, ICT in COMPASS 341 ). It also contributes indirectly through

the science careers line by improving scientific careers and mobility (e.g. EURAXIND).

R.4.4.3. Addressing the major societal challenges

SWAFS aims to ensure R&I plays a greater role in responding to societal challenges (see Section R.1.1) and many projects contribute to this objective. SWAFS experiments with

new ways of doing R&I in specific fields. For instance, STARBIOS 2 342 focuses on the

biosciences and PROSO focuses on nanotechnology, food and health and the bioeconomy. Other projects invite citizens to participate in defining R&I agendas (e.g. CIMULACT, MARINA).

R.4.4.4. Spreading excellence and widening participation.

SWAFS takes into consideration the quadruple helix (e.g. project ONLINE-S3). Also relevant are the results of FP7 Knowledge Incubation in Innovation and Creation for

Science (KIICS) 343 , which aim to break silos between disciplines and business sectors at

local levels by promoting incubation actions between artists, creators, scientists, creative and technology businesses and young adults across 13 countries through creative European collaboration processes. As noted in Section R.2.2.3 and Section R.5.2.2, SWAFS is relatively successful compared to other parts of Horizon 2020 at involving EU-13 participants in its activities. The NCP network (see Section R.5.2.1) is important in this respect, as it directly aims to open up to new participants. Nevertheless, the level of EU-13 and third country participation in SWAFS could be even higher.

R.4.4.5. Science for policy

SWAFS supports citizen science for policy making (e.g. CIMULACT, and FP7's VOICES). The FP7 SiS programme supported work on open access, ethics, and gender equality, all of which have since become important policy priorities in Horizon 2020. Policy emphasis is still relatively undeveloped in the areas of public engagement and science education (or indeed in areas that bridge these two lines, such as citizen science and user-led innovation), and these will likely be key areas for future policy making to come out of SWAFS.

339 http://cordis.europa.eu/project/rcn/203167_en.html .

340 http://cordis.europa.eu/project/rcn/203531_en.html .

341 https://innovation-compass.eu/ .

342 http://starbios2.eu/ .

343 http://cordis.europa.eu/project/rcn/102260_en.html .

Box 26 - Contribution to the achievement and functioning of the ERA 344

The science careers line and in particular Euraxess contribute to more effective national research systems, improved transnational co-operation and competition, and an open labour market for researchers. EURAXESS, for instance, contributes to this last ERA objective by listing thousands of job opportunities, supporting the provision of mobility assistance to researchers so that they can move across borders, providing personalised assistance to researchers and support to refugee researchers so that they can join the labour market.

The gender equality line contributes to the last ERA objective as well as to gender equality and gender mainstreaming in research. Actions undertaken as part of the open access/open data line contribute to the optimal circulation and transfer of scientific knowledge.

Furthermore, all projects that promote RRI should contribute to gender equality and the optimal circulation and transfer of scientific knowledge.

Source: European Commision services.

Overall, the contribution of SWAFS to the overall Horizon 2020 objectives is in line with expectations. The greatest contribution can be seen in the area of fostering excellent science in scientific and technological research, and in involving broad stakeholder involvement to address the major societal challenges. More balanced participation by all organisation types, and even greater involvement of EU-13 and third countries, would strengthen the contribution to Horizon 2020 objectives. The area of science for policy is likely to build on previous successes in the areas of gender equality, open access and ethics, and focus on increasing public engagement and science education through framework programmes.

R.4.5. Early success stories

Hypatia (2015-2018) is a CSA that aims to encourage more teenagers, especially girls, to take up STEM careers in school and as a future career choice, and to make the way the sciences are communicated to young people more gender inclusive. The project will produce a toolkit, setup 14 national hubs and organise a series of events in science centres and museums. Events will also be organised for educationalists and teenagers, to translate, adapt and implement Hypatia’s learning modules. In addition, a campaign will be organised to target teenagers all around Europe. The total requested EC contribution is EUR 1.5 million, the total project costs are EUR 1.57 million, and it involves a total of 10 partners from 6 countries.

CIMULACT (2015-2018) is a CSA that aims to gather the views of a representative sample of 2500 citizens from 30 countries on future EU R&I policies and research topics. It will do this in a highly participatory debate and consultation process to build scenarios for desirable sustainable futures and research. It will then provide input for the preparation of Work Programme 2018-2020 of Horizon 2020 for at least 3 societal challenges. The total requested EC contribution is EUR 3.3 million, total project costs are EUR 3.4 million, and it involves a total of 29 partners from 28 countries.

344 http://ec.europa.eu/research/era/index_en.htm .

MARINA (2016-2019) is a CSA focused on responsible marine R&I. It aims to create an all-inclusive knowledge-sharing platform that catalyses and organises already existing networks, communities, on-line platforms and services and to provide an online environment that stimulates the direct engagement of all stakeholders to improve RRI. The total requested EC contribution (and total project cost) is EUR 3 million and it involves a total of 14 partners from 11 countries.

R.4.6. Other issues related to effectiveness

As noted in Section R.3.4, some projects appear to be spending considerable effort reconceptualising RRI. While this should be welcomed as a means of increasing the relevance of SWAFS, delivery mechanisms may need to be reviewed to ensure this reflection does not negatively impact on the effectiveness of the funded actions.

R.4.7. Lessons learnt/Areas for improvement

Given the ambition–budget mismatch, it is important that SWAFS acts as a catalyst to continued action rather than funding one-off actions. With the exception of the gender equality line, institutional changes are not generally specified as expected outcomes of projects and/or not defined in easily understandable terms. The effectiveness of SWAFS would be increased by emphasising the need for institutional changes in all lines and explaining what those institutional changes consist of. Cultural changes, while important, should not necessarily be favoured over sustainable institutional (regulatory) changes that could in themselves lead to cultural changes. The sustainability of the changes is also important. Other potential areas to explore include requiring applicants to develop business or long-term sustainability plans and using project reviews as an opportunity to enhance alignment with EC policy priorities.

SWAFS does not fund activities that directly involve all parts of society in 'doing' R&I. Projects often target citizens rather than involve them in R&I. There is a corresponding lack of focus on citizen science and user-led innovation, which should be rectified in the Work Programme 2018-2020. When projects do involve for-profits or citizens/CSOs, their involvement does not appear to be 'core' to many projects. The implicit notion appears to be that everything coming from the 'public' is 'societal', but this requires much more critical appraisal. In addition to increasing the involvement of CSOs/OTH-type organisations, greater involvement of PRC (for profits), including social entrepreneurs, would likely help work towards longer-term results related to open innovation.

Some areas, such as science education, can only be effective if the national policy and education institutions are politically 'on board'. To be effective, more 'upstream' policy actors need to be included in project consortia to ensure political 'buy in'; this would help ensure that SWAFS outcomes are applied in different operating contexts. This would increase effectiveness and avoid the 'drop in the ocean' effect that some SWAFS lines currently suffer from. Over the medium term this could also encourage greater external policy coherence.

R.5. E FFICIENCY

R.5.1. Budgetary resources

As part of the evaluation, all projects were mapped and assigned to one primary line of activity. Figure 256 shows the total project costs by SWAFS line of activity. Given the

number of projects in each line, this allocation of resources is in line with expectations.

The average time to grant is 250 days and the median is 240 345 ; this meets the Horizon

2020 target (240 days).

Considering the importance of the relationship between science and society, the variety

of stakeholders, the urgent need to cope with rapid societal and environmental transformations driven by innovation, the need for a scientifically literate and engaged

population, the deficit in gender equality and disparities in access to scientific

knowledge, the amount dedicated to the SWAFS programme appears low compared to

the needs. This is despite the fact that two of SWAFS' eight lines have so far had no projects funded under them.

Figure 256 - Total project costs by SWAFS line of activity 346

Source: CORDA data/Expert Group project mapping exercise; January 2017.

R.5.2. Programme's attractiveness

R.5.2.1. Mobilisation of stakeholders

The relatively early stage of implementation is likely to give a skewed picture of the

implementation state of play. Nevertheless, SWAFS is clearly over-subscribed in some lines. There were a very high number of eligible proposals compared to the number that

were retained. Of the 766 proposals received for closed calls on 1 January 2017, 409

(53.4%) were evaluated as being of a high-quality (i.e. above the evaluation threshold)

and only 16.9% of high-quality proposals were retained. Overall, SWAFS has a proposal success rate of 9.0%. This is slightly lower success rate than the Horizon 2020 average

(12.9%), where 44.7% of all Horizon 2020 proposals were of a high quality (slightly

lower than SWAFS) and 26.4% of high-quality proposals were retained (higher than SWAFS).

345 CORDA data, 1 January 2017, Selected Projects and Signed Grants by Type of Action.

346 Note: GE=Gender equality, GOV=Governance and ethics, OA=Open access/open data, PE=Public engagement,

Sci-Car=Science careers, SE=Science education.

Broken down by type of action, RIAs have a proposal success rate of 5.4% and CSAs have a success rate of 11.3%. This is influenced by the relatively high number of grants to identified beneficiaries (all CSAs), which have 100% success rates providing that the proposals are evaluated as being high quality. Accordingly, removing the 6 CSAs

(Scientix 3 347 , EUCYS 2014 348 , EUCYS2015 349 , EUCYS2016 350 , ESOF2016 351 , SIS- RRI 352 ) results in lower success rates: 10.5% for CSAs and 9.0% overall. The science

education line is characterised by particularly low success rates (5.7% in 2014, 2.9% in 2015, and 4.4% in 2016).

This situation is not new, but is certainly more acute than in the past – mirroring the lower success rate seen in Horizon 2020 compared to FP7. The SiS programme in FP7

was described as suffering from low success rates in its ex-post evaluation 353 ;

nevertheless, in hindsight the proposal success rate of 22% (higher than the FP7 average

of 18%) appears satisfactory compared to SWAFS 354 . The success rate of FP6's Science

and Society programme was also described as "clearly over-subscribed", with just 1 in 8 (12%) of proposals being funded (compared to an average success rate of 18% for the

FP6 average) 355 .

Table 253 - Data on proposals per type of action: Number of eligible and retained proposals, EC contribution requested (in EUR million) and success rates (as % of proposals submitted, and as % of budget available)

Type of action Nr of Nr of EC EC Success Success eligible retained contribution contribution rate rate

proposals proposals requested by to retained proposals funding eligible proposals

proposals (EUR (EUR million) million)

CSA 531 60 1091.5 118.1 11.3% 10.8%

ERA-NET 1 1 3.8 3.8 100% 100% Cofund

RIA 239 13 445 28.4 5.4% 6.4%

Sum (all) 771 74 1540.2 150.2 9.6% 9.8%

CSA (ex. grants 526 55 1087.3 113.9 10.5% 10.5% to identified

beneficiaries)

Sum (ex. grants 766 69 1536.0 146.0 9.0% 9.5% to identified

beneficiaries)

Source: CORDA data, 1 January 2017, Success Rates by Type of Action (General).

Table 254 shows the success rates of applications by type of organisation in SWAFS and SiS. The highest application success rates in SWAFS are enjoyed by PUB (19.3%), followed by REC (14.3%), OTH (11.9%) and HES (8.3%). The picture in SiS was rather different, as success rates ranged from 29.6% (REC) to 20.6% (HES); the largest

347 http://www.scientix.eu/home .

348 http://media.eucys2014.pl/ .

349 http://www.eucys2015.eu/ .

350 http://eucys2016.eu/ .

351 https://www.esof.eu/ .

352 http://cordis.europa.eu/project/rcn/193320_en.html .

353 Ex-post Evaluation of Science in Society in FP7 - Final Report, p142.

354 https://bookshop.europa.eu/en/ex-post-evaluation-of-science-in-society-in-fp7-pbKI0216492/

355 https://ec.europa.eu/research/evaluations/pdf/archive/fp6-evidence-base/statistics/statistics_fp6_final_review.pdf

percentage differences in success rates compared to the average success rate were found in HES (-14%) and REC (+22.1%). By contrast, in SWAFS the percentage differences in success rates compared to the average vary much more widely: from -67.6% (PRC) to +68.1% (PUB). The difference in the success rate of OTH (which includes most CSOs) compared to all organisation types has increased in SWAFS (+22.4%) compared to SiS

(-9.3%). 356

As noted in Section R.2.2.1, HES-type organisations have more than double the number of applications than PUB and PRC and almost double that of OTH. Even though HES has a below average application success rate, the higher number of applications leads to it being much more greatly represented in SWAFS participations. HES appears to crowd out other organisation types for funding, receives the largest average EC contributions, makes up the largest group of participants, is least likely to be a new-comer and most likely to be a project co-ordinator. This suggests a kind of 'market capture' of SWAFS funding by HES. The information report produced by the EESC on SWAFS/RRI makes a

similar point, suggesting that "theory groups are still dominating". 357

SiS was highlighted as being particularly effective at involving OTH stakeholders

(15.3% of project participations compared to the FP7 average of 3%) 358 . Indeed, the

relative success of SiS at involving CSOs compared to FP7 as a whole was recently verified by the Study on network analysis of civil society organisations’ participation in research framework programmes, which found that SiS was 'the only theme in which

citizen-oriented and society-oriented CSOs play more than a marginal role'. 359 The

importance of involving a diverse set of stakeholders was also highlighted by a recent meta-analysis study of SiS and SaS projects, which found that 'best-practice projects' in SaS and SiS were successful in large part because they managed to engage relevant

stakeholders and involve them in partnerships within the consortia. 360

SWAFS builds on this success, with 16.3% OTH compared to a Horizon 2020 average of 5.3%. When compared to participations in the FP7 SiS programme, SWAFS has 6.3% fewer HES, 1% more OTH, 0.8% more PRC, 0.1% less PUB and 4.7% more REC, which suggests some progress has been made in balancing consortia compositions. Even if over the long run there is a move towards a more balanced participation of different organisation types and SWAFS still manages to involve proportionally more OTH than other parts of Horizon 2020, this is still not sufficient compared to what is necessary to achieve the objectives of SWAFS and projects generally do not represent all parts of

quadruple helix formations 361 , including giving significant roles to CSOs and for-profit

entities .

356 SiS figures are from the Ex-post Evaluation of Science in Society in FP7 - Final Report. See also the RRI crosscutting

issue annex, which discusses the issue of the identification and classification of CSOs.

357 Draft Information Report – Section for the Single Market, Production, and Consumption: Interim evaluation of

Horizon 2020); European Economic and Social Committee. INT/807. This point was clarified as relating specifically to SWAFS.

358 See Table 251 for SWAFS. Data for Horizon 2020 from CORDA data extraction 1 January 2017.

359 Study on network analysis of civil society organisations’ participation in research framework programmes

(CONTRACT NO. RTD-B6-2014-SI2.687781) - D5: Draft final report. Note that involvement of CSOs across Horizon 2020 lines is discussed in the cross-cutting issue annex on RRI.

360 DRAFT PRELIMINARY LIST OF FINDINGS; Stock-tacking and meta-analysis of Science in Society projects

throughout FP6 and FP7 (RTD-B6-PP-00965-2013).

361 The quadruple helix model considers particular services, products and solutions as being co-identified, codeveloped

and co-created through co-operation between industry, government, universities and society (e.g. citizens and CSOs.

Table 254 - Key data on success rates of applications for SWAFS and participation success rates for SiS

SWAFS SiS

% % % % % %

Success rate Difference Participations Participation Difference Participations of from success rate from

applications average average SiS

success rate success rate

HES 8.3 -13.5 39.5 20.6 -14.0 45.8

OTH 11.9 +22.4 16.3 21.6 -9.3 15.3

PRC 4.7 -67.6 11.0 23.7 0 10.2

PUB 19.3 +68.1 7.8 23.1 -2.6 7.9

REC 14.3 +40.3 25.4 29.6 +22.1 20.7

Average 9.5 23.7

Source: CORDA data, 1 January 2017, Applicants and Applications by Type of Organisation (General); Ex-post Evaluation of Science in Society in FP7 - Final Report.

REA manages 44 SWAFS projects and RTD manages nine 362 . REA's management appears to be carried out efficiently (e.g. concerning evaluations, time-to-grant, etc.) 363 .

Project co-ordinators are in touch with their REA project officer, but not necessarily with their DG-RTD policy officer, meaning there is a potential inefficiency in terms of policy exchange, though efforts have been made recently to improve this through the establishment of a policy feedback contact point in REA. This decentralisation could also contribute to the lower levels of internal coherence seen in some SWAFS lines compared to the SWAFS objectives (see Section R.6.1.1).

Main activities to promote stakeholder participation

Network of Science with and for Society National Contact Points: SiS.net 364 aims to

provide more effective and better quality services to potential SWAFS applicants. Sixteen countries are represented in the project consortium and there are four main lines of activity:

• 1. 

  Mutual learning and capacity building. This is carried out through trainings, surveys, and network meetings, as well as at the individual level through welcoming packs and bilateral visits. Mapping of NCP experiences and needs is important for keeping the objectives focused, not forgetting the need for knowledge in the wider area of EU policy in the field. There have so far been four bilateral visits, four NCP training sessions, three network meetings (in Rome, Tallinn, and Madrid) and one welcome package distributed to new NCPs.

• 2. 

  Mobilisation of stakeholders. This aims to recruit new participants to SWAFS, reach stakeholders such as SMEs and CSOs, and participants in third countries. Two brokerage

362 The projects managed by DG-RTD B7 as of 1 January 2017 are Scientix 3, European Union Contest for Young

Scientists 2014/2015/2016, ESOF 2016 and the Network of Science with and for Society National Contact Points. RTD A3 manages the National Contact Points for quality standards and horizontal issues. DG-RTD B2 manages EURAXESS TOP III.

363 See for instance

https://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/rea_evaluation_report.p df , and REA Interim Report to the parent DGs and the Steering Committee First semester (2016) (unpublished).

364 http://cordis.europa.eu/project/rcn/193344_en.html .

events have taken place. The 2015 event attracted 170 participants from 28 countries and saw 159 participants have 239 bilateral meetings (233 were international). The 2016 event attracted 220 participants from 28 countries and saw 180 participants have 414 bilateral meetings (395 were international).

• 3. 

  Increasing the visibility of SWAFS including its objectives and outcomes. SiS.net does this by disseminating information to NCPs, discussing SWAFS objectives and outcomes at EU and national levels, collecting information about the embedding of SWAFS objectives, and supporting NCPs to raise awareness of SWAFS-related issues among national and regional policy makers.

• 4. 

  Outreach and communication activities. These improve the visibility of SWAFS stakeholders to policy makers and the media through targeted outreach activities. This line of work also includes internal communication within the NCP network.

EuroScience Open Forum (ESOF): The EuroScience Open Forum (ESOF) has grown to become the largest pan-European science event and an equivalent of the American Association for the Advancement of Science (AAAS). The EC has supported ESOF since 2014, and it is now one of the biggest founding supporters.

With an attendance of 3,500 delegates from 80 countries, 700 high-level speakers including Nobel laureates, businesses and policy leaders, ESOF is a huge platform for science, where all stakeholders and the general public can meet and debate cutting-edge research, R&I policies and global challenges. The opportunities presented at ESOF for networking and establishing contacts between the scientific disciplines, business sectors, and the entire range of European and international stakeholders present makes it a unique event for the European R&I community. As such, it is mobilising and engaging important stakeholders for the 3Os strategy.

DG-RTD B7 manages the SWAFS website 365 , which provides information about RRI

and SWAFS-funded activities. The SWAFS Twitter account multiplies information disseminated by SWAFS projects and publicises relevant new calls, publications and events.

R.5.2.2. Geographical dimension

Figure 257 shows the application success rates for SWAFS per EU28 Member State. Malta has the highest applicant success rate (18.9%), followed by Luxembourg (17.4%), Austria (14.8%) and France (13.1%); it should be noted that Malta and Luxembourg have a low number of retained applications (7 and 4 respectively, compared to 38 for Austria and 37 for France, which no doubt affects this ranking). The lowest application success rates are seen in Latvia (2.6%), Romania (4.2%), Croatia (5.0%) and Slovenia (5.8%) – all EU-13 countries.

365 http://ec.europa.eu/research/swafs/index.cfm .

Figure 257 - Success rates (as % of proposals submitted) per EU-28 country and

average in country groups for SWAFS

Source: CORDA data, 1 January 2017, Applicants and Applications by EU-28 Member States (General) and Applicants and Applications by Country groups (General).

As with other parts of Horizon 2020, there is much lower involvement of the EU-13 than the EU-15. However, SWAFS does much better than Horizon 2020 as a whole at

involving the EU-13 (17.8% of participations in SWAFS compared to 8.5%

participations in Horizon 2020). The information report produced by the EESC found that the lack of EU-13 involvement in SWAFS/Horizon 2020 was a result of lack of

awareness, experience and expertise in preparing applications, and low levels of cooperation

between researchers and industry/SMEs 366 .

As seen in Figure 258, the participation patterns in SWAFS and SiS are roughly the

same. Notable differences are that the UK, France and Sweden appear to have a lower

level of participation in SWAFS than in SiS. On the other hand, Austria, Greece and

Poland appear to have a higher level of participation in SWAFS than SiS. In SWAFS, the EU-15 made up 361 of the participations in signed grants (79.7%) while the EU-13 made

up 92 participations in signed grants (20.3%). By contrast, SiS saw 1323 (82.3%) of

participations from EU-15 and 285 (17.7%) participations from the EU-13. Thus, there is a slight increase in the percentage participation of EU-13 in SWAFS compared to SiS

and a slight decrease in participation of EU-15 in SWAFS compared to SiS. In SWAFS,

the EU-15 has a total of 42 co-ordinators of projects (11.8% of the total participants),

while the EU-13 has just four co-ordinators of projects (1.4% of the total participants).

366 Draft Information Report – Section for the Single Market, Production, and Consumption: Interim evaluation of

Horizon 2020); European Economic and Social Committee. INT/807

Figure 258 - Number of participations from the EU28 in signed grants in SWAFS

and SiS

Source: CORDA data, 1 January 2017, Participations by EU-28 Member States (General). Figures for

SWAFS are in bold, figures for SiS are in italic.

As seen in Figure 259, associated and candidate countries have roughly the same level of

participation in SWAFS as in SiS, while third country participation has dropped slightly.

Overall, candidate, associated and third countries made up 13% of participations in SiS and make up 12% in SWAFS; this compares to an average of 8.9% in Horizon 2020.

There is half the level of international participation in Horizon 2020 compared to FP7, so

the drop in SWAFS has been much less pronounced. Indeed, SWAFS compares very

favourably in terms of international co-operation to many other parts of Horizon 2020. 367

Given the nature of the issues tackled by SWAFS, this relatively greater involvement of

EU-13 and non-EU countries should be expected. Nevertheless, many of the challenges

and solutions focused on by SWAFS, including those related to R&I, are transversal and global in nature (Section R.3.1.1 and Section R.6.2.2). An even higher level of

international participation would help SWAFS better respond to these challenges and

learn from practices and solutions that are emerging globally.

Figure 259 - Participation by country groups in SWAFS (left) and SiS (right)

(numbers and % of totals)

Source: CORDA data, 1 January 2017, Participants by Country group (SWAFS); CORDA data, 1 January 2017, Participants by Country Group (FP7).

367 European Commission - Performance Analysis of International Participation in Horizon 2020 - A support study for

the interim evaluation of Horizon 2020.

R.5.2.3. Cross-cutting issues

As noted in Section R.2.3, SWAFS-funded projects work towards several Horizon 2020

cross-cutting issues. The attention paid to RRI and the gender dimension in R&I content and sustainability is satisfactory, though greater attention could be paid to climate change

and biodiversity. SWAFS has the highest proportion of SSH-relevant projects in Horizon

2020 as a result of its societal focus and the methods used/stakeholders involved.

R.5.3. Cost-benefit analysis

The mean size of SWAFS projects managed by REA is EUR 2.14 million EC

contribution and the median is EUR 1.83 million EC contribution; this is some way from the EC's expected average project size of EUR 3.2 million EC contribution.

Figure 260 - Requested EC contributions for SWAFS projects externalised and

managed by REA (EUR)

Source: CORDA data, 1 January 2017, Selected Projects and Signed Grants by Type of Action.

The relatively low level of funding (Section R.5.1) directed towards issues of importance

(Section R.1.1) and the fact that this appears to have a high level of relevance, added

value, and importance attached to it by Member States (cf. NCP/SPC survey, Section R.6.2.2 and Section R.7), suggests a rather satisfactory level of efficiency in terms of

responding to needs, providing a focal point for these issues, maintaining policy profile

and generating policy messages.

At the level of the individual lines and projects, the resources allocated to the planned

tasks suggest a satisfactory level of efficiency. Some projects do not appear to aim for

any kind of institutional change, which suggests a different benchmark for efficiency than those that do. Likewise, temporary institutional changes imply a lower level of

efficiency than those that are sustained beyond the lifetime of funding. At present, few

projects appear to aim explicitly for sustainable changes (with the notable exception of

NUCLEUS).

The analysis by line of activity performed by the expert group of all 53 projects suggests

the level of efficiency overall is satisfactory. Further details are provided below:

Gender equality: The funded projects appear to have a satisfactory level of efficiency

and demonstrate a good fit between the objectives, resources and planned activities. It is

not clear how initiatives can be taken up in national contexts or how sustainable the

changes promoted under this line will prove to be. Greater focus on post-funding sustainability could increase efficiency.

Public engagement: Many projects funded under this line appear to be 'one-offs' and they do not work towards sustainable institutional change. The lack of involvement of for-profits and CSOs/citizens (see Section R.2.2 and Section R.4) could be considered both a cause and consequence of the lack of sustainability – and therefore efficiency – of the actions (i.e. projects are considered opportunities to obtain funding, e.g. for HES, rather than longer-term catalysts for business or action, e.g. by PRC or OTH).

Open access/open data: Project efficiency is rather high in terms of the stated aims and objectives. The two projects funded under this line are strategically well placed to inform policy making.

Governance: It is difficult to assess the efficiency of this line due to the early level of implementation and lack of information about the scale and impact of institutional changes.

R.5.4. Other issues related to efficiency

"Open topics" have been promoted by the EC as a way to allow emerging and novel practices to receive funding. However, feedback from stakeholders and a low number of applications to certain calls (e.g. SWAFS-04-2016 and SWAFS-09-2016) suggests that they confuse applicants because the lack of prescription makes it hard to know what evaluators will look for. Open calls appear to result in lower subscription rates and an

increase in the number of questions submitted through the Research Enquiry Service 368 (and subsequent FAQs published on the Participant Portal 369 ).

R.5.5. Lessons learnt/Areas for improvement

SWAFS appears to be rather efficient considering the relatively low budget, its high relevance to the societal challenges and R&I policy (Section R.3), its EU added value (Section R.7), and the importance attached to it by stakeholders (Section R.6.2.2). Most projects are managed by REA and this seems to be carried out efficiently (e.g. concerning evaluations, time-to-grant, etc.), though there is room to improve policy connections between DG-RTD and the funded projects.

The individual projects funded by SWAFS appear to be efficient and in line with expectations when compared to their intended actions/outcomes. One area of uncertainty concerns institutional changes and whether they are sustainable; these kinds of outcomes would imply a higher level of efficiency. "Open topics" could result in increased inefficiency. The SWAFS NCP network and ESOF appear to be successful at mobilising interest, focusing policy, and attracting participants to the programme. Further consideration could be given to improving the attractiveness of the programme and success rates among organisations based in the EU-13.

SWAFS projects are smaller in size than the expected EUR 3.2 million average EC contribution for externalised projects. While there could be administrative efficiency

368 http://ec.europa.eu/research/index.cfm?pg=enquiries .

369 http://ec.europa.eu/research/participants/portal/desktop/en/support/faq.html .

savings in moving to larger projects, smaller project types could well be more effective at mobilising a balanced set of participant types, as found by the SWAFS interim evaluation

expert group and the EESC's recent interim evaluation of Horizon 2020/SWAFS/RRI 370 .

SWAFS is clearly over-subscribed and has low success rates, particularly in certain topics such as those in the science education line; this could be a result of it filling an important policy niche, as found in the flash survey of NCPs and SPCs about SWAFS/RRI (see Sections R.6.2.2 and R.7.2) and the PPMI's Horizon 2020 survey on

added value. 371 The situation has worsened (i.e. over-subscription has increased) since

the last Framework Programme but this mirrors wider changes seen in Horizon 2020.

SWAFS is relatively successful in terms of EU-13 and international co-operation. Nevertheless, the global reach of the issues (Section R.3.1.1 and Section R.6.2.2), and the grassroots and disparate emergence of practices that could inform SWAFS approaches (and vice versa) mean that SWAFS should aim for even higher levels of international participation.

R.6. C OHERENCE

R.6.1. Internal coherence

R.6.1.1. Internal coherence of the actions implemented for Science with and for Society

Expert group mapping and analysis shows that all objectives are supported by between 5 to 6 lines of activity (see Figure 255). Three lines of activity directly support all three SWAFS objectives (public engagement, science education, and governance), two lines of activity directly support two SWAFS objectives (gender equality, and science communication), and three activities directly support just one SWAFS objective (science careers, open access/open data, and due and proportional precaution). As such, some lines are less internally coherent with the three SWAFS objectives than others. Many of the SWAFS lines of activity also reinforce each other in practice, e.g. science education pays attention to gender equality, and public engagement pays attention to ethical issues. This is in line with findings of the MoRRI project (FP7), which finds substantial

permeability and co-support between the different dimensions of RRI. 372

A review of all 53 projects and contributions to the SWAFS public consultations suggest that SWAFS has an appropriate mix of different actions (mainly CSAs and some RIAs) and most lines have a satisfactory level of internal coherence. Additional comments follow:

Gender equality: The internal coherence of the gender equality line is high for projects funded under GERI-4-2014-2015 (Support to research organisations to implement gender equality plans), as two-thirds of the projects are developing and implementing GEPs

370 Draft Information Report – Section for the Single Market, Production, and Consumption: Interim evaluation of

Horizon 2020); European Economic and Social Committee. INT/807.

371 See Framework contract for the provision of services to the Commission in the field of evaluation of research and

innovation programmes and policies (2012/S 144-240132): Overview of Horizon 2020 Survey Results (draft, unpublished).

372 The dimensions of RRI do not exactly correspond with the eight SWAFS lines, but do correspond closely enough for

this analysis to hold. See for instance: Synthesis report on existing indicators across RRI dimensions

Progress report D3.1 (unpublished).

following the same toolbox and the same set of policy recommendations. 373 Some of the

institutional change projects are specific to a particular field of science while others are cross-disciplinary. Such differences are likely to influence a number of crucial aspects, such as intra-consortium collaboration, synergies between projects and follow-up initiatives. It is not clear how different consortia developing and implementing GEPs interact, but topic SWAFS-08-2017 (European Community of Practice to support institutional change) should help in this respect.

The project EFFORTI, which aims to develop methodologies to assess the impact of gender equality actions and to develop common evaluation frameworks across gender projects, should make link to the projects that are actually implementing their own GEPs, as this would likely benefit projects under both topics. Hypatia appears to be the sole project that makes direct links to other lines (in this case science communication). While this arguably decreases the coherence within the gender equality line, it increases the internal coherence of SWAFS.

While internally coherent, most projects do not directly contribute to the first SWAFS

objective ("build effective co-operation between science and society"). Hypatia 374

(GERI-1-2014) which encourages girls to study science, and EFFORTI (GERI-3-2015) which assesses the impact of gender equality initiatives on society at institutional/system levels, are exceptions.

Open access/open data: Based on a review of the two projects in this line, the internal

coherence of this line appears to be high. However, while FutureTDM 375 and OpenUp 376

work towards the SWAFS objective of pairing scientific excellence with social awareness and responsibility, they seem to have very little direct connection to building effective co-operation between science and society and fostering recruitment of new talent for science. The level of relevance of this line is therefore moderate.

Governance: There is generally high internal coherence in this line. The two NCP topics cover separate issues. GARRI-7-2014 (Science with and for Society National Contact Points (NCPs) in Horizon 2020) is dedicated to SWAFS (see Section R.5.2.1), while GARRI-8-2014 (National Contact Points for quality standards and horizontal issues) is focused on quality standards and horizontal issues in Horizon 2020. The latter of these projects does not appear particularly coherent with SWAFS. The projects focused on ethics so far only indirectly appear to be relevant to fostering effective co-operation between science and society and fostering the recruitment of new talent for science. The level of relevance of this line is therefore moderate.

R.6.1.2. Internal coherence with other Horizon 2020 intervention areas

SWAFS tackles transversal issues that are coherent with approaches in other parts of Horizon 2020 (e.g. gender, emerging technologies, ethics). It provides complementary

approaches that support other lines; good examples include VOICES (FP7 SiS) 377 ,

CIMULACT, and SWAFS-09-2016 (Moving from constraints to openings, from red lines to new frames in Horizon 2020) which provide direct citizen and stakeholder input into Work Programmes.

373 https://ec.europa.eu/programmes/horizon2020/en/Horizon 2020-section/promoting-gender-equality-research-and

href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/promoting-gender-equality-research-and-innovation">innovation .

374 http://www.expecteverything.eu/ .

375 http://project.futuretdm.eu/ .

376 http://openup-Horizon 2020.eu/ .

377 http://www.voicesforinnovation.eu/ .

SWAFS appears to be most coherent with the societal challenges (e.g. SC2 – Food, and SC5 – Climate), and parts of LEIT (e.g. ICT, Nanotechnologies). This is evidenced by SWAFS projects focusing on societal challenges (e.g. MARINA, BigPicnic, EDU- ARCTIC) and emerging technologies (e.g. PRISMA, SMART-map). There do not appear to be such strong links with other areas of Horizon 2020, such as the European Research Council (ERC), Access to Risk Finance, Innovation in SMEs or Spreading Excellence and Widening Participation (SEWP). The mainstreaming of SWAFS approaches is supported by the RRI and gender cross-cutting issues; these help ensure coherence with other parts of Horizon 2020.

Citizen science is promoted by several DGs and agencies, for instance DG-RTD, the Joint Research Centre, DG Connect, DG Climate Action (CLIMA), and EASME. Coherence will be increased by the "Environment Knowledge Community", which aims to co-ordinate services in the environmental field.

Figure 261 shows that there are areas of complementarities and synergies between SWAFS and FET, MSCA, SC5, SC2 and LEIT NMBP.

Figure 261 - Internal coherence of SWAFS with other Horizon 2020 specific objectives

Source: EC Internal survey.

R.6.1.3. Ensuring that every euro spent counts twice

MARINA (2016-2019) is a CSA focused on responsible marine R&I. It aims to create an all-inclusive knowledge sharing platform that catalyses and organises the convergence of already existing networks, communities, on-line platforms and services and provide an online environment that facilitates and stimulates the direct engagement of researchers, CSOs, citizens, industry and others to improve RRI. The total requested EC contribution (and total project cost) is EUR 3 million and it involves a total of 14 partners from 11 countries.

CIMULACT (2015-2018) is a CSA that aims to gather the views of a representative sample of 2500 citizens from 30 countries on future EU R&I policies and topics. It will do this in a highly participatory debate, consultation and process to build scenarios for desirable sustainable futures and research. It will then provide concrete input for the preparation of Work Programmes 2018-2020 of Horizon 2020 for at least 3 societal challenges. The total requested EC contribution is EUR 3.3 million, total project cost EUR 3.4 million, and it involves a total of 29 partners from 28 countries.

R.6.2. External coherence

R.6.2.1. Coherence with other EU funding programmes

Fully evaluating SWAFS' coherence with all other EU policies would be an enormous task given that the relationship between science and society can be found in all fields and sectors, and that the context is constantly shifting. Nevertheless, external coherence can be found with the European Structural and Investment Funds (ESIFs), which will dedicate around EUR 110 billion to innovation activities, ICT, (SME) competitiveness

and the low carbon economy 378 . Thus, there is clear external coherence with SWAFS but

it is not clear how well or to what extent these policies are informed by SWAFS (or RRI). The Work Programme 2018-2020 is likely to focus on the territorial dimension of R&I, meaning that links between SWAFS and ESIF/regional policies will be made more strongly in the second half of SWAFS, for instance by promoting quadruple helix-type approaches at local and regional levels. Project Online-S3 (see Box 27 below) should also be useful in this regard.

Box 27 - External coherence – Online-S3

This SWAFS project (2016-2018) is a RIA that aims to develop an e-policy platform augmented with a toolbox of applications to assist national and regional authorities to elaborate and revise their smart specialisation agendas. The platform will leverage existing methodologies, initiatives and tools developed by the EC for the RIS3 strategy. It will investigate, develop and test new and innovative technologies, tools, and services that strengthen European capacity for knowledge-based policy advice.

The platform will become an online mechanism for policy advice, integrated with a welldefined, commonly accepted and broadly used methodology for regional policy (RIS3 Guide). Data and sources of information to feed the platform and the services will be collected from institutional websites, RSS feeds, online databases, newsgroups, web forums, social networks data, web analytics, content management systems, measurement scoreboards, focus groups and other online mechanisms for collaboration and policy codesign, and assessment. Taken together, these elements will offer all the necessary tools, methods, and roadmaps to help elaborate, implement and analyse the impact of smart

specialisation policies". 379

The total requested EC contribution (and total project cost) is EUR 3.89 million and it involves 12 partners from 8 countries.

Gender equality: Gender equality is an issue of importance in many EU programmes. From this point of view, SWAFS' gender equality line can be seen as complementary to these broader efforts, as it zooms in on specific challenges within organisations through the implementation of GEPs, and fills in the existing gaps, e.g. through the GEDII project. This line's objectives are coherent and correspond with those set by Member States in the European Research Area to remove gender inequalities at EU and national levels. As such, it supports the overall objective of Horizon 2020 as defined in Annex 1

378 https://ec.europa.eu/growth/industry/innovation/funding/esif_en .

379 http://cordis.europa.eu/project/rcn/203172_en.html and http://www.onlines3.eu/.

of Regulation 1291/2013 i under "broad lines of the specific objectives and activities" in terms of helping achieve "the Europe 2020 strategy and other Union policies as well as the achievement and functioning of the European Research Area".

Science education: There appears to be a lack of external coherence with the EC's

strategic framework, Education & Training 2020 380 , which does not mention SWAFS or RRI. 381

Open access/open data: GARRI-3-2014 (Scientific Information in the Digital Age: Text and Data Mining) was launched in order to provide input to the extensive discussions

held during Licences for Europe 382 , which was part of the move towards new legal

framework announced in September 2016.

Governance: Raising the bar on ethics is at the core other EU policies, e.g. the EU international development and co-operation policy, where the EU uses its funding as currency to improve democratic values and raise human rights across the globe. As such, the emphasis placed on improving ethics in R&I is fully coherent with this approach.

R.6.2.2. Coherence with other publicly supported initiatives at regional, national and international level

The relationship between science and society is increasingly addressed at national level, with specific programmes dedicated to enhancing the capabilities of R&I organisations (e.g. Region Ile de France (FR), CGE Plan Vert (FR) and Responsible Innovation (UK, DE)). The OECD has its own research programme on inclusive innovation “Innovation

for Inclusive Growth”. 383 Some SWAFS-funded projects, such as TRUST, aim to

increase the policy coherence of EU and national-level responses to SWAFS issues.

Projects in the science education line and funded under SEAC-1-2015 (Innovative ways to make science education and scientific careers attractive to young people) and SEAC-2- 2014 (Responsible Research and Innovation in Higher Education Curricula) appear to be coherent with the education and innovation policy recommendations of the European Committee of the Regions as well as many of the Regional Smart Specialisation

Strategies (RIS3) 384 . Scientix 3 is coherent with similar activities in Member States and it attempts to complement activities carried out by national institutions 385 . The joint EC DG Education and Culture and OECD's "heinnovate" initiative 386 , and the World Economic

Forum's (WEF) competitiveness indicator (along with its related methodology measuring

the return of investment on higher education and training) 387 also appear complementary

to SWAFS.

However, the level of coherence between Member States approaches is not clear. While there is a new paradigm in R&I at the grassroots, and publicly supported movements in the fields of citizen science and user-led innovation, according to the expert group there appear to be several 'pilots in the cockpit' and initiatives in different Member States are moving in divergent policy directions at varying speeds.

380 http://ec.europa.eu/education/policy/strategic-framework_en .

381 https://ec.europa.eu/programmes/erasmus-plus/node_en .

382 https://ec.europa.eu/licences-for-europe-dialogue/en/content/about-site.html .

383 http://www.oecd.org/sti/inno/knowledge-and-innovation-for-inclusive-development.htm .

384 http://s3platform.jrc.ec.europa.eu/ .

385 http://www.scientix.eu/ .

386 https://heinnovate.eu/ .

387 http://reports.weforum.org/global-competitiveness-report-2015-2016/ .

A flash survey of NCPs and SPCs on SWAFS/RRI conducted for this interim

evaluation 388 (see also Section R.7.2) showed that while most countries do have

initiatives in areas related to SWAFS/RRI, no responding country has a national scheme with the same objectives. Most responding countries have schemes/programmes that tackle SWAFS issues, often relating to gender equality, science education, science careers and science communication, but may appear to be based on quite different rationales and aims. Some countries, such as Norway and Finland, appear to take a more holistic approach that is comparable with SWAFS, but even then programmes are much more nationally focused and there is tellingly no significant mention of international cooperation

in any country response. 389

SWAFS also has tangible influence at national level, even in countries with schemes covering similar areas of activity or taking a comparable approach. While countries were split on having a programme similar to SWAFS (2 do, 10 not really, 1 not clear), or legislation/policies/schemes or guidance on RRI (6 do, 5 not really, 1 not clear), 12 out of 13 countries stated that activities related to SWAFS/RRI have tangible impact and influence at national level. Indeed, except for Sweden (where EC action appears less influential) and the UK (where its influence is not clear), SWAFS is influential in three main ways:

 First, it strengthens and complements action at national level, providing

inspiration to develop new policies and programmes and aiding progress towards

the ERA Roadmap. 390  Second, international collaborations are highly valued and participation in

SWAFS projects increases the knowledge and capacities of participating organisations, enabling stakeholders to act as advocates for SWAFS-like

approaches at nation level.

 Third, EC action influences national policies, shapes funding priorities and helps

national funding agencies benchmark and learn. This is backed by the Stocktacking and meta-analysis of Science in Society projects throughout FP6 and FP7, which found that SaS and SiS contributed to the developments of national discourses, and "[i]n many areas the programmes had an initiating role opening new research agendas. In addition to that they contributed to consistency and alignment of policies and research approaches across Europe especially for

bridging the gap between research and society". 391

In these ways SWAFS helps achieve results, creates links, and complements, stimulates and leverages action to raise standards and create synergies. This suggests SWAFS has a high level of coherence across countries and helps increase external coherence by

reducing fragmentation of approaches and creating trans-national links. 392

388 Three questions were sent in November 2016 to the SWAFS NCP network and to the members of the Strategic

programme Committee SWAFS Working Group. As of 24 January 2017, responses were received from AT, BE, CH,

CY, DE, DK, ES, FI, FR, NL, NO, SE, UK. A response was also received from the Ministry of Science, Technology and Productive Innovation of Argentina.

389 This is backed by the findings of the meta-analysis study, which also found that EU action in these areas EU-funded

projects are considered more important than national ones.

390 See https://era.gv.at/directory/230 and http://ec.europa.eu/research/era/key-documents_en.htm .

391 DRAFT PRELIMINARY LIST OF FINDINGS; Stock-tacking and meta-analysis of Science in Society projects

throughout FP6 and FP7 (RTD-B6-PP-00965-2013). National-level impacts were particularly apparent for gender equality, open access, "non-conventional methods in science education" and embedding citizen participation in national policies.

392 This is backed by the ex-post evaluation of SiS and the study on "Stock-tacking and meta-analysis of Science in

Society projects throughout FP6 and FP7".

R.6.3. Other issues related to coherence

The MoRRI project 393 is currently developing an indicator system to measure the

evolution and benefits of RRI. The study covers all EU-28 countries and (where possible) countries associated to FP7 and Horizon 2020. It has identified 36 indicators for the evolution of RRI and 11 indicators for the benefits of RRI. These collectively cover the 5 dimensions of RRI and also governance. Data collection on these indicators will provide a much clearer view of national-level differences and could eventually lead to greater coherence between countries in their approach to RRI. MoRRI will also inform development of topics for Horizon 2020 Work Programme 2018-2020 (in particular SWAFS and RRI-flagged projects) and the eventual successor programme to Horizon 2020.

R.6.4. Lessons learnt/Areas for improvement

Internal coherence

SWAFS is internally coherent but some lines could refer more explicitly to the three SWAFS objectives as overarching aims, in particular the first objective of building effective co-operation between science and society. Tools that facilitate dialogue and exchange of good practice between projects would increase internal coherence.

SWAFS tackles transversal issues and is complementary and coherent with many lines of Horizon 2020, particularly with the Societal Challenges and parts of LEIT. Coherence with the rest of Horizon 2020 has been ensured to a certain extent by the mainstreaming of the RRI CCI. There is less coherence with some other parts of Horizon 2020, such as the ERC, Access to Risk Finance, Innovation in SMEs and SEWP.

External coherence

The external coherence of SWAFS is also high. Gender, public engagement, ethics projects funded under the governance line, and open access/open data appear to be well aligned with other EU funding programmes. Science education does not appear coherent with Education & Training 2020, though it does appear to be coherent with regional, national and other international initiatives. There are strong links with the ESIFs, though it is not clear how well or to what extent SWAFS contributes to ESIF policy and implementation.

The science and society relationship can be found in all fields and all sectors, and the situation is constantly shifting. This makes it challenging to ensure external coherence with all other EU funding programmes. Nevertheless, approaches similar to those promoted by SWAFS are increasingly addressed by Member States and SWAFS helps increase the coherence of Member State and EU approaches. The MoRRI project will provide useful evidence on the evolution and benefits of RRI, and help encourage greater external coherence.

393 Study on "Monitoring the evolution and benefits of Responsible Research and Innovation" (Tender No. RTD-B6-

PP-00964-2013).

R.7. EU A DDED V ALUE

R.7.1. Horizon 2020 projects demonstrating EU Added Value

Citizen and Multi-Actor Consultation on Horizon 2020 (CIMULACT)

CIMULACT (2015-2018) is a CSA that aims to gather the views of a representative sample of 2500 citizens from 30 countries on future EU R&I policies and topics. It will do this in a highly participatory debate, consultation and process to build scenarios for desirable sustainable futures and research. It will then provide concrete input for the preparation of Work Programmes 2018-2020 of Horizon 2020 for at least 3 societal challenges. The total requested EC contribution is EUR 3.3 million, total project cost EUR 3.4 million, and it involves a total of 29 partners from 28 countries.

Marine Knowledge Sharing Platform for Federating Responsible Research and Innovation Communities (MARINA)

MARINA (2016-2019) is a CSA focused on responsible marine R&I. It aims to create an all-inclusive knowledge sharing platform that catalyses and organises the convergence of already existing networks, communities, on-line platforms and services and provide an online environment that facilitates and stimulates the direct engagement of researchers, CSOs, citizens, industry and others to improve RRI. The total requested EC contribution (and total project cost) is EUR 3 million and it involves a total of 14 partners from 11 countries.

SPARKS 394

SPARKS (2015-2018) is a CSA that aims to familiarise and engage European citizens with the concept and practice of RRI through technology shifts in health and medicine. Sparks promotes RRI by inviting key stakeholders to actively question and experiment with science in a way that makes it relevant to today’s society. It will communicate the importance of RRI in a new way, by using the creative and “disruptive” visions of artists on how our future will be affected by the use of technologies in health and medicine. The total requested EC contribution is EUR 3.5 million, total project cost EUR 3.6 million, and it involves 15 partners from 13 countries (with "local implementation teams" covering 29 countries).

R.7.2. Other issues related to EU added value

The very high EU added value of SWAFS is confirmed by a recent survey of Horizon

2020 co-ordinators 395 , which found that just 8.7% of SWAFS projects could have gone

ahead without EU funding with no or minor modifications (this is lower than the average of 13.7% for Horizon 2020 as a whole). The vast majority of projects (66.4%) would not have gone ahead even with significant modifications (the average for Horizon 2020 as a whole is 53.2%). The reasons for this were: not having alternative sources of funding for the type of activities foreseen; not being able to address pan-European issues solely at national level; lack of access to necessary knowledge, expertise and skills in other

394 http://www.sparksproject.eu/ .

395 See Framework contract for the provision of services to the Commission in the field of evaluation of research and

innovation programmes and policies (2012/S 144-240132): Overview of Horizon 2020 Survey Results (draft, unpublished).

countries without Horizon 2020; and SWAFS providing more funding than is available from national/regional sources.

As such, the added value can be found in terms of effectiveness ('EU action is the only way to get results to create missing links'), efficiency ('resources can be pooled'), and synergies ('EU action is necessary to complement national-level initiatives'). The expert group argued that this is particularly the case for gender equality – an issue that should be addressed by creating transnational networks and cross-cutting solutions – and where the EU's funding has been and continues to be indispensable.

Results from the NCP flash survey of SWAFS/RRI (see also Section R.6.2.2) paints a similar and complementary picture of SWAFS filling a rather important policy priority niche. Moreover, this situation shows considerable continuity with FP6 and FP7 – where SaS/SiS projects demonstrated a strong European added value in terms of enhanced visibility and relevance of covered issues, resource pooling and scaling up effects of science–society topics in national research agenda. This EU added value stemmed from several factors including: co-ordination gains between science organisations; enhanced networking and research capacities of scientists; availability of new sources of information and EU funding; and knowledge circulation among institutions, policy

makers, and societal actors. 396

The evidence suggests that the consequences of stopping SWAFS would be a divergence of national and regional approaches, the removal of an important source of inspiration and learning for national programmes and research funding, reduced input and evidence for national policy making, a reduction in national-level attention to SWAFS issues, a reduction in the scale of funding available to tackle SWAFS issues, a reduction in the transfer of knowledge on SWAFS issues and mobility of researchers, and ultimately a

reduction in the potential of R&I to provide solutions to solving societal challenges. 397

R.7.3. Lessons learnt/Areas for improvement

SWAFS appears to have very high added value in terms of effectiveness, efficiency and synergies. It is highly appreciated by stakeholders and national funders across Europe and acts as a European 'beacon' for all those dealing with the links between science and society. Thousands of people working across Europe rely on SWAFS to help orientate their own activities, to provide evidence of good practice, to join fora focused on fields that matter to them, to join forces with peers in other countries (including third countries), and to help people around them to change their governance frameworks. SWAFS provides a scale of action and an EU and international dimension that does not appear to exist at national level. It increases internal coherence of approach within Horizon 2020 and external approach in terms of Member State policies and programmes.

R.8. S UCCESS STORIES FROM PREVIOUS F RAMEWORK P ROGRAMMES

RRI Tools 398

396 DRAFT PRELIMINARY LIST OF FINDINGS; Stock-tacking and meta-analysis of Science in Society projects

throughout FP6 and FP7 (RTD-B6-PP-00965-2013).

397 Evidence includes the survey on the added value of Horizon 2020, the NCP/SPC SWAFS/RRI flash survey, the

opinion of the SWAFS/RRI interim evaluation expert group, and the study on "Stock-tacking and meta-analysis…"

398 http://www.rri-tools.eu .

RRI Tools (2014-2016) was an FP7-funded CSA-SA project launched in 2014 that aimed to provide tools for the application of RRI and its key issues: public engagement with science, gender equality, open access, ethics and research integrity, and science education. It launched the online RRI toolkit, which provides more than 450 resources developed by renowned experts. It also undertook an ambitious training and advocacy programme on RRI, which it ran through RRI Tools Hubs and networks across Europe. It nurtured the growth of a community of practice consisting of nearly 1,000 members who actively contributed to the toolkit and acted as multipliers in their own countries, organisations and fields of work. The total requested EC contribution was EUR 6.9 million, total project cost EUR 7.7 million, and it involved a total of 29 partners from 28 countries.

GAP2 (Bridging the gap between science, stakeholders and policy makers Phase 2: Integration of evidence-based knowledge and its application to science and management of fisheries and the marine environment)

GAP2 (2011-2015) was an FP7-funded CSA-SA project which aimed to promote stakeholder participation in debates on policies on fisheries and the marine environment. To achieve this, it carried out 13 participatory research case studies that brought together scientists, fishermen and policymakers from 11 European countries. By combining the accumulated knowledge of local and regional fisheries stakeholders, the GAP2 team was able to involve them in decision-making processes that would ultimately affect them. GAP2 organised two workshops and held a three-day international symposium to discuss the challenges and opportunities arising from participatory research in fisheries. The total requested EC contribution was EUR 5.9 million, total project cost EUR 7.5 million, and it involved a total of 39 partners from 12 countries.

R.9. L ESSONS LEARNT /C ONCLUSIONS

R.9.1. Relevance

The overall relevance of SWAFS is high. It builds on support and good practices at European level since 2000. In addition, it is highly relevant – perhaps even ahead of the curve – when it comes to the emergence of a 'new R&I paradigm'. As such, it plays an important role connecting grassroots initiatives worldwide and implementing new modes of R&I in Horizon 2020. The relevance of SWAFS to tackling societal challenges is high but could be increased by paying closer attention to COP21/22, the SDGs and other recent overarching international agreements.

Without SWAFS there would be a lack of positive signals coming from 'Europe' about the need to increase public engagement in science and promote scientific careers, ensure that R&I is carried out to the highest ethical standards and in a responsible manner, and align the interests of science and society so that the outcomes of R&I are more appropriate and successful. Without SWAFS there would be much less co-ordination and contact between stakeholders across Member States and between Member States and the rest of the world, and a reduction in policy developments conducive to bringing science closer to society.

R.9.2. Effectiveness

The overall effectiveness of SWAFS is limited. The main reason for this is the small budget allocation for the scale of the ambition and the correspondingly broad range of activities. Even a significant increase in the budget would not necessarily significantly increase SWAFS effectiveness in all lines of activity. Nevertheless, an increase in the budget allocation to particular strategic areas (such as citizen science) would likely increase the programme's effectiveness; however, such an approach would simultaneously limit the programme's breadth. An additional solution to increase effectiveness would be to massively increase the mainstreaming of SWAFS approach in policies at all governance levels from local to global, and in the first place in Horizon 2020. This will not be achieved without strong co-ordination backed by sufficient human and financial resources at the EC unit(s) responsible. Importantly, this would require political will.

Additional reasons identified as part of this interim evaluation for limited effectiveness include: a lack of focus on implementing sustainable institutional changes, a lack of involvement of all stakeholder/organisation types in actually 'doing R&I', and funding one-off actions rather than acting as an enabler and catalyst for changing the R&I landscape. As such, more focus should be placed on sustainable institutional changes, requiring applicants to develop business or sustainability plans, or integrate their activities with existing structures. As part of this, the SWAFS KPI needs to be operationalised in each of the SWAFS lines more clearly and expected SMART impacts more precisely described in Work Programmes. The SDGs and MoRRI project's basket of indicators could provide additional indicators for projects to work towards.

A balance should be struck between openness to refresh the concept of RRI and the need to have concrete impact on the conduct of R&I.

Finally, the unbalanced representation of societal actors in SWAFS projects reduces effectiveness. While SWAFS is rather successful compared to the rest of Horizon 2020 at including CSOs (i.e. OTH-type participants) it is still not adequate compared to the needs and objectives. HES-type actors appear to have captured the 'SWAFS market', particularly as co-ordinators, and this has knock-on effects in terms of the activities conducted.

R.9.3. Efficiency

The overall efficiency of SWAFS is rather satisfactory given the relatively small budget, the high level of relevance and European added value, the importance attached to it by stakeholders and its effects in terms of increasing coherence of Member State approaches. The individual projects funded by SWAFS appear to be efficient and in line with expectations when compared to their intended actions/outcomes. ESOF and the SWAFS NCP network both play important roles in attracting participants to the programme and raising the profile of SWAFS internationally. Greater thought could be given, however, to attracting even more EU-13 and international participants.

SWAFS projects are smaller in size than the expected EUR 3.2 million. While there could be administrative efficiency savings in moving to larger projects, smaller project types might be more efficient at mobilising the balanced set of participant types required for this programme. Hence, higher administrative efficiency for the executive agency should be assessed against potential drawbacks in terms of effectiveness in policy/impact terms.

Additional actions to improve efficiency could include building stronger policy links between DG-RTD and REA, and finding a better balance between "open topics", guidance and taking a strategic approach.

R.9.4. Coherence

The internal coherence of SWAFS is high. SWAFS tackles transversal issues and is complementary and coherent with many lines of Horizon 2020, particularly the Societal Challenges and parts of LEIT. SWAFS is less coherent with other parts of Horizon 2020 such as the ERC, Access to Risk Finance, Innovation in SMEs and SEWP. Coherence with Horizon 2020 has been ensured to a certain extent by the mainstreaming of the RRI CCI.

The external coherence of SWAFS is also high. Gender, public engagement, ethics projects funded under governance, and the open access/open data lines appear to be well aligned with other EU funding programmes. Science education does not appear coherent with Education & Training 2020 though it does appear to be coherent with regional, national and international initiatives. Strong links can also be found with ESIF, though it is not clear how well or to what extent SWAFS contributes to ESIF policy and implementation agendas.

In the field of SWAFS/RRI, Member State approaches diverge somewhat though SWAFS helps reduce this divergence. The MoRRI project will provide useful evidence on the evolution and benefits of RRI and should help encourage greater coherence between funded actions and focus on desired impacts, and increase external coherence between different Member State approaches. It would be advantageous to start preparing EC policies and funding priorities in the field of SWAFS together with national institutions, in order to increase synergies and in order to promote greater external coherence.

R.9.5. EU Added Value

The EU added value of SWAFS is very high. SWAFS has become a beacon for all those in Europe dealing with the links between science and society. SWAFS fills a niche in the R&I funding landscape and brings a scale and an international focus that is lacking in similar national-level programmes. SWAFS helps set policy orientations at Member State level, provides evidence and results that inform Member State activities, promotes cross-fertilisation of innovative approaches, and greater coherence in policy responses.

Without EC support most funded projects would not go ahead due to the lack of alternative sources of funding, an inability to address pan-European issues at the national level, lack of access to knowledge, skills and expertise outside Horizon 2020, and a level of funding that is not available from national/regional schemes. The likely consequences of stopping SWAFS would be a divergence of national and regional approaches, a reduction in the scale of funding available to tackle these issues, a reduction in the transfer of knowledge and mobility of researchers, a reduction in national-level attention to SWAFS issues, and a corresponding reduction in the potential of R&I to provide solutions to societal challenges.