Scoping report

Directorate-General for Research and Innovation (European Commission)

Novembre – 2021

Executive Summary

Assessment of research quality and impact, and of researchers’ performance, is fundamental to selecting research proposals to fund, to deciding which researchers to recruit, promote or reward, and to identifying which research units and institutions to support.

The research process is undergoing digital transformation, and is becoming less linear and more collaborative and open, and more multidisciplinary with a larger diversity of outputs. At the same time, the current research assessment system often uses inappropriate and narrow methods to assess the quality, performance and impact of research and researchers. Notably, the quantity of publications in journals with high Journal Impact Factor and citations are currently the dominant proxies for quality, performance and impact. Many research funding and performing organisations are already taking steps to reform and improve the way they assess research and researchers, but progress remains slow, uneven and fragmented across Europe.

From March to November 2021, the European Commission consulted European and international stakeholders on how to facilitate and speed up reform so that the quality, performance and impact of research and researchers are assessed on the basis of more appropriate criteria and processes. The consultation identified objectives and outlines of a reformed research assessment system, with principles and actions that could be agreed between research funding and research performing organisations, as they have the responsibility to define their criteria and processes to assess their researchers and research projects.

The proposed way forward consists of a European agreement that would be signed by individual research funding organisations, research performing organisations and national/regional assessment authorities and agencies, as well as by their associations, all willing to reform the current research assessment system. The aim is for research and researchers to be evaluated based on their intrinsic merits and performance rather than on the number of publications and where these are published, promoting qualitative judgement with peer-review, supported by a more responsible use of quantitative indicators. The way in which the system is reformed should be appropriate for each type of assessment: research projects, researchers, research units, and research institutions. A reformed system should also be sufficiently flexible to accommodate the diversity of countries, disciplines, research cultures, research maturity levels, the specific missions of institutions, and career paths.

The agreement would confirm the commitment of the signatories to changes, along commonly agreed objectives, principles and actions. It would offer a space for individual institutions to test changes, for mutual learning, and to more safely and efficiently engage in reforms. An implementation plan would be established by the signatories, including deliverables, milestones and timeframes, in order to translate the commitments into effective changes. Measures for monitoring the progress made and for exchanging information would also be agreed among the signatories to ensure that commitments translate into tangible changes, and to ensure mutual learning for evidence-based changes. Researchers would need to be closely associated to the implementation and monitoring processes.

© European Union, 2021