Committee for Open Science, Version 1, November 2019

Many stakeholders agree that the implementation of open science requires all research processes and activities to be taken into account in evaluation. This concerns both publications and research projects, individuals, collectives or institutions, and their interactions with society. The National Open Science Plan emphasizes in particular the importance of taking into account a qualitative rather than a quantitative approach and cites the Leiden manifesto (Hicks & al., 2015) and the San Francisco Declaration on Research Evaluation (DORA, 2012). Even if similar approaches existed before the open science movement, it aims to make them visible, legitimate and sustainable.

These recommendations of the Open Science Committee are based on those of the Open Science Platform Policy (OSPP), which unites the efforts and strategies of the European Union around 8 priority pillars, two of which are specific to evaluation issues: encouraging open practices and their recognition; and the deployment of a new generation of research indicators. In these recommendations, intended for all institutions piloting evaluation systems, we have identified two essential dimensions that are at the heart of open science.

1/ The Open Science Committee encourages the promotion of good practices in open science, in particular those that push for the availability of all types of documents and more broadly of research products, including but not limited to data and publications. Indeed, this availability will facilitate, on the one hand, the full recognition of the diversity of individual and collective contributions to research and, on the other hand, will encourage people to discover the content of these productions and to evaluate their quality.

2/ The Open Science Committee supports the recognition of the full range of research activities, in the plurality of disciplinary fields, as opposed to mechanisms that would focus only on scientific publications, or even on a limited part of them or on resulting indicators (McKiernan & al., 2019). Similarly, the recognition of new configurations of multidisciplinary forms of knowledge, and in particular of citizen science, must be strengthened.

To do this, the following are distinguished:

• 17 types of products and documents; Table 1: “Types of documents and productions developed within the entire research cycle and their modes of open dissemination”,
• 12 types of activities; Table 2: “Types of activities valued by open science and eligible for evaluation”. The activities in Table 2 may result in documentary production, and in this case it falls under the recommendations in Table 1.

In compliance with the legal and regulatory principles governing the opening and dissemination of research results and data, we recommend the acknowledged principle of “as open as possible, as closed as necessary” be followed.

The documents and productions referred to in Table 1 may be considered fully open only if they are made public under the conditions of a so-called free or open licence. Indeed, an object distributed without a licence can certainly be consulted by third parties, which constitutes a minimum form of opening, but its methods of reuse are not known. The final choice of licence is made by those who make their production public, in accordance with Decree No. 2017-638 (Prime Minister, 2017).

Beyond a licence, the systems for publishing, hosting and archiving documents and productions are intended to comply with the exemplary criteria defined by the Open Science Committee (2019).

The Open Science Committee recommends that all bodies and institutions involved in evaluation activities discuss, disseminate and adapt these good practices to their context. This includes, in particular, research funding agencies, publication and evaluation bodies. Communities and institutions remain sovereign in their adaptation of these recommendations, particularly in the aggregation, qualification and prioritization of types of productions and documents. The Open Science Committee expects transparency from these operations, including the justification of the reasons why certain productions or activities would not be taken into account in their evaluations.

Contact Information

This document is intended to be enriched and updated. To comment on it, please write to coso@recherche.gouv.fr

References cited and documents used

Open Science Committe (2019). Examplarity criteria for funding from the National Open Science Fund through platforms, infrastructures and editorial content. Available at: https://www.ouvrirlascience.fr/examplarity-criteria-for-funding-from-the-national-open-science-fund/

European Commission (2017). Evaluation of Research Careers fully acknowledging Open Science Practices Rewards, incentives and/or recognition for researchers practicing Open Science, 2017. Available at: https://ec.europa.eu/research/openscience/pdf/ospp_rewards_wg03112017.pdf

DORA (2012). San Francisco Declaration on Research Assessment. Available at: https://sfdora.org/read/

HCERES, Guides des produits de la recherche, 2019. Available at: https://www.hceres.fr/fr/guides-des-produits-de-la-recherche-et-activites-de-recherche-0

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520, pp. 429-431. Available at: http://www.leidenmanifesto.org/

McKiernan EC, Schimanski LA, Muñoz Nieves C, Matthias L, Niles MT, Alperin JP. (2019). Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations. PeerJ Preprints DOI : 10.7287/peerj.preprints.27638v2

Prime Minster (2017). Décret n° 2017-638 du 27 avril 2017 relatif aux licences de réutilisation à titre gratuit des informations publiques et aux modalités de leur homologation, JORF n°0100 du 28 avril 2017. Available at: https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000034502557&categorieLien=id

Indicative bibliography to go further

European Commission. (2016). Evaluation of Research Careers fully acknowledging Open Science practices. Available at: https://ec.europa.eu/research/openscience/pdf/os_rewards_wgreport_final.pdf

Krzton, A. (2019). Support Scholars Who Share: Combating the Mismatch between Openness Policies and Professional Rewards. In ACRL 2019 (pp. 578–586). ACRL. Available at: http://aurora.auburn.edu/handle/11200/49374

Nichols, D. M., Twidale, M. B., (2017). Metrics for Openness. JASIST 68 (4), 1048–1060.
Available at: https://researchcommons.waikato.ac.nz/bitstream/handle/10289/10842/nichols-metrics-openness-jasist.pdf

Turckheim, E. de, Legouy, É. L., & Leclerc, L.-A. (2016). EREFIN Groupe de travail inter-établissements sur l’évaluation de la recherche finalisée. Rapport. Available at: https://prodinra.inra.fr/?locale=fr#!ConsultNotice:352302

Wynne, R. (2019). Using ORCID to Re-imagine Research Attribution. Available at: https://orcid.org/blog/2019/07/25/using-orcid-re-imagine-research-attribution

Table 1: Types of documents and productions developed within the entire research cycle and their modes of open dissemination

Table 2: Types of activities valued by open science and eligible for evaluation

Read the report