How-to guides
Participation KPIs that are not headcounts
The number of participants tells a reader how far an activity reached. It does not say whether the process was fair, whether people could speak and be heard, or whether anything changed afterwards. Published frameworks already separate these questions and offer indicators for each.
What Horizon Europe itself tracks
Annex V of Regulation (EU) 2021/695, which established Horizon Europe, sets out the Key Impact Pathways, the indicators used to monitor the programme. One of the three societal pathways is called strengthening the uptake of R&I in society. None of its three indicators is a count of participants.
- Short term, co-creation: the number and share of projects where citizens and end-users contribute to the co-creation of R&I content.
- Medium term, engagement: the number and share of participating organisations that have citizen and end-user engagement mechanisms in place after the project ends.
- Longer term, societal R&I uptake: the uptake and outreach of co-created results and solutions.
These are programme-level indicators, compiled across all projects. They show what the funder looks for: a real contribution to content, and mechanisms that outlast the grant.
Indicator types and where they come from
| Indicator type | Example of what is assessed | Source |
|---|---|---|
| Process design | Clear purpose and mandate, unbiased framing, a group broadly representative of the public, barriers to taking part removed | OECD Evaluation Guidelines (2021) |
| Quality of the experience | Neutral facilitation, equal opportunity to speak, breadth of evidence, knowledge gains perceived by members | OECD Evaluation Guidelines (2021) |
| Influence on decisions | The commissioning body responded publicly, and accepted recommendations are monitored with public progress reports | OECD Evaluation Guidelines (2021) |
| Aftercare | Members were told how to follow the uptake of their recommendations | OECD Evaluation Guidelines (2021) |
| Institutional change | Public engagement mechanisms at research institutions (PE5), public engagement as a criterion in proposal evaluation (PE8), citizen science activities in research organisations (SLSE4) | MoRRI indicators report (2018) |
| Impact by domain | Society, economy, environment, governance, science and technology | MICS platform (Sprinks et al., 2022) |
| Scientific and participant outcomes | Scientific output, data quality, participant experience, wider societal or policy impact | ECSA Ten Principles (2015), principle 9 |
| Lasting engagement | Engagement mechanisms still in place after the project ends | Regulation (EU) 2021/695, Annex V |
Notes on each source
OECD. The Evaluation Guidelines for Representative Deliberative Processes (2021) describe three steps: process design integrity, deliberative experience and pathways to impact. They present their criteria as a minimum set that can be extended. The measurement methods they list are a member survey, a public survey, an organiser or expert survey, document review, observation of the sessions, interviews, media coverage review and policy analysis. They were written for processes with randomly selected members, such as citizens' assemblies, so not every criterion transfers to a workshop.
MoRRI. This project, reported by the Commission's DG Research and Innovation, built indicators for responsible research and innovation in six dimensions: gender equality, science literacy and science education, open access, public engagement, ethics and governance. Its public engagement indicators are measured for countries and research organisations, not for single projects, so they need adapting. The version opened for this post is the pre-publication annex of February 2018.
MICS. The Horizon 2020 project MICS (2019-2022) built an online tool with about 200 indicators, asked as questions, across five domains. According to the 2022 paper by its developers, the output includes impact scores and recommendations. The tool was reachable at mics.tools in October 2026, while its information site was under maintenance.
Citizen science evaluation. Principle 9 of the ECSA Ten Principles states what a citizen science programme should be evaluated for. The open framework by Kieslinger and colleagues (2017) goes further and proposes indicators from three perspectives, social, scientific and socio-ecological or economic, each at process level and impact level.
Choosing a small set
One workable pattern follows the OECD structure: one indicator on design, one on experience and one on impact, each with a named method. For example, the share of participants who report that they had an equal chance to speak (member survey), and whether the body that received the recommendations gave a public response (document review). A headcount can stay in the table as a measure of reach.
Sources
- Regulation (EU) 2021/695 establishing Horizon Europe, Annex V
- OECD (2021), Evaluation Guidelines for Representative Deliberative Processes
- European Commission (2018), The evolution of Responsible Research and Innovation in Europe: The MoRRI indicators report, annex, pre-publication version
- Sprinks et al. (2022), Design of a platform to measure the impact of citizen science
- CORDIS, MICS project, grant 824711
- ECSA (2015), Ten Principles of Citizen Science
- Kieslinger et al. (2017), The Challenge of Evaluation: An Open Framework for Evaluating Citizen Science Activities