Rapporten Elsevier: The Future of Evaluation + Back to Earth – Landing Real World Impact in Research Evaluation

The first report, Back to Earth – Landing Real World Impact in Research Evaluation is based on a global survey of 400 institutional leaders, funding body executives and researchers. It finds broad consensus that current evaluation systems should be more holistic and include measures of impact; the report also considers possible ways forward.

· 66% of respondents believe academia has a moral responsibility to incorporate real-world impact into standard research evaluation.

· 53% state that a more holistic evaluation approach would enhance research cost-effectiveness.

· 51% identified at least one serious problem with current research evaluation methods.

· 56% cited a “lack of common frameworks or methodologies,” while 48% mentioned a “lack of consensus on what constitutes impact.” as barriers to change

There are some interesting and notable differences between the views of funders and academic leaders.

The second report, The Future of Evaluation – Emerging Consensus on a More Holistic System is the result of in-depth discussions with over 40 university leaders and funders around the world. Some highlights of their collective insights include:

• There is wide support for reform towards a system that also addresses education and societal impact.
• Any new processes must be trustworthy and scalable, including both qualitative assessment and quantitative measures of broader impact.
• A move towards a more interdisciplinary approach, emphasizing aspects such as team science, and diversity and inclusion, is seen as an important ingredient for success.
• In Europe, Open Science is a central theme: It is being widely embraced across Europe. Together with the drive for a more inclusive rewards and recognition system, this is creating a new research culture.

These findings echo the themes of the first report. This report also advances a possible high-level framework for evaluation, comprising five pillars: Resources; Education; Knowledge Creation Process; Knowledge Created; and Outcomes and Impact. This framework is Elsevier’s attempt to summarize the broad consensus emerging from our discussions about what a future evaluation system should address.