Queen Mary signs international declaration to improve research evaluation

The San Francisco Declaration on Research Assessment (DORA)

The San Francisco Declaration on Research Assessment (DORA) is a worldwide initiative aimed at improving the ways in which outputs of scholarly research are evaluated.  

The declaration promotes responsible use of metrics when measuring and evaluating research and aims to eliminate the use of journal-based metrics, such as Journal Impact Factors, in relation to funding, appointment and promotion considerations.  

Instead it encourages assessment of research on its own merits rather than on the basis of the journal in which it is published. 

Professor Colin Bailey, President and Principal of Queen Mary University of London, said: “Queen Mary is very proud to be signing up to DORA. The quality and impact of our research is recognised all over the world and committing to this will ensure that it is assessed fairly, using responsible metrics. We are also demonstrating to our researchers that we value the quality of their work rather than where it is published.” 

Committing to best practice

A number of high profile universities across the world have signed up to DORA, which also continues to gain commitment among funding agencies and journal publishing organisations to adopt the initiative.  

By signing up to DORA, Queen Mary is committing to the adoption of a number of practices in research assessment.  

These include the following recommendations for the University: 

• Queen Mary should be explicit about the criteria used to reach hiring, tenure, and promotion decisions. It should clearly highlight, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published 
• For the purposes of research assessment, Queen Mary should consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative measures 

DORA also includes the following recommendations for the University’s researchers: 

• When involved in committees making decisions about funding, hiring, tenure, or promotion, researchers should make assessments based on scientific content rather than publication metrics alone 
• Wherever appropriate, researchers should cite primary literature in which observations are first reported rather than reviews in order to give credit where credit is due 
• Researchers should use a range of article metrics and indicators on personal/supporting statements, as evidence of the impact of individual published articles and other research outputs 
• Researchers should challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs

A world-leader in fairer research

Professor Andrew Livingston, Vice-Principal (Research and Innovation) at Queen Mary University of London, said: “By adopting these recommendations, Queen Mary demonstrates its commitment to being a world-leader in fairer research evaluation. DORA aligns with our Strategy 2030 aim to produce world-leading research that is accessible by adherence to recognised open science principles. In particular, it aligns with our aim to regularly assess how we support, review and undertake research to foster diversity and inclusion and ensure that our research practice is of the highest quality in terms of research ethics and integrity.” 

The recommendations included in DORA are also complemented by other initiatives to improve the evaluation of research such as the Leiden Manifesto (2015) and the establishment of the Forum for Responsible Research Metrics in the UK.  

Taking these developments forward, Queen Mary has developed a series of institutional principles which will guide the University on the use of research metrics. These include:  

• Quantitative evaluation should support qualitative, expert assessment 
• Use a combination of indicators 
• Research evaluation should have clear objectives 
• Differences between research disciplines should be acknowledged 
• Data sources should be robust, accurate and open for verification 
• Research indicators and data sources should be regularly reviewed and updated 
• A rounded and comprehensive assessment of research should be used 

More information:

• Find out more about DORA