Call for commentaries on two target papers: "Responsible Research Assessment: Implementing DORA for hiring and promotion in psychology“

2022-12-14

The San Francisco Declaration on Research Assessment (DORA) calls on academic institutions to abandon the use of invalid quantitative metrics of research quality and productivity in hiring and promotion. While it is relatively easy to agree on what should not be done anymore  (e.g., use journal impact factors for the evaluation of an individual’s performance), it seems to be much harder to agree on what should be done instead.  In fact, multiple parallel initiatives are currently working on answering that question. The German Psychological Society (DGPs) signed DORA in 2021 and then tasked a group of experts among its members with preparing a proposal on how DORA’s key principles may be practically implemented in academic hiring and promotion procedures. The outcome consists of two closely related target papers that have now been published as preprints: The first paper (https://psyarxiv.com/rgh5b) proposes four broad principles of responsible research assessment and a two-step procedure that integrates quantitative and qualitative aspects of assessment. The second paper (https://psyarxiv.com/5yexm/) proposes a specific,  ready-to-use procedure for assessing research quality that may be used by hiring and promotion committees. 

 

Here, we ask the members of the scientific community to provide substantive criticism, interpretation, or elaboration on one or both articles and submit their comments to Meta-Psychology. It is our goal to ultimately publish these comments alongside the final versions of the two target papers. Comments may focus on either of the two target papers, or on both of them. Comments may be supportive or critical, focus on the broad picture or more specific aspects of it, and they may even contain alternative proposals as to how current assessment practices may be improved. We expect the final versions of the two target papers to be significantly improved through this process. We also hope that, by engaging the community this way, we will be able to motivate as many colleagues as possible to join us in this attempt to improve the ways in which highly consequential assessments in academia are conducted.   

Formalities:

  • 2023-03-30: Deadline for submission.
  • Regular commentaries should not exceed 1500 words and must link to both target papers in a note on their front page.
  • If alternative ways of implementing DORA principles are suggested (e.g., for other areas than research, such as teaching), which is encouraged, longer manuscripts can be submitted. In this case, please contact the special issue editor, Daniel Lakens, d.lakens@tue.nl, before submission.
  • Submit your commentary at https://open.lnu.se/index.php/metapsychology/about/submissions with the regular workflow (including uploading a preprint on PsyArXiv). In the submission form, please select “Special issue” from the drop-down list.
  • Commentaries will be sent to an editorial review team made up of both board members and external peer reviewers.