The value of replications goes beyond replicability and is associated with the value of the research it replicates: Commentary on Isager et al. (2025)

Gilad Feldman

doi:10.15626/MP.2024.4326

Downloads

PDF

Authors

Gilad Feldman University of Hong Kong

DOI:

https://doi.org/10.15626/MP.2024.4326

Keywords:

replications, value, commentary, replicability

Abstract

Replications are essential for rigorous credible science yet are still grossly undervalued and very rare. The value of replications is directly tied to the value of the research they aim to replicate, and replications offer many benefits that go far beyond the mere testing of replicability, such as including verifications and error detection, promoting long-term reproducibility of all research outputs, clarifying theory, refining measurement, and testing generalizability. We need far more independent pre-registered well-powered direct replications to strengthen the credibility of scientific findings. Isager et al. (2025)’s aim to define a formula for the value of replications based on over-simplified metrics of citation count and sample size is misaligned, already misunderstood, and may backfire by hindering the pursuit and publication of replications.

Metrics

Metrics Loading ...

References

Aiyer, S., Chan, W. Y., & Feldman, G. (2024). Outcome Bias in Evaluations of Ethical Decisions: Replication and Extensions of Gino, Moore, and Bazerman (2009). Collabra: Psychology, 10(1). https://doi.org/10.1525/collabra.126266

Borborema, R. S., de Moraes Ferreira, M., da Silva, A. L. M., Bastos, R. V. S., Fatori, D., Feldman, G., Seda, L., & Batistuzzo, M. C. (2023). Inaction Inertia Effect: Foregoing Opportunities as a Consequence of an Initial Failure to Act - a Replication-Extension Study in the Brazilian Population. OSF. https://doi.org/10.31234/osf.io/4u98x

Chandrashekar, S., & Feldman, G. (2024). On the process and value of direct close replications: Reply to shafir and cheek (2023) commentary on Chandrashekar et al. https://doi.org/10.17605/OSF.IO/RVPZF

Chen, J., Hui, L. S., Yu, T., Feldman, G., Zeng, S., Ching, T. L., Ng, C. H., Wu, K. W., Yuen, C. M., Lau, T. K., Cheng, B. L., & Ng, K. W. (2020). Foregone Opportunities and Choosing Not to Act: Replications of Inaction Inertia Effect. Social Psychological and Personality Science, 12(3), 333–345. https://doi.org/10.1177/1948550619900570

Clarke, B., Lee, P., Schiavone, S., Rhemtulla, M., & Vazire, S. (2023). The prevalence of direct replication articles in top-ranking psychology journals. In Advances in methods and practices in psychological science. https://doi.org/10.1037/amp0001385

Cook, B., Therrien, W., Waterfield, D., McClain, S., Fleming, J., Robinson, H., & Boyle, J. (2024). Same as it ever was. An Updated Review of Replication Studies in Special Education Journals. Remedial and Special Education, 07419325241248766. https://doi.org/10.1177/07419325241248766

CORE. (2024). Collaborative open-science and meta research [Retrieved from]. http://osf.io/5z4a8

Corker, K. (2020). Post of twitter/x [Retrieved from]. https://x.com/katiecorker/status/1222592990411214851

Dougherty, M., & Horne, Z. (2022). Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences. Royal Society Open Science, 9(8), 220334. https://doi.org/10.1098/rsos.220334

Edlund, J., Cuccolo, K., Irgens, M., Wagge, J., & Zlokovich, M. (2022). Saving science through replication studies. Perspectives on Psychological Science, 17(1), 216–225. https://doi.org/10.1177/1745691620984385

Giner-Sorolla, R. (2021a). Post on twitter/x [Retrieved from]. https://x.com/RogertheGS/status/1222847696140218369

Giner-Sorolla, R. (2021b). Time for change, part i reflections on the makeup and historical methodological practices in social [Retrieved from]. Society for Personality and Social Psychology talk (timestamp, 20(45). https://www.youtube.com/watch?v=0KIVLn4C5U8&t=1245s

Imada, H., Chan, W. F., Ng, Y. K., Man, L. H., Wong, M. S., Cheng, B. L., & Feldman, G. (2022). Rewarding More Is Better for Soliciting Help, Yet More So for Cash Than for Goods: Revisiting and Reframing the Tale of Two Markets With Replications and Extensions of Heyman and Ariely (2004). Collabra: Psychology, 8(1). https://doi.org/10.1525/collabra.32572

Ip, H. C., & Feldman, G. (2025). The complex misestimation of others’ emotions: Underestimation of emotional prevalence versus overestimation of emotional intensity and their associations with well-being. Personality and Social Psychology Bulletin. https://doi.org/10.17605/OSF.IO/BWMTR

Isager, P., van ’t Veer, A., & Lakens, D. (2025). Replication value as a function of citation impact and sample size. Meta-Psychology, 9. https://doi.org/10.15626/MP.2022.3300

Isager, P., Aert, R., Bahník, Š., Brandt, M., DeSoto, K., Giner-Sorolla, R., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and. knowledge constraints. Psychological Methods, 28(2), 438. https://doi.org/10.1037/met0000438

Koppel, L., Andersson, D., Tinghög, G., Västfjäll, D., & Feldman, G. (2023). We are all less risky and more skillful than our fellow drivers: Successful replication and extension of Svenson (1981). Meta Psychology, 7. https://doi.org/10.15626/MP.2021.2932

Makel, M., & Plucker, J. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513

Makel, M., Plucker, J., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688

Nosek, B., Hardwicke, T., Moshontz, H., Allard, A., Corker, K., Dreber, A., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157

Vazire, S. (2018). Implications of the credibility revolution for productivity, creativity, and progress. Perspectives on Psychological Science, 13(4), 411–417. https://doi.org/10.1177/1745691617751884

Zhu, M., & Feldman, G. (2023). Revisiting the Links Between Numeracy and Decision Making: Replication Registered Report of Peters et al. (2006) With an Extension Examining Confidence. Collabra: Psychology, 9(1). https://doi.org/10.1525/collabra.77608

Zwaan, R., Etz, A., Lucas, R., & Donnellan, M. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41, 120. https://doi.org/10.1017/S0140525X17001972