Replication Value Usage and its Performance for Large Sample Sizes - Commentary on Isager et al. (2025)
DOI:
https://doi.org/10.15626/MP.2024.4324Keywords:
Replication Value, RVCn, replication, study selection, study comparisonAbstract
The Replication Value (RVCn) metric was introduced to help researchers prioritize studies for replication based on expected utility. While we welcome the introduction of this straightforward and systematic replication decision approach, we identify two limitations of the RVCn. First, when testing the “repeatability” of a study or systematically incorporating replication into a research workflow, the RVCn may not always be the most suitable metric to guide decisions. Use cases should consider the scope conditions of the metric. Second, the RVCn shows limited sensitivity in distinguishing between studies with large sample sizes. To address this, we propose a simple adjustment: a log transformation of the sample size component. This modification improves the metric’s discriminatory power for high-N studies and better aligns the (RVCn) with its intended purpose: guiding efficient and meaningful replication efforts.
Metrics
References
Amsalem, E., & Zoizner, A. (2022). Real, but limited: A meta-analytic assessment of framing effects in the political domain. British Journal of Political Science, 52(1), 221–237.
Arel-Bundock, V., Briggs, R. C., Doucouliagos, H., Mendoza Aviña, M., & Stanley, T. D. (2024). Quantitative political science research is greatly underpowered. The Journal of Politics.
Chambers, C. (2017). The seven deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton University Press.
Dubèl, R., Schumacher, G., Homan, M. D., Peterson, D., & Bakker, B. N. (2024). Replicating and extending soroka, fournier, and nir: Negative news increases arousal and negative affect. Media and Communication, 12.
Freese, J., & Peterson, D. (2017). Replication in social science. Annual Review of Sociology, 43(1), 147–165.
Huber, C., Dreber, A., Huber, J., Johannesson, M., Kirchler, M., Weitzel, U., Abellán, M., Adayeva, X., Ay, F. C., Barron, K., et al. (2023). Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs. Proceedings of the National Academy of Sciences, 120(23), e2215572120.
Isager, P., van ’t Veer, A., & Lakens, D. (2025). Replication value as a function of citation impact and sample size. Meta-Psychology, 9. https://doi.org/10.15626/MP.2022.3300
Kertzer, J. D. (2022). Re-assessing elite-public gaps in political behavior. American Journal of Political Science, 66(3), 539–553.
Lang, A., Newhagen, J., & Reeves, B. (1996). Negative video as structure: Emotion, attention, capacity, and memory. Journal of Broadcasting & Electronic Media, 40(4), 460–477.
Soroka, S., Fournier, P., & Nir, L. (2019). Cross-national evidence of a negativity bias in psychophysiological reactions to news. Proceedings of the National Academy of Sciences, 116(38), 18888–18892.
Soroka, S., & McAdams, S. (2015). News, politics, and negativity. Political communication, 32(1), 1–22.
Sun, Y., Shen, L., Pan, Z., & Qian, S. (2025). Toward a more powerful experimental communication science: An assessment of two decades’ research (2001–2023). Communication Research, 00936502241308599.
Published
Issue
Section
License
Copyright (c) 2025 Linda Bomm, Delaney J. Peterson, Bert N. Bakker
This work is licensed under a Creative Commons Attribution 4.0 International License.
