Measurements of Susceptibility to Anchoring are Unreliable: Meta-Analytic Evidence From More Than 50,000 Anchored Estimates

Lukas Röseler; Lucia Weber; Katharina A. C. Helgerth; Elena Stich; Miriam Günther; Paulina Tegethoff; Felix Stefan Wagner; Astrid Schütz

doi:10.15626/MP.2022.3236

Downloads

PDF

Authors

Lukas Röseler University of Bamberg
Lucia Weber University of Bamberg
Katharina A. C. Helgerth University of Bamberg
Elena Stich University of Bamberg
Miriam Günther University of Bamberg
Paulina Tegethoff University of Bamberg
Felix Stefan Wagner University of Bamberg
Astrid Schütz University of Bamberg

DOI:

https://doi.org/10.15626/MP.2022.3236

Keywords:

anchoring effect, meta-analysis, reliability, open data, personality, moderator

Abstract

Theories on anchoring effects—the assimilation of numerical estimates toward previously considered numbers—have been used to derive hypotheses that susceptibility to anchoring is correlated with certain personality traits. Thus, for the last decade, a considerable amount of research has investigated relationships between people’s susceptibility to anchoring and personality traits (e.g., intelligence, the Big Five, narcissism). However, many of the findings are contradictory. We suspect that this inconsistency is grounded in imprecise measurements. Unfortunately, few reports have disclosed estimates of the susceptibility scores’ reliability (e.g., Cronbach’s alpha). We created a large and open data set of anchoring susceptibility scores and conducted a meta-analysis to test how extensive the reliability problem is. Results suggest that the reliability of most tasks is very low. In the few cases in which the reliability is acceptable, the validity of anchoring scores is questionable. We discuss requirements for further attempts to solve the reliability problem.

Metrics

Metrics Loading ...

References

Auguie, B. (2017). GridExtra: Miscellaneous functions for "grid" graphics [R package version 2.3]. Retrieved from https://CRAN.R-project.org/package=gridExtra

Bahník, Š. (2021a). Anchoring does not activate examples associated with the anchor value. Advance online publication.

Bahník, Š. (2021b). Anchoring without scale distortion. Judgment and Decision Making, 16(1), 131.

Bergman, O., Ellingsen, T., Johannesson, M., & Svensson, C. (2010). Anchoring and cognitive ability. Economics Letters, 107(1), 66–68.

Berthet, V. (2021). The measurement of individual differences in cognitive biases: A review and improvement. Frontiers in Psychology, 12, 630177.

Bonaccio, S., & Dalal, R. S. (2006). Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences. Organizational Behavior and Human Decision Processes, 101(2), 127–151.

Bystranowski, P., Janik, B., Próchnicki, M., & Skórska, P. (2021). Anchoring effect in legal decision-making: A meta-analysis. Law and Human Behavior, 45(1), 1–23.

Chapman, G. B., & Johnson, E. J. (1994). The limits of anchoring. Journal of Behavioral Decision Making, 7(4), 223–242.

Cheek, N. N., & Norem, J. K. (2018). On moderator detection in anchoring research: Implications of ignoring estimate direction. Collabra: Psychology, 4(1), 12.

Cheek, N. N., & Norem, J. K. (2019). Are big five traits and facets associated with anchoring susceptibility? Social Psychological and Personality Science, 9(2), 194855061983700.

Cheek, N. N., & Norem, J. K. (2022). Individual differences in anchoring susceptibility: Verbal reasoning, autistic tendencies, and narcissism. Personality and Individual Differences, 184, 111212.

Cooper, W. H., & Withey, M. J. (2009). The strong situation hypothesis. Personality and Social Psychology Review, 13(1), 62–72.

Dragulescu, A. A. (2014). Xlsx: Read, write, format Excel 2007 and Excel 97/2000/XP/2003 files [R package version 0.5.7]. Retrieved from https://CRAN.R-project.org/package=xlsx

Epley, N., & Gilovich, T. (2001). Putting adjustment back in the anchoring and adjustment heuristic: Differential processing of self-generated and experimenter-provided anchors. Psychological Science, 12(5), 391–396.

Epley, N., & Gilovich, T. (2006). The anchoring-and-adjustment heuristic: Why the adjustments are insufficient. Psychological Science, 17(4), 311–318.

Fechner, G. T. (1860). Elemente der Psychophysik. Breitkopf und Härtel.

Frederick, S. W., & Mochon, D. (2012). A scale distortion theory of anchoring. Journal of Experimental Psychology: General, 141(1), 124–133.

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference; 11.0 update (4th ed.). Allyn & Bacon.

Gertner, A., Zaromb, F., Schneider, R., & Matthews, G. (2016). The assessment of biases in cognition: Development and evaluation of an assessment instrument for the measurement of cognitive bias (Tech. Rep. No. MTR160163). The MITRE Corporation.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (7th ed.). Pearson Education Limited.

Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186.

Janiszewski, C., & Uy, D. (2008). Precision of the anchor influences the amount of adjustment. Psychological Science, 19(2), 121–127.

Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13).

Lee, C.-Y., & Morewedge, C. K. (2022). Noise increases anchoring effects. Psychological Science, 33(1), 60–75.

Li, L., Maniadis, Z., & Sedikides, C. (2021). Anchoring in economics: A meta-analysis of studies on willingness-to-pay and willingness-to-accept. Journal of Behavioral and Experimental Economics, 90, 101629.

Mochon, D., & Frederick, S. (2013). Anchoring in sequential judgments. Organizational Behavior and Human Decision Processes, 122(1), 69–79.

Mussweiler, T., & Strack, F. (1999). Comparing is believing: A selective accessibility model of judgmental anchoring. European Review of Social Psychology, 10(1), 135–167.

Mussweiler, T., Strack, F., & Pfeiffer, T. (2000). Overcoming the inevitable anchoring effect: Considering the opposite compensates for selective accessibility. Personality and Social Psychology Bulletin, 26(9), 1142–1150.

Parsons, S., Kruijt, A.-W., & Fox, E. (2018). Psychological science needs a standard practice of reporting the reliability of cognitive behavioural measurements [Advance online publication].

Pohl, R. F., Eisenhauer, M., & Hardt, O. (2003). SARA: A cognitive process model to simulate the anchoring effect and hindsight bias. Memory, 11(4–5), 337–356.

Röseler, L. (2021). Anchoring effects: Resolving the contradictions of personality moderator research [Doctoral dissertation, University of Bamberg].

Röseler, L., Bögler, H. L., Koßmann, L., Krueger, S., Bickenbach, S., Bühler, R., della Guardia, J., Köppel, L.-M. A., Ponader, S., Roßmaier, K., et al. (2022). Replicating Epley and Gilovich: Need for cognition, cognitive load, and forewarning do not moderate anchoring effects. Advance online publication.

Röseler, L., & Schütz, A. (2022). Hanging the anchor off a new ship: A meta-analysis of anchoring effects. Advance online publication.

Röseler, L., Schütz, A., & Starker, U. (2019). Cognitive ability does not and cannot correlate with susceptibility to anchoring effects. Advance online publication.

Schindler, S., Querengässer, J., Bruchmann, M., Bögemann, N. J., Moeck, R., & Straube, T. (2021). Bayes factors show evidence against systematic relationships between the anchoring effect and the big five personality traits. Scientific Reports, 11(1), 7021.

Schultze, T., Gerlach, T. M., & Rittich, J. C. (2018). Some people heed advice less than others: Agency (but not communion) predicts advice taking. Journal of Behavioral Decision Making, 31(3), 430–445.

Schultze, T., Mojzisch, A., & Schulz-Hardt, S. (2017). On the inability to ignore useless advice. Experimental Psychology, 64(3), 170–183.

Smeesters, D., Yzerbyt, V. Y., Corneille, O., & Warlop, L. (2009). When do primes prime? The moderating role of the self-concept in individuals’ susceptibility to priming effects on social behavior. Journal of Experimental Social Psychology, 45(1), 211–216.

Stanley, D. (2021). apaTables [R package version 2.0.8]. Retrieved from https://CRAN.R-project.org/package=apaTables

Team, R. C. (2018). R [R Foundation for Statistical Computing. Vienna, Austria]. Retrieved from https://www.R-project.org/

Teovanović, P. (2019). Individual differences in anchoring effect: Evidence for the role of insufficient adjustment. Europe’s Journal of Psychology, 15(1), 8–24.

Thorsteinson, T. J. (2011). Initiating salary discussions with an extreme request: Anchoring effects on initial salary offers. Journal of Applied Social Psychology, 41(7), 1774–1792.

Townson, C. D. (2019). The anchoring effect: A meta-analysis [Doctoral dissertation, Michigan State University]. ProQuest Dissertations Theses Global.

Tsuji, S., Bergmann, C., & Cristia, A. (2014). Community-augmented meta-analyses: Toward cumulative data assessment. Perspectives on Psychological Science, 9(6), 661–665.

Turner, B. M., & Schley, D. R. (2016). The anchor integration model: A descriptive model of anchoring effects. Cognitive Psychology, 90, 1–47.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

Wickham, H. (2016). ggplot2: Elegant graphics for data analysis (2nd ed.). Springer.

Xie, Y. (2021). knitr [R package version 1.36]. Retrieved from https://CRAN.R-project.org/package=knitr

Yoon, H., Scopelliti, I., & Morewedge, C. K. (2021). Decision making can be improved through observational learning. Organizational Behavior and Human Decision Processes, 162, 155–188.