Super-Size Me: An Unsuccessful Preregistered Replication of the Effect of Product Size on Status Signaling

https://doi.org/10.15626/MP.2020.2538 Article type: Replication Report Published under the CC-BY4.0 license Open data: Yes Open materials: Yes Open and reproducible analysis: Yes Open reviews and editorial process: Yes Preregistration: Yes Edited by: Rickard Carlsson Reviewed by: Evelina Thunell & Sarahanne Field Analysis reproduced by: Lucija Batinović All supplementary files can be accessed at the OSF project page: https://doi.org/10.17605/OSF.IO/KVGWR

Why do consumers choose larger portions, for example at a fast-food restaurant, despite the potential negative consequences of such choices on their health and well-being? Given that larger portion sizes give rise to increased consumption (i.e, the portion size effect; Vandenbroele et al., 2019;Zlatevska et al., 2014), understanding the factors influencing portion choice is of importance to efforts in reducing excessive eating-related problems like obesity. Various underlying mechanisms of preferences for larger portions are available in the literature. These explanations include value for money (larger portions have a lower price per unit), mindless eating (consumers are inattentive to their choices, for example while dining with others), and estimation bias (consumers are often uninformed about reference portion sizes and they fail to estimate appropriate amount to eat; for a review, see Steenhuis & Poelman, 2017). To add to these explanations, Dubois et al. (2012) presented a novel postulation: consumers choose larger-sized food options to signal social status. Put simply, people do not choose larger portions to satiate the need for food, but to satiate the need for status.
The evidence for this novel hypothesis attracted attention. At the time of writing, the study has been cited over 290 times on Google Scholar, and also appeared in mainstream media such as The New York Times (Warren, 2011), Scientific American (Grewal, 2011), and The Atlantic (Villarica, 2011). To illustrate the extent of publicity, Warren (2011) reported the study in The New York Times as a "fascinating research that links obesity with status" and proposed that anti-obesity campaigns could benefit from altering the size-to-status relationship. While the impact, theoretical importance, and relevance of Dubois et al.'s (2012) research make it a worthy candidate for replication, Field et al. (2019) recommend also evaluating the statistical findings in the original work to judge the necessity of a replication.
When converted into Jeffreys-Zellner-Siow Bayes factors (see Field et al., 2019, p. 2), these differences corresponded to Bayes factors of 2026 (large vs. small), 5 (large vs. medium), and 1 (medium vs. small). According to the classification recommended by Field et al. (2019), while the difference between large and small portion sizes show very strong evidence for the effect (i.e., the data are 2026 times more likely to occur under the alternative hypothesis relative to the null hypothesis), the Bayes factor values drop substantially for the remaining comparisons and indicate ambiguous evidence for the comparison between medium and small size portions with a Bayes factor of 1. We therefore argue that these statistical discrepancies in the original results also warrant replication.
Despite the abovementioned implications of Dubois et al.'s (2012) findings for marketing theory as well as policymaking, independent replications of this study are not available in the literature, to the best of our knowledge. In this replication report, we present a preregistered direct replication of the first experiment in Dubois et al. (2012), which tested the primary hypothesis that choosing larger sizes in a set of food options is associated with greater status but does not significantly influence other dimensions that are not related to status such as perceived niceness, honesty, or attractiveness.

Choice of direct replication
Direct replications strive to keep as many factors as possible identical to the original (e.g., survey procedure, wording, measures), while conceptual replications aim to test the central notion of a past finding, for instance by changing the procedure or the stimuli involved. While some authors argue that replications should focus on conceptual extensions of original hypotheses (e.g., Lynch et al., 2015;Stroebe & Strack, 2014), Simons (2014) contends that the space of possible effects that can moderate a finding is infinite and all findings, reliable or unreliable, can be attributed to some unidentified moderators, thereby hindering accumulation of scientific evidence. Thus, Simons (2014) maintains that the purpose of replications is to determine reliability of findings, which can be achieved most successfully via direct replications by independent researchers. In accord with this perspective, we chose to conduct a direct replication to test the primary hypothesis in Dubois et al. (2012) and examine the reliability of the portion size effect on status perceptions.
Open science practices Data collection and analyses plans were preregistered prior to data collection (see https://aspredicted.org/BUC_HUQ). Data, analyses, and materials are available at https://osf.io/ue458/.

Design and sample
As with the original experiment (Dubois et al., 2012, Study 1), our replication comprised a 3 (size of observed choice: small, medium, large) x 3 (product: coffee, pizza, smoothie) x 2 (dimension: status, nonstatus) mixed design with size and product as between-subject factors and dimension as a withinsubject factor. Participants were randomly assigned to one of nine conditions.
The original experiment was based on 183 undergraduates (74 males); data for the replication experiment were collected from the Amazon MTurk participant pool (N = 415, 214 males, Mage = 37.9, SDage = 11.8). Participants received $0.50 as compensation. Power analyses indicated that our sample size had 99% power to detect the original effect size of product size on status inferences and 99.9+% power to detect the original mixed interaction effect between product size and status dimensions (alpha level of 5%; see supplementary materials for effect size conversion and power analyses).

Procedure and Measures
Identical to the original study, participants were asked to read about a consumption scenario in which a consumer chose small, medium, or large option for one of three products (coffee, pizza, or smoothie). While price was not presented in the pizza and smoothie conditions, the coffee was presented as free. An example scenario is presented below: "You're at a local smoothie shop. An individual enters in the smoothie shop, and asks for a smoothie. The cashier explains to him that the smoothies come in three sizes: small (16 oz), medium (24 oz) and large (30 oz), and asks him which size he would like to choose. The individual orders the largest size." Following the consumption scenario, participants were asked to make judgments of the target person on status-related (this person has high status, is respected; a = 0.75) and nonstatus-related (this person is honest, nice, attractive; a = 0.80) dimensions. These judgments were recorded with a 7-point Likert-type scale (1 = "strongly disagree" to 7 = "strongly agree").
Participants also responded to an attention check, in which they had to identify which among six traits had been presented to them (Roughness, Ruggedness, Niceness, Brutality, Suavity, Intelligence). 338 participants correctly identified "niceness" in the attention check (171 males, 167 females, Mage = 39.05, SDage = 11.90). Excluding the participants who failed to identify the attention check did not affect the results. To maximize statistical power and give the original effect the best chance to be detected, we chose to report all data without any exclusion (results obtained excluding participants who failed the attention check are available on the project OSF page).

Main Analyses
Descriptive statistics for each condition and measure are presented in Table 1. We first analyzed the full 3 (size) x 3 (product) x 2 (dimension) mixedfactorial model using repeated measures ANOVA. Note that this analysis for the entire design has not been reported in the original study. As seen in Table 2, the only significant effect was the main effect of dimension, such that, regardless of the experimental condition, participants rated the target consumers lower on the status-related dimensions (M = 4.43, SE = 0.05) than the nonstatus-related dimensions (M = 4.77, SE = 0.05; t(406) = 9.56, p < .001). Next, following the analysis steps in the original study, we collapsed the different product scenarios and conducted a 3 (size) x 2 (dimension) repeated measures ANOVA. Again, the main effect of dimension was significant (F(1, 412) = 91.69, p < .001, η²p = 0.18), indicating that participants rated the individual higher on nonstatus compared to status dimension; yet, more pertinent to the main findings of the original study that larger size choices would be associated with perceived status, the interaction between size and dimension was not sig-   Comparisons of status-and nonstatus-related dimensions across different product sizes.
In Table 3, we also present a comparison between the results of the original study and the replication study, including following the replication classification of LeBel et al. (2019). Although the original study reported rather large effects of product size on status perceptions, in the replication study effects were nonsignificant in the opposite of the hypothesized direction (status perceptions were lowest in the large product size condition). Table 3 Comparisons for status-related dimensions between the replication and original study.

Supplementary Bayesian Analyses
Because frequentist statistics and interpretation of p-values are generally not informative in quantifying evidence for the null hypothesis, Bayesian analyses are recommended for establishing evidence of absence (Keysers et al., 2020). We therefore supplemented our analyses with Bayes factors, which provide us the plausibility of the observed data under different models employing the null versus alternative hypothesis. These were not preregistered. Open source software JASP was used to conduct the Bayesian ANOVAs reported in this section (Bergh et al., 2020;JASP Team, 2020).
First, we ran a Bayesian repeated measures ANOVA for the 3 (size: small, medium, large) x 2 (dimension: status, nonstatus) mixed model. Table 4 compares the likelihood of all possible models relative to the null model; however, given that our focus is on the predictive performance of each component, analysis of effects presented in Table 5 is more informative (Bergh et al., 2020;Keysers et al., 2020). To generate the analysis of effects, we used the recommended matched models' option, which, in our case, compares the model with the interaction effect only with the models that exclude the interaction, thereby providing a more conservative estimate for each factor's contribution (Keysers et al., 2020;Mathôt, 2017). As seen in Table 5, the results revealed that data are 16 times more likely to occur under the models that exclude the interaction effect, which can be concluded as decisive evidence for the null hypothesis with respect to Kass and Raftery's (1995) reference values.
We also conducted a Bayesian ANOVA to examine the effect of portion size on status dimensions only, as we did in the main analyses. As seen in Tables 6 and 7, data are about 24 times more likely to occur under the models excluding the portion size effect, thereby providing decisive evidence for the null hypothesis stating that portion size choices are not associated with status perceptions. We further illustrated this lack of evidence by plotting the model with portion size effect. As seen Figure 2, 95% credible intervals for different levels of portion size overlap substantially, indicating no differences among the levels.

Replication Study
Original  Table 4 Model comparison for all models under consideration for the replication data. Note. Compares models that contain the effect to equivalent models stripped of the effect. Higherorder interactions are excluded. Table 6 Comparison of the portion size effect model with the null model for the replication data  Table 7 Analysis of the effect of portion size Effects P(incl) P(incl|data) BFexcl SIZE 0.500 0.040 23.899 In order to compare the effect sizes, we recalculated the original effect sizes from the reported Fvalues using two independent tools (Lakens, 2013;Uanhoro, 2017). For both the interaction and the effect of product size on status, we found smaller effect sizes and slightly higher p-values than the original. We compared effect sizes in the present replication, effect sizes reported in the original paper, and effect sizes we recalculated based on the summary statistics provided in the original paper, and we provided 90% confidence intervals (as it is customary with η²p , which cannot be smaller than 0).  Table 3.

General Discussion
In this work, we failed to replicate the first experiment from Dubois et al. (2012), which found that consumers associated larger portion choices with higher status. What could be the reasons for this replication failure? First, although we conducted a direct replication in terms of materials and procedure, one major difference from the original study was the study sample. The original experiment was based on 183 undergraduate students (average age not available); the replication was based on 415 participants from MTurk (Mage = 37.9). Assuming that the undergraduate sample was much younger, age might have influenced the results, such that while students could associate mundane products like coffee, smoothie, or pizza with status, older consumers could not. Second, another possible explanation is that students on average have lower socioeconomic status, and given the link between low socioeconomic status and higher propensity for status consumption (Dubois & Ordabayeva, 2015), students were more likely to associate larger portion sizes with status. However, there are plenty of successful replications in which an original finding obtained with students was successfully replicated on MTurk samples (Ziano, Wang, et al., 2021;Ziano, Yao, et al., 2020).
A third explanation could be the association between status and health behaviors, which have long been associated with higher socioeconomic status (Pampel et al., 2010). Particularly, healthy food consumption is prevalent among middle and upper social class while consumption of unhealthy choices such as fast food is common among lower social class (Hupkens et al., 2000;Pechey & Monsivais, 2016). This connection between status and health behaviors has been further augmented in today's popular culture. For instance, several famous social media influencers are portraying a wealthy lifestyle coupled with healthy behaviors such as eating well, meditating, and doing physical exercise (Vaterlaus et al., 2015). Consequently, it is possible that consumers today perceive larger portion choices to be unhealthy, and do not associate such unhealthy behaviors with status.
The findings of Dubois et al. (2012) have been greatly influential in the marketing literature; yet our preregistered direct replication casts doubt on the reliability of the relationship between larger food portions and status. We therefore strongly recommend conducting further preregistered conceptual and direct replications to ascertain whether larger portions in fact signal higher status. Obesity and other excessive food consumption related health problems have significant consequences; thus, scientific research findings that inform policymaking in these areas must be robust. The postulation that consumers signal status via choosing larger portions is certainly novel and worthwhile to examine. Nevertheless, we conclude that the evidence for this postulation remains inconclusive until further replications are available in the literature.

Author Contributions
Ignazio supervised Xu Wenting on her master's thesis. Burak verified the analyses and conclusions and performed new ones and completed the manuscript submission draft. Ignazio and Burak jointly finalized the manuscript for submission.
Wenting Xu conducted the replication as part of her "Grand Memoire" (Master's Thesis) at Grenoble Ecole de Management during the academic year 2018-19.

Open Science Practices
This article earned the Preregistration+, Open Data and the Open Materials badge for preregistering the hypothesis and analysis before data collection, and for making the data and materials openly available. It has been verified that the analysis reproduced the results presented in the article. The entire editorial process, including the open reviews, is published in the online supplement.