Contemporary Understanding of Mediation Testing

This is a commentary about contemporary understanding of mediation testing. Specifically, this commentary highlights that outdated concepts of mediation testing are still highly prevalent in the mindsets of researchers and that many researchers use software based on contemporary mediation testing wrongly, misinterpret results or describe mediation in terms of outdated concepts while inappropriately referring to literature about contemporary concepts.

A common question in psychological research is how or through which mechanism(s) an effect occurs.The most basic form of this question can be represented in a simple mediation model, which consists of an antecedent (or independent) variable (X) that is linked to a consequent (or dependent) variable (Y) through an intermediary (or mediator) variable (M; Figure 1).
In 1986, Baron and Kenny published a seminal article, in which they present a mediation analysis method, which is often referred to as the causal steps approach. 1ccording to this approach, "a variable functions as a mediator when it meets the following conditions: (a) variations in levels of the independent variable significantly account for variations in the presumed mediator (i.e., Path a), (b) variations in the mediator significantly account for variations in the dependent variable (i.e., Path b), and (c) when Paths a and b are controlled, a previously significant relation between the independent and dependent variables is no longer significant, with the strongest demonstration of mediation occurring when Path c is zero" (Baron & Kenny, 1986, p. 1176).Thus, a mediation effect can only occur when there is a significant correlation between the independent and the dependent variable and when this relationship is no longer significant when controlling for the mediator.If the mediator does not entirely account for the association between the independent and the dependent variable, it partially mediates that effect.Finally, Baron and Kenny (1986) recommend using the Sobel test (Sobel, 1982) as a formal significance test for the existence of a mediation effect.
The popularity of the Baron and Kenny approach can be seen in that-as of this writing-their article has been cited more than 36,000 times according to Web of Science and more than 80,000 times according to Google Scholar.In 2018 alone, the article has been cited more than 2,000 times according to Web of Science and more than 3,000 times according to Google Scholar.Although the causal steps approach has intuitive appeal, it is now widely accepted that the approach of mediation testing by Baron and Kenny (1986) is obsolete (e.g., Hayes, 2009Hayes, , 2013Hayes, , 2017;;Hayes & Rockwood, 2017;MacKinnon & Fairchild, 2009;Zhao, Lynch, & Chen, 2010).Specifically, neither are significant relationships between the independent, mediator, and dependent variable a prerequisite for a mediation effect to be possible nor is a significant reduction of the relationship between the independent and the dependent variable by the mediator necessary for inferring mediation.This Adrian Meule, PhD.Department of Psychology, University of Salzburg, Hellbrunner Straße 34, 5020 Salzburg, Austria.E-mail: adrian.meule@sbg.ac.at.
understanding also renders the concept of "partial mediation" moot (e.g., Hayes, 2013;Hayes & Rockwood, 2017;Zhao, Lynch, & Chen, 2010).Finally, inferential tests about mediation based on Monte Carlo or bootstrap confidence intervals rather than the Sobel test are now recommended (Hayes & Scharkow, 2013).In current understanding of mediation testing, the relationship between the independent variable and the dependent variable is called the total effect (i.e., the c path).The relationship between the independent variable and the dependent variable while controlling for the mediator is called the direct effect (usually denoted as c').The indirect (i.e., mediation) effect is the product of the a and b path (i.e., the relationship [a] between the independent variable and the mediator and [b] between the mediator and the dependent variable when controlling for the independent variable).The total effect is the sum of the direct and indirect effect (total effect = direct effect + indirect effect or c = c' + a × b).Thus, the statistical significance of both the total and the direct effect is irrelevant for the existence of an indirect effect.The indirect effect can be significant (1) in the absence of a direct effect (indirect-only mediation), (2) in addition to a significant direct effect of the same sign (complementary mediation), and even (3) in addition to a significant direct effect of opposite sign (competitive mediation; Zhao et al., 2010).In each case, the significance of the total effect is irrelevant for the presence of a direct or indirect effect.
Note the crucial differences between the causal steps and the contemporary approach.Researchers who adhere to the Baron and Kenny thinking stop when at least one of the three paths (a, b, and c) is not significant.That is, they do not test for mediation because the alleged assumptions to do that are not met (also note that many researchers falsely assume that the b path is the correlation between M and Y although it is the relation between M and Y when controlling for X).In addition, situations like a complementary and competitive mediation (i.e., a significant indirect effect in addition to a significant direct effect) have no place in the Baron and Kenny framework because a significant relationship between X and Y when controlling for M (i.e., the direct effect) would indicate that there is no full mediation effect.Finally, in contemporary thinking about mediation analysis, the indirect effect is either significant or not significant, regardless of the significance of the total effect.As there is, therefore, no need for an "effect to be mediated", the concept of "partial mediation" is incompatible with the contemporary approach.
Most researchers find it difficult to imagine how an indirect effect in the absence of a total effect can be possible.As an example, let us consider the relationship between an impulsive personality and body weight.Several mediation studies showed that higher impulsivity is associated with a higher body weight through eating behavior-related variables (Meule, 2017).In other words, people that are more impulsive tend to overeat, which in turn relates to a higher body weight.However, higher impulsivity also relates to higher substance use (e.g., Stanford, et al., 2009).In turn, active substance users usually have a lower body weight than non-substance users (e.g., Crossin, Lawrence, Andrews, & Duncan, in press).Thus, there may be several indirect effects of impulsivity on body weight with opposite signs (e.g., a positive indirect effect through eating behavior and a negative indirect effect through substance use), which cancel each other out and, thus, there is no significant total effect of impulsivity on body weight (Meule, 2017).
Among other tools, mediation analysis can be easily conducted using a macro called PROCESS (www.processmacro.org;Hayes, 2013Hayes, , 2017)).In PROCESS, test-ing a simple mediation model is based on two linear regression analyses.First, the mediating variable is predicted by the independent variable (a).Second, the dependent variable is predicted by both the mediating variable (b) and the independent variable (c').The test for inferring whether there is a significant indirect effect (ab) is based on bootstrap confidence intervals.If the confidence interval does not include zero, then it is inferred that the indirect effect is significant, that is, that there is a mediation effect.
Demonstrating the popularity of PROCESS, a white paper about it (Hayes, 2012) has been cited more than 3,000 times and Hayes' book (Hayes, 2013(Hayes, , 2017))which is the official reference for PROCESS-has been cited more than 18,000 times according to Google Scholar.Unfortunately, while many researchers use PROCESS, it seems like many of them are reluctant to abandon the causal steps logic.Table 1 lists 20 of the more than 5,000 articles that cited Hayes' book in 2018 (according to Google Scholar).As can be seen from this compilation, researchers seem be stuck in the Baron and Kenny thinking although they are using PROCESS or refer to its manual.This is actually mentioned by Hayes himself on his FAQ page: (1) Highlighting the fact that the relationship between X and Y was significant, but was no longer significant when controlling for M, although both of these statements are irrelevant for the existence of an indirect effect when using PROCESS.
(2) Stating that a variable partially mediated an effect when the direct effect was significant, although the concept of partial mediation is incompatible to what PROCESS does and what Hayes writes in its documentation.(3) Reporting results of a Sobel test as an inferential test for mediation, although this is either redundant (because it will suggest the same conclusion as the bootstrap confidence intervals) or the Sobel test is not significant because it has lower power than bootstrap sampling, which is why it is explicitly recommended by Hayes (2013Hayes ( , 2017) ) and others (Zhao et al., 2010) to prefer the latter.
In conclusion, researchers still follow the causal steps logic, even when they are using programs that are not based on this logic or when they are referring to literature about contemporary mediation testing.The consequences of these fallacies include minor subtleties (e.g., writing about a direct effect when researchers actually want to refer to the total effect or stating that "an effect was mediated by" a variable) but also redundancy (e.g., reporting results from a Sobel test in addition to bootstrap intervals), possibly unstable results (e.g., when using 1,000 bootstrap samples instead of 5,000 as recommended by Hayes, 2013), prematurely concluding that there was no mediation effect (e.g., when not testing for mediation because one of the paths was not significant), and misinterpreting results (e.g., describing results as full vs. partial mediation).Zhao and colleagues ( 2010) and Hayes & Rockwood (2017) provide brief and easy-to-read articles in which they outline the drawbacks of the causal steps approach and guide through the rationale and practical implementation of contemporary mediation testing.With such guidance, researchers will hopefully not only use statistical software based on state-of-the-art thinking about mediation, but also adjust their own mindsets accordingly when writing about mediation results.

Open Science Practices
The nature of this commentary meant that there are no data or research materials to be shared.
Table 1 Some recent examples of articles referring to mediation analyses with PROCESS (Hayes, 2013) Reference Indirect effect(s) of… Issues Alt et al. (in press) …stereotype valence on favorable evaluations through perceived offensiveness -Followed the causal steps logic by highlighting that "the direct effect […] was no longer significant" (pp.6, 7, 10) Bender et al. (2018) …closeness, dependability, and anxiety on resilience through self-care and selfefficacy -Followed the causal steps logic by stating that significant direct effects suggest "partial mediation" (p.19) Bhalla et al. (2018) …exposure to potentially morally injurious events on sexual anxiety through emotional numbing -Followed the causal steps logic by stating that "this was a partial mediation, as the direct effect […] remained significant" (p.424) Buckner et al. (2018) …cannabis use on condom use through protective behavior strategies -Followed the causal steps logic by highlighting that "there was no longer a significant direct effect" (p.52) Feinberg et al. (2018) …experiential avoidance on accommodation through beliefs about anxiety -Followed the causal steps logic by highlighting that "when negative beliefs about child anxiety were incorporated into the model this direct effect was no longer significant" (p.652) Friese et al. (in press) …ego depletion on performance through monitoring processes and selfefficacy -Followed the causal steps logic by describing mediation in terms of the Baron and Kenny approach (p.9) -Confused total effect with direct effect Karriker-Jaffe et al. (2018) …treatment type on alcohol consumption through treatment duration and Alcoholics Anonymous attendance -Followed the causal steps logic by stating that "effects of treatment type on alcohol consumption [are] partially mediated by Alcoholics Anonymous attendance" (p.596) Li et al. (2018) …injunctive and descriptive norms on willingness to quit smoking through smoking risk perception -Used 1,000 bootstrap samples although Hayes (2013) recommends using a minimum of 5,000 bootstrap samples -Followed the causal steps logic by stating that "smoking risk perception partially mediated the association between descriptive norms and willingness to quit smoking" (p.277) Lum et al. (2018) …medication adherence on glycated hemoglobin concentration through diabetes-related distress and perception of hyperglycaemia -Tested simple mediation models with the causal steps approach and the Sobel test, but a subsequent multiple mediation model with PROCESS/bootstrapping (p.810) -Followed the causal steps logic by referring to full/complete vs. partial mediation Mackenbach et al. (in press) …access to fast food outlets on overweight through perceived availability and usage of fast food outlets -Tested mediation effects with the Sobel test -Followed the causal steps logic by not testing for mediation for certain outcomes because "none of the a-or b-paths were statistically significant" (p.17) and highlighting how much of the total effect can be explained by the indirect effect ("proportion mediated", p. 9) Maroney et al. (in press) …depression, loneliness, and social anxiety on problem video game use through escapism and social interaction motives for gaming -Followed the causal steps logic by stating that "these effects being partially mediated by escapism and social interaction motives for gaming" (p. 1) Peña et al. (2018) …cognitive rehabilitation on functional improvement through changes in processing speed and verbal memory -Followed the causal steps logic by stating that the mediating variable "partially mediated" the relationship between the independent and dependent variable (p.23) Polanco-Roman et al. (2018) …emotion reactivity on suicidal ideation through depressive symptoms -Followed the causal steps logic by highlighting that "the direct effect of emotion reactivity on suicidal ideation, though reduced, remained statistically significant after comfort expressing love, happiness, anger, and sadness were entered into the model […], but was no longer significant after depressive symptoms were entered in the model" (p.128) Poless et al. (2018) …narcissism on ethical behavior by guilt repair through guilt -Tested mediation effects with the Sobel test -Followed the causal steps logic by highlighting the significant total and non-significant direct effects (Figures 1-4 (2018) …diastolic blood pressure on cognitive functioning through brain plague and tangle burden -Used 1,000 bootstrap samples although Hayes (2013) recommends using a minimum of 5,000 bootstrap samples -Followed the causal steps logic ("If the 95% confidence interval for a b does not include 0 and the association between the predictor and outcome variables including the mediator variable (i.e., the c'-path) is no longer significant, then a significant (p < 0.05) mediation has occurred.",p. 16) Varni et al. (2018) …gastrointestinal symptoms on healthrelated quality of life through perceived medication adherence barriers -Tested mediation effects with the Sobel test in addition to bootstrap sampling -Confused total effect with direct effect and followed the causal steps logic ("mediators are intervening variables that are hypothesized to account partially or in full for the relationship between a predictor variable and an outcome variable […] the predictor variable is hypothesized to have a direct effect on the outcome variable", p. 198) Widmer et al. (2018) conflict on health through individual stress -Used 1,000 bootstrap samples although Hayes (2013) recommends using a minimum of 5,000 bootstrap samples -Additionally tested mediation effects with the Sobel test -Confused total effect with direct effect and followed the causal steps logic ("the inclusion of stress reduces the direct association between conflict structures and the psychological dimensions of health", p. 1590) Yildirim & Dark (2018) …media multitasking on mind wandering through trait mindfulness -Followed the causal steps logic ("Given that all paths were statistically significant, that the inclusion of trait mindfulness as a mediator led to reductions in the magnitude of the effect of media multitasking frequency on mind wandering tendency (path c' < path c), and that the indirect effect of media multitasking frequency on mind wandering tendency through trait mindfulness was significant, it can be concluded that trait mindfulness partially mediated the relationship between media multitasking frequency and mind wandering tendency.",p. 3)

Figure 1 .
Figure 1.Conceptual diagram of a simple mediation model.X denotes the independent or antecedent variable.M denotes the mediator or intermediary variable.Y denotes the dependent or consequent variable.Path a represents the relationship between X and M. Path b represents the relationship between M and Y when controlling for X.Path c represents the relationship between X and Y (total effect).Path c' represents the relationship between X and Y when controlling for M (direct effect).The indirect effect is the product of a × b.The total effect is the sum of the direct and the indirect effect (c = c' + a × b).
Good academic practice is to cite something only if you have actually read it and are familiar with its content.I don't recommend using PROCESS without familiarity with what it does, as described in the book.It may not be doing what you think it is doing.I have seen many instances of researchers reporting results from the output of PROCESS that are inconsistent with what PROCESS actually is doing.These mistakes are easily avoided by reading the documentation.(http://www.processmacro.org/faq)Common mistakes include the following: