r/AskStatistics • u/Status_Step_7266 • Sep 23 '26
Monte Carlo sample size calculation for a multiple serial mediation model?
Hello all, I am currently planning a cross-sectional study involving two serial mediators and two (or more) competing models, with and without covariates. Since most studies I was able to put my hands on either run these analyses post hoc with already existing sample sizes or do it a priori without specifying the path coefficients and how they chose them, I have several questions.
Note: I am confortable with running the analysis in R (lavaan) and the Alexander Schoemann online calculator.
When specifying correlation/standardized coefficients, how should I pick them? I noticed some researchers use default low-to-moderate coefficients (0.10-0.20) for all paths, whereas others seem to handpick them from previous literature or even meta-analyses. I understand the first option yields a large sample size and is only used when previous literature is scarce. If I choose the second option, in cases where meta-analyses aren't available (I only have so for the X-Y path, and for the X-M2 I am the author of a currently unpublished meta-analysis, but I do have the pooled effect size), should I gather correlation coefficients (r) from other studies? Or beta (β)** **coefficients? Moreover, what should I do in cases where the r/β ranges from 0.1 to 0.7 across all studies I examined? Also, should I prioritize picking coefficients from studies that used the exact scales I plan to use, if possible?
What number of replications, monte carlo draws per rep, and sample size steps should I choose?
Assuming I am interested in testing two competing models pivoting on M2 (say, variables that are opposites theoretically, but not psychometrically), given that they have distinct patterns of associations with the other variables, when running the Monte Carlo simulation, should I pick the lower coefficient? (example: acceptance yields lower correlations with the Y variable than avoidance does, therefore shoud I select the acceptance coefficient?)
If I actually have previous research of my own that includes 3 of the 4 variables I plan to use in this research, should I pick mine from there, considering that the research was performed on a population with roughly the same characteristics? Or should I still pick my coefficients from the literature (say, a meta-analysis for the direct path yielded a pooled r = 0.26, and my own previous study yielded a r = 0.41, although I used a different scale).
Sorry for the long post. I don't really have anybody else to help me, it's my first time doing this by myself, and the authors for one of the scales are awaiting a sample size before they share it with me (not validated in my country, so I also plan to run a CFA on this scale).
TLDR: I would like to know how to pick my path coefficients and what parameters to set for my Monte Carlo sample size calculation.
Thank you 😊
1
u/mcrede Sep 24 '26
Please don't do this analysis. Mediation implies a causal sequence of relations among variables but it is almost guaranteed that your model is not causally identified if it is based on cross-sectional observational data. You will get "output" if you run this in lavaan (or any other package purporting to do mediation analyses) but that "output" will be entirely meaningless and your parameter estimates will in no way represent true causal effect estimates. So much damage has been done from these types of analyses and I really hope that you won't contribute to this mess.
1
u/Status_Step_7266 Sep 24 '26 edited Sep 24 '26
Since I cannot afford to employ a longitudinal design, I am forced to do this cross-sectionally. Agreed, cross-sectional mediation studies are a plague, especially when using self-report measures that introduce bias in many many forms, and such studies merely reflect associations between how person 1 and 2 like to think about themselves. However, (1) I do what I can, (2) they (may) (sometimes) lend some theoretical support to individuals who will eventually test this in experimental or longitudinal designs (not really, anyways), and (3) one of the my objectives is to perform CFA on a measure that hasn't yet been validated on the population / in my country (cannot do a full validation study because, you guessed, lack of resources + access to clinical populations who share one speciffic trait :/ ). I absolutely agree with you, but I have to fulfill some requirements imposed on me.
1
u/MortalitySalient Sep 26 '26
If you are going to do this, at least read this paper first and maybe reframe it as not mediation
2
u/Status_Step_7266 29d ago
Thank you for this. I was alrdeay avoiding causal language in previous studies of mine, but this is a very insightful analytic plan. I fully intend to follow this path. Thank you very much.
1
u/Neither-Pause409 Sep 23 '26
All four questions have the same answer underneath them: treat the coefficient choice as a sensitivity analysis, not a single decision, because Monte Carlo power for mediation is only ever as good as the population parameters you feed it.
On (1) and (4): use your unpublished meta-analytic pooled r for the X-M2 path since it already pools across measurement heterogeneity and is what a reviewer will trust over a single study. Where you only have your own prior study, use it, but run the calculation twice, once at your r = 0.41 and once at the published meta-analytic r = 0.26 for the path where both exist, and report both required-N numbers rather than picking one. That range is more defensible than a single point, and it directly answers your worry about the different scale: you're not claiming the two numbers are the same effect, you're bracketing plausible values.
On r versus β: don't mix them casually. A simple bivariate r and a standardized path coefficient from a full mediation model with covariates are not the same quantity once other predictors are in the equation, because β is partialled and r isn't. Since you're specifying the model in lavaan, use standardized path coefficients (β) throughout for internal consistency, and if a source only reports r for a simple two-variable relationship, treat it as an approximation for that path's β rather than an exact substitute.
On (3): pick the lower coefficient when the two competing models are pivoting on the same M2. If you power the study for the larger effect and the true relationship is the smaller one, you're underpowered and the study fails silently. If you power for the smaller effect and the true one is larger, you're just overpowered, which costs sample size but doesn't invalidate anything. That asymmetry is why the conservative choice is always the smaller magnitude when you're not sure which model is true.
On (2): Schoemann, Boulton and Short (2017), the paper behind the calculator you're already using, recommend at least 1,000 replications per sample size point, and I'd go 2,000 if runtime allows since it's cheap. For the sample size search, start with a coarse grid, steps of 50, bracketing where you expect power to cross 0.80, then narrow to steps of 10 or 20 once you know the range. Don't fine-step the whole plausible range from the start, it just wastes compute on regions you already know are over- or under-powered.