Repository navigation
Add covariates to Hayes configuration - #232
JohnnyDoorn wants to merge 7 commits into
Conversation
Covariates enter every mediator and outcome equation (the PROCESS default) and can be continuous or categorical. They are excluded from the mediation/total effects tables, the conceptual path plot and Hayes number recognition, are labelled f (into mediators) and g (into the outcome) as in the processR package, get their own plot colour, and are drawn below X in the statistical path plot. Closes jasp-stats#63 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With listwise deletion, .procMeanCenter and .procModProbesSingleModel took complete.cases() over the whole dataset, i.e. over every variable in the variable boxes and every dummy/interaction column of every model. lavaan, however, deletes listwise only on the variables of the fitted model. When any other listed variable had missing values (for example a covariate used only in another model), a model with complete data of its own was centered on the smaller sample, while the excluded rows entered the fit uncentered. The moderator probes used the same restricted sample. Example (debug.csv, Hayes model 1, X = contGamma, W = contcor1, next to a second model with covariate debMiss30, 30 NAs): the X coefficient of the first model was -0.0718 instead of -0.0285 (lm on the centered n = 100 sample). The new helper .procModelCompleteCases restricts the complete cases to the model's own variables, and is used for both centering and probing. The new verification test compares both models with lm() on their own centered samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arator
"__" separates the terms of interaction variables (<X>__<W>), and factor
dummy variables are named <factor><level>. A factor level that contains "__"
(e.g. a covariate level "treat__A") or ends with "_" (e.g. "treat_" for a
factor X in an interaction, giving "<X>treat___<W>") is therefore split at
the wrong place when the interaction and centering variables are built. The
analysis then crashed with an unrelated error ("object ... not found" or
"subscript out of bounds").
.procErrorHandling now checks the factor levels, after the same name-safe
relabelling that .procSafeFactorLevels applies (shared helper
.procSafeLevelLabels), and stops with a message that names the levels to
rename.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The node fill colours come from JASPcolors(asFunction = TRUE), which interpolates the palette, so every colour depends on the number of node types in a plot. The statistical plot has a "Covariate" node type and the conceptual plot (which leaves covariates out) does not, so the same variable got different colours in the two plots of one model: in Hayes model 7, the moderator was grey in the conceptual plot while grey meant "Covariate" in the statistical plot. Adding a covariate also recoloured the mediator, moderator and dependent variable in the statistical plot. Covariates now get a fixed light grey outside the palette, and the palette is interpolated over the other node types only. Plots without covariates are unchanged, so the existing plot snapshots still match. The new test checks that adding a covariate leaves the conceptual plot and the other nodes of the statistical plot unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bayesian analysis has the "Mean-centered moderation" option (on by default), and the moderator probes are computed by the shared .procModProbes on mean-centered data, but .procBayesResultsFitModel passed the raw data to blavaan::bsem(). The conditional effects were therefore evaluated at moderator values shifted by the moderator mean. In the new test (global_warming.csv, Hayes model 1, W = Age), all three conditional effects were off by b_XW * mean(Age) = 0.35 (posterior SD about 0.04). The data are now centered with .procMeanCenter before fitting, as in the classic analysis. The new test compares the posterior means with the OLS conditional effects at the raw percentiles of Age (skip_on_ci, like the other Bayesian tests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The help pages of both analyses listed the Hayes configuration boxes X, M, W and Z but not the new Covariates box. Describe what covariates do (as in PROCESS: added to every mediator and outcome equation, paths f and g), what they are left out of, and that a covariate cannot also have another role. Add NEWS entries for the centering, Bayesian centering and factor-level fixes in this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FBartos
left a comment
There was a problem hiding this comment.
I reviewed this together with Codex. The covariate feature itself works:
- Path coefficients, R², direct and total effects match
lm()for Hayes models 1, 4 (parallel), 6, 7 (centered), 14, and 1 with a factor X, with both continuous and categorical covariates. - Across all implemented Hayes numbers, runs with and without covariates give no new errors, the same recognized model number, and the same effect labels.
I pushed fixes for the problems we found as commits to this branch. Most of them change lines outside this PR's diff, where GitHub cannot attach a suggestion; the one change inside the diff is an inline suggestion (below).
- With listwise deletion (the default), mean-centering and the probes used the wrong sample. Fixed in a77464e.
.procMeanCenterand.procModProbesSingleModeltookcomplete.cases()over the whole dataset, but lavaan deletes listwise only on the model's own variables.- So if any other listed variable has missing values, a model with complete data is centered on a subset, and the remaining rows enter the fit uncentered. This PR makes that common, for example through a covariate used in another model.
- Example on
debug.csv: model 1 (X = contGamma, W = contcor1) next to a model with covariatedebMiss30gave an X coefficient of −0.0718 instead of −0.0285. - The fix adds a helper,
.procModelCompleteCases, and a verification test againstlm()for both models.
- The Bayesian analysis fitted uncentered data, but its probes were centered. Fixed in 217f7cb.
- The probes come from the shared
.procModProbeson centered data, butbsem()got the raw data. With the default "Mean-centered moderation", conditional effects were therefore evaluated at moderator values shifted by the moderator mean. In the new test they were off by 0.35, which is b_XW × mean(Age). - The data are now centered as in the classic analysis. The new test compares the posterior means with the OLS conditional effects and uses
skip_on_ci, like the other Bayesian tests. - Separately, the stored Bayesian expectations already fail locally with the current CRAN blavaan, which reports extra LOO, WAIC and ppp columns. They need regenerating on the reference platform either way.
- The probes come from the shared
- Factor levels containing
__or ending in_crashed the analysis. Fixed in a418834.__separates the terms of interaction variables, so a level liketreat__A(as a covariate) ortreat_(factor X in an interaction) is split in the wrong place..procErrorHandlingnow stops early with a message naming the levels to rename. It checks the levels after the same relabelling as.procSafeFactorLevels.
- Adding covariates recoloured the other nodes in the path plots. Fixed in ea9bd0a.
JASPcolors(asFunction = TRUE)interpolates the palette, so every node colour depends on how many node types the plot has.- The statistical plot has a covariate type and the conceptual plot does not. In model 7 the moderator was therefore grey in the conceptual plot, while grey meant "Covariate" in the statistical plot. Adding a covariate also recoloured the mediator, moderator and outcome.
- Covariates now get a fixed light grey outside the palette. Plots without covariates keep exactly the same colours, and a new test covers this.
- The help pages did not mention the Covariates box. Fixed in ccfbc04, which updates both help pages and adds NEWS entries for 1–3.
🤖 Generated with Claude Code
| covNotInGraph <- covariates[!covariates %in% igraph::V(graph)$name] | ||
| graph <- igraph::add_vertices(graph, length(covNotInGraph), name = covNotInGraph) |
There was a problem hiding this comment.
A covariate that is also X, M, W, Z or Y is not caught in R. The dialog prevents this because all boxes share one variable list, but through R syntax:
- a covariate equal to X stops the whole analysis with an unclear
sort.int(...): 'x' must be atomicerror; - one equal to a mediator gives the misleading "Model must be a directed acyclic graph" message, because the covariate gets a path into itself.
covNotInGraph already expects covariates that are in the graph, so I suggest rejecting them here. The message then appears where the DAG message does (on that model's tables and plots), and the other models still run. I checked this locally with the covariate equal to X, M and Y.
| covNotInGraph <- covariates[!covariates %in% igraph::V(graph)$name] | |
| graph <- igraph::add_vertices(graph, length(covNotInGraph), name = covNotInGraph) | |
| # A covariate must be a separate variable: as X, M or Y it would be regressed on | |
| # itself or duplicate a focal path, and as W or Z it would duplicate the | |
| # moderator's paths. The dialog prevents this, but R syntax does not. | |
| inModel <- covariates[covariates %in% igraph::V(graph)$name] | |
| if (length(inModel) > 0) | |
| stop(gettextf("Covariates must not also be the independent variable, a mediator, a moderator, or the dependent variable. Remove %s from the covariates.", paste(inModel, collapse = ", ")), call. = FALSE) | |
| graph <- igraph::add_vertices(graph, length(covariates), name = covariates) |
A related feature gap, not part of this suggestion: since PROCESS v3.2 (19 Nov 2018), a variable that moderates one equation can be a covariate in the other equations (Hayes, PROCESS v3 history and bug fixes). Supporting that needs two changes:
- the dialog must allow a W or Z variable in the Covariates box;
- this function must add only the missing paths, without marking the moderator as a covariate.
That could be a follow-up.
With "Independent variables" selected under "Residual Covariances" (the default), lavaan and blavaan fit with fixed.x = FALSE, so the likelihood also describes the independent variables, moderators, and covariates. Models with different covariates then describe different data, and their AIC, BIC, and log-likelihoods cannot be compared. In a simulation where a covariate truly predicts M and Y, adding it raised AIC by 484 (lm(): a decrease of 93), and rescaling the covariate changed that difference. Different cases have the same effect. Without independent covariances the likelihood is conditional on the predictors, as in lm(), so models that differ only in their predictors or covariates remain comparable. The model summary tables now add a footnote when the compared models differ in the variables whose distribution they model or in their cases. The Bayesian analysis also stopped with an error from loo_compare() when the models had different numbers of cases: without independent covariances, blavaan removes cases with missing predictors, for example when only one model has a covariate with missing values. The LOO difference is now left out in that case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Follow-up to the review: ea1997e adds a footnote to the model summary tables when the compared models are fitted to different data, and fixes a related crash in the Bayesian analysis.
Tests: a verification test checks that without independent covariances the AIC difference matches 🤖 Generated with Claude Code |
Added