Skip to content

Add covariates to Hayes configuration - #232

Open
JohnnyDoorn wants to merge 7 commits into
jasp-stats:masterfrom
JohnnyDoorn:feature/hayes-covariates
Open

JohnnyDoorn wants to merge 7 commits into
jasp-stats:masterfrom
JohnnyDoorn:feature/hayes-covariates

Conversation

@JohnnyDoorn

@JohnnyDoorn JohnnyDoorn commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Added

  • Covariates box for the "Hayes configuration" input type. As in PROCESS, covariates are added to every mediator and outcome equation, so the path coefficients, effects, and R² are adjusted for them. Covariates can be continuous or categorical, and they are left out of the mediation and total effects tables and the conceptual path plot (Issue #63, Issue #4388, Issue #3481).
  • Covariate paths are labelled f (into mediators) and g (into the outcome), as in the processR package, and are drawn below X in the statistical path plot.

JohnnyDoorn and others added 6 commits November 6, 2026 13:11
Covariates enter every mediator and outcome equation (the PROCESS
default) and can be continuous or categorical. They are excluded from
the mediation/total effects tables, the conceptual path plot and Hayes
number recognition, are labelled f (into mediators) and g (into the
outcome) as in the processR package, get their own plot colour, and are
drawn below X in the statistical path plot.

Closes jasp-stats#63

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With listwise deletion, .procMeanCenter and .procModProbesSingleModel took
complete.cases() over the whole dataset, i.e. over every variable in the
variable boxes and every dummy/interaction column of every model. lavaan,
however, deletes listwise only on the variables of the fitted model. When any
other listed variable had missing values (for example a covariate used only
in another model), a model with complete data of its own was centered on the
smaller sample, while the excluded rows entered the fit uncentered. The
moderator probes used the same restricted sample.

Example (debug.csv, Hayes model 1, X = contGamma, W = contcor1, next to a
second model with covariate debMiss30, 30 NAs): the X coefficient of the first
model was -0.0718 instead of -0.0285 (lm on the centered n = 100 sample).

The new helper .procModelCompleteCases restricts the complete cases to the
model's own variables, and is used for both centering and probing. The new
verification test compares both models with lm() on their own centered samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arator

"__" separates the terms of interaction variables (<X>__<W>), and factor
dummy variables are named <factor><level>. A factor level that contains "__"
(e.g. a covariate level "treat__A") or ends with "_" (e.g. "treat_" for a
factor X in an interaction, giving "<X>treat___<W>") is therefore split at
the wrong place when the interaction and centering variables are built. The
analysis then crashed with an unrelated error ("object ... not found" or
"subscript out of bounds").

.procErrorHandling now checks the factor levels, after the same name-safe
relabelling that .procSafeFactorLevels applies (shared helper
.procSafeLevelLabels), and stops with a message that names the levels to
rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The node fill colours come from JASPcolors(asFunction = TRUE), which
interpolates the palette, so every colour depends on the number of node types
in a plot. The statistical plot has a "Covariate" node type and the conceptual
plot (which leaves covariates out) does not, so the same variable got
different colours in the two plots of one model: in Hayes model 7, the
moderator was grey in the conceptual plot while grey meant "Covariate" in the
statistical plot. Adding a covariate also recoloured the mediator, moderator
and dependent variable in the statistical plot.

Covariates now get a fixed light grey outside the palette, and the palette is
interpolated over the other node types only. Plots without covariates are
unchanged, so the existing plot snapshots still match. The new test checks
that adding a covariate leaves the conceptual plot and the other nodes of the
statistical plot unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bayesian analysis has the "Mean-centered moderation" option (on by
default), and the moderator probes are computed by the shared
.procModProbes on mean-centered data, but .procBayesResultsFitModel passed
the raw data to blavaan::bsem(). The conditional effects were therefore
evaluated at moderator values shifted by the moderator mean. In the new test
(global_warming.csv, Hayes model 1, W = Age), all three conditional effects
were off by b_XW * mean(Age) = 0.35 (posterior SD about 0.04).

The data are now centered with .procMeanCenter before fitting, as in the
classic analysis. The new test compares the posterior means with the OLS
conditional effects at the raw percentiles of Age (skip_on_ci, like the other
Bayesian tests).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The help pages of both analyses listed the Hayes configuration boxes X, M, W
and Z but not the new Covariates box. Describe what covariates do (as in
PROCESS: added to every mediator and outcome equation, paths f and g), what
they are left out of, and that a covariate cannot also have another role.
Add NEWS entries for the centering, Bayesian centering and factor-level fixes
in this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@FBartos FBartos left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this together with Codex. The covariate feature itself works:

  • Path coefficients, R², direct and total effects match lm() for Hayes models 1, 4 (parallel), 6, 7 (centered), 14, and 1 with a factor X, with both continuous and categorical covariates.
  • Across all implemented Hayes numbers, runs with and without covariates give no new errors, the same recognized model number, and the same effect labels.

I pushed fixes for the problems we found as commits to this branch. Most of them change lines outside this PR's diff, where GitHub cannot attach a suggestion; the one change inside the diff is an inline suggestion (below).

  1. With listwise deletion (the default), mean-centering and the probes used the wrong sample. Fixed in a77464e.
    • .procMeanCenter and .procModProbesSingleModel took complete.cases() over the whole dataset, but lavaan deletes listwise only on the model's own variables.
    • So if any other listed variable has missing values, a model with complete data is centered on a subset, and the remaining rows enter the fit uncentered. This PR makes that common, for example through a covariate used in another model.
    • Example on debug.csv: model 1 (X = contGamma, W = contcor1) next to a model with covariate debMiss30 gave an X coefficient of −0.0718 instead of −0.0285.
    • The fix adds a helper, .procModelCompleteCases, and a verification test against lm() for both models.
  2. The Bayesian analysis fitted uncentered data, but its probes were centered. Fixed in 217f7cb.
    • The probes come from the shared .procModProbes on centered data, but bsem() got the raw data. With the default "Mean-centered moderation", conditional effects were therefore evaluated at moderator values shifted by the moderator mean. In the new test they were off by 0.35, which is b_XW × mean(Age).
    • The data are now centered as in the classic analysis. The new test compares the posterior means with the OLS conditional effects and uses skip_on_ci, like the other Bayesian tests.
    • Separately, the stored Bayesian expectations already fail locally with the current CRAN blavaan, which reports extra LOO, WAIC and ppp columns. They need regenerating on the reference platform either way.
  3. Factor levels containing __ or ending in _ crashed the analysis. Fixed in a418834.
    • __ separates the terms of interaction variables, so a level like treat__A (as a covariate) or treat_ (factor X in an interaction) is split in the wrong place.
    • .procErrorHandling now stops early with a message naming the levels to rename. It checks the levels after the same relabelling as .procSafeFactorLevels.
  4. Adding covariates recoloured the other nodes in the path plots. Fixed in ea9bd0a.
    • JASPcolors(asFunction = TRUE) interpolates the palette, so every node colour depends on how many node types the plot has.
    • The statistical plot has a covariate type and the conceptual plot does not. In model 7 the moderator was therefore grey in the conceptual plot, while grey meant "Covariate" in the statistical plot. Adding a covariate also recoloured the mediator, moderator and outcome.
    • Covariates now get a fixed light grey outside the palette. Plots without covariates keep exactly the same colours, and a new test covers this.
  5. The help pages did not mention the Covariates box. Fixed in ccfbc04, which updates both help pages and adds NEWS entries for 1–3.

🤖 Generated with Claude Code

Comment thread R/classicProcess.R
Comment on lines +198 to +199
covNotInGraph <- covariates[!covariates %in% igraph::V(graph)$name]
graph <- igraph::add_vertices(graph, length(covNotInGraph), name = covNotInGraph)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A covariate that is also X, M, W, Z or Y is not caught in R. The dialog prevents this because all boxes share one variable list, but through R syntax:

  • a covariate equal to X stops the whole analysis with an unclear sort.int(...): 'x' must be atomic error;
  • one equal to a mediator gives the misleading "Model must be a directed acyclic graph" message, because the covariate gets a path into itself.

covNotInGraph already expects covariates that are in the graph, so I suggest rejecting them here. The message then appears where the DAG message does (on that model's tables and plots), and the other models still run. I checked this locally with the covariate equal to X, M and Y.

Suggested change
covNotInGraph <- covariates[!covariates %in% igraph::V(graph)$name]
graph <- igraph::add_vertices(graph, length(covNotInGraph), name = covNotInGraph)
# A covariate must be a separate variable: as X, M or Y it would be regressed on
# itself or duplicate a focal path, and as W or Z it would duplicate the
# moderator's paths. The dialog prevents this, but R syntax does not.
inModel <- covariates[covariates %in% igraph::V(graph)$name]
if (length(inModel) > 0)
stop(gettextf("Covariates must not also be the independent variable, a mediator, a moderator, or the dependent variable. Remove %s from the covariates.", paste(inModel, collapse = ", ")), call. = FALSE)
graph <- igraph::add_vertices(graph, length(covariates), name = covariates)

A related feature gap, not part of this suggestion: since PROCESS v3.2 (19 Nov 2018), a variable that moderates one equation can be a covariate in the other equations (Hayes, PROCESS v3 history and bug fixes). Supporting that needs two changes:

  • the dialog must allow a W or Z variable in the Covariates box;
  • this function must add only the missing paths, without marking the moderator as a covariate.

That could be a follow-up.

With "Independent variables" selected under "Residual Covariances" (the
default), lavaan and blavaan fit with fixed.x = FALSE, so the likelihood
also describes the independent variables, moderators, and covariates.
Models with different covariates then describe different data, and their
AIC, BIC, and log-likelihoods cannot be compared. In a simulation where a
covariate truly predicts M and Y, adding it raised AIC by 484 (lm(): a
decrease of 93), and rescaling the covariate changed that difference.
Different cases have the same effect. Without independent covariances the
likelihood is conditional on the predictors, as in lm(), so models that
differ only in their predictors or covariates remain comparable.

The model summary tables now add a footnote when the compared models
differ in the variables whose distribution they model or in their cases.

The Bayesian analysis also stopped with an error from loo_compare() when
the models had different numbers of cases: without independent
covariances, blavaan removes cases with missing predictors, for example
when only one model has a covariate with missing values. The LOO
difference is now left out in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FBartos

FBartos commented Nov 8, 2026

Copy link
Copy Markdown

Follow-up to the review: ea1997e adds a footnote to the model summary tables when the compared models are fitted to different data, and fixes a related crash in the Bayesian analysis.

  1. Information criteria of models with different covariates are not comparable under the default settings.
    • With "Independent variables" selected under "Residual Covariances" (the default), the models are fitted with fixed.x = FALSE. The likelihood then also covers X, the moderators, and the covariates, so models with different covariates describe different data.
    • In a simulation where a covariate truly predicts M and Y, adding it raised AIC by 484 under the default, while lm() and lavaan with fixed.x = TRUE give a decrease of 93. Rescaling the covariate by 1000 raised the difference to 3247.
    • Without independent covariances, the likelihood is conditional on the predictors, so models that differ only in their covariates stay comparable and get no footnote.
    • The footnote also appears when the models use different cases.
  2. The Bayesian analysis stopped when the models had different numbers of cases.
    • Without independent covariances, blavaan drops cases with missing predictors. A covariate with missing values in only one model then made loo_compare() stop with "All models must have the same number of observations", and the whole analysis failed.
    • The LOO difference is now left out in that case.

Tests: a verification test checks that without independent covariances the AIC difference matches lm() with no footnote, and that the footnote appears with independent covariances and with different cases. A Bayesian test (skip_on_ci, like the others) covers the crash. The help pages and NEWS are updated.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants