Skip to content

add stream importance to mmn(), and fix three defects large fits exposed - #127

Merged
MaximilianPi merged 1 commit into
mainfrom
cpp-data-loader
Oct 17, 2026
Merged

MaximilianPi merged 1 commit into
mainfrom
cpp-data-loader

Conversation

@MaximilianPi

Copy link
Copy Markdown
Contributor

summary() on an mmn() model repeated print(). It now reports what each data stream contributed: one stream is shuffled across the batch while the others are left intact, and the increase in loss is that stream's contribution. This is the interpretive tool a multi-modal model admits, since per-feature effects are not comparable across modalities of different kinds. The standard deviation across permutations is reported beside the estimate, because a stream that contributes little can come out negative by chance; on simulated data where the images carry the signal and the covariates almost none, the two come out as 0.134 +/- 0.005 and 0.031 +/- 0.014.

plot() works on a fitted MMN. plot_type = "arch" draws the model as the graph it is -- one row per data stream with its submodel, the width each stream contributes to the concatenated representation, and the fusion network on top -- instead of drawing each convolutional submodel's layers in turn and erroring out when there is no convolutional submodel at all.

mmn() gains weights, which dnn() and cnn() already had. The argument alone was not enough: has_weights was hardcoded FALSE in training_properties, so train_model() would have handed the weights to the network as another input rather than to the likelihood. Zeroing half the weights now halves the loss, 0.2525 -> 0.1305.

predict() on an MMN checks newdata against the fitted model. A wrong image size surfaced as "mat1 and mat2 shapes cannot be multiplied" from several frames down, and streams of differing length were silently truncated to the shortest.

The three defects below were invisible in ordinary use and only appeared when the package was pointed at a real analysis.

A fitted model carried its training data. get_loss() takes five arguments and forced none of them; the loss module's methods close over that frame, so an unevaluated promise kept the caller's frame alive, and with it X. saveRDS() then wrote the whole training set into the model: 3.3 MB of images produced a 4.8 MB file, and the cross-validation models of a 15k-image analysis reached 14 GB each, 137 GB for ten models whose weights are 9.4 MB. object.size() does not follow a closure's environment, which is why every component looked small. With the arguments forced, the saved object no longer scales with the data -- X grown sixteenfold leaves it at 0.56 MB.

The validation loop built an autograd graph it never used. $train(FALSE) switches dropout and batch normalization but leaves autograd on, so monitoring a validation split cost as much memory as training on it. Fine-tuning a pretrained backbone with a validation split ran out of memory on a 24 GB card at 23.3 GB in a single process; under with_no_grad() the same fit uses 16 to 19 GB. The losses are unchanged.

predict() on an MMN returned a matrix without row names, where predict() on a cnn() and a dnn() set them. format_input_data() converts each stream to a tensor, which carries no dimnames, so the names have to be taken before that. Without them a lookup such as metadata[rownames(pred), ] returns a zero-length vector and a downstream accuracy evaluates to NA instead of raising an error.

tests/testthat/test-regressions.R pins all three: the saved model must not grow with the data, a validation split must still produce a usable loss curve, and predict() must carry the names through and must not invent any when the input has none.

The MMN section of the vignette was not runnable. It passed a bare response name without dataList, gave predict() an unnamed list, and claimed cito applies no inverse link. Every chunk was eval=FALSE, which is why none of it had been caught. It now carries a worked example that runs at build time and ends in summary() and plot().

summary() on an mmn() model repeated print(). It now reports what each
data stream contributed: one stream is shuffled across the batch while
the others are left intact, and the increase in loss is that stream's
contribution. This is the interpretive tool a multi-modal model admits,
since per-feature effects are not comparable across modalities of
different kinds. The standard deviation across permutations is reported
beside the estimate, because a stream that contributes little can come
out negative by chance; on simulated data where the images carry the
signal and the covariates almost none, the two come out as 0.134 +/-
0.005 and 0.031 +/- 0.014.

plot() works on a fitted MMN. plot_type = "arch" draws the model as the
graph it is -- one row per data stream with its submodel, the width each
stream contributes to the concatenated representation, and the fusion
network on top -- instead of drawing each convolutional submodel's layers
in turn and erroring out when there is no convolutional submodel at all.

mmn() gains weights, which dnn() and cnn() already had. The argument
alone was not enough: has_weights was hardcoded FALSE in
training_properties, so train_model() would have handed the weights to
the network as another input rather than to the likelihood. Zeroing half
the weights now halves the loss, 0.2525 -> 0.1305.

predict() on an MMN checks newdata against the fitted model. A wrong
image size surfaced as "mat1 and mat2 shapes cannot be multiplied" from
several frames down, and streams of differing length were silently
truncated to the shortest.

The three defects below were invisible in ordinary use and only appeared
when the package was pointed at a real analysis.

A fitted model carried its training data. get_loss() takes five
arguments and forced none of them; the loss module's methods close over
that frame, so an unevaluated promise kept the caller's frame alive, and
with it X. saveRDS() then wrote the whole training set into the model:
3.3 MB of images produced a 4.8 MB file, and the cross-validation models
of a 15k-image analysis reached 14 GB each, 137 GB for ten models whose
weights are 9.4 MB. object.size() does not follow a closure's
environment, which is why every component looked small. With the
arguments forced, the saved object no longer scales with the data -- X
grown sixteenfold leaves it at 0.56 MB.

The validation loop built an autograd graph it never used. $train(FALSE)
switches dropout and batch normalization but leaves autograd on, so
monitoring a validation split cost as much memory as training on it.
Fine-tuning a pretrained backbone with a validation split ran out of
memory on a 24 GB card at 23.3 GB in a single process; under
with_no_grad() the same fit uses 16 to 19 GB. The losses are unchanged.

predict() on an MMN returned a matrix without row names, where predict()
on a cnn() and a dnn() set them. format_input_data() converts each
stream to a tensor, which carries no dimnames, so the names have to be
taken before that. Without them a lookup such as metadata[rownames(pred),
] returns a zero-length vector and a downstream accuracy evaluates to NA
instead of raising an error.

tests/testthat/test-regressions.R pins all three: the saved model must
not grow with the data, a validation split must still produce a usable
loss curve, and predict() must carry the names through and must not
invent any when the input has none.

The MMN section of the vignette was not runnable. It passed a bare
response name without dataList, gave predict() an unnamed list, and
claimed cito applies no inverse link. Every chunk was eval=FALSE, which
is why none of it had been caught. It now carries a worked example that
runs at build time and ends in summary() and plot().
@MaximilianPi
MaximilianPi merged commit aec71d5 into main Oct 17, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant