Repository navigation
feat: partial template formatting and stricter wildcard value validation - #532
Conversation
There was a problem hiding this comment.
Pull request overview
Adds support for incremental / partial Snakemake template formatting by preserving missing wildcards (allow_missing=True), and introduces stricter validation for wildcard values to prevent underscore/slash state ambiguities during multi-step formatting.
Changes:
- Extend
SnakemakeFormatterwithallow_missingbehavior and internal underscore-state refactor. - Add
_validate()to reject wildcard values that could cause inconsistent single-step vs multi-step formatting. - Expand and adjust tests, including new validation/allow-missing suites and a BIDS match→format roundtrip test.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
src/snakebids/utils/snakemake_templates.py |
Implements allow_missing, underscore-state refactor, and stricter value validation. |
tests/test_snakemake_templates/test_snakemake_formatter.py |
Adds/updates property-based tests for missing-wildcard preservation and validation rules. |
tests/test_paths/test_bids.py |
Simplifies template construction, adds match→format roundtrip test, and changes active spec-version test suites. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #532 +/- ##
==========================================
+ Coverage 95.18% 95.26% +0.07%
==========================================
Files 56 56
Lines 2451 2492 +41
==========================================
+ Hits 2333 2374 +41
Misses 118 118 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
0269bf3 to
6284bf8
Compare
Implement the `allow_missing` flag from the snakemake expand function, allowing wildcards to be omitted from the function call. To ensure a 1-step format with `allow_missing=False` would be equivalent to a multistep format with `allow_missing=True`, validation was added to prevent specially crafted values from stymieing the underscore squelching behaviour
6284bf8 to
a564279
Compare
- Replace snakemake_templates.py with PR #532's version: - Uses _current_field (not _field_constraints dict) - Uses _underscore: str (not squelch_underscore: bool) - Adds _validate() for entity value validation - Wraps get_value() to catch KeyError for allow_missing - Replace test_snakemake_formatter.py with PR #532's version - Update test_bids.py with latest main version - expanding.py and datasets.py expand reimplementation unchanged Co-authored-by: pvandyken <87136354+pvandyken@users.noreply.github.com>
Changes
SnakemakeFormatter: Addallow_missingmodeSnakemakeFormatternow accepts anallow_missing: bool = Falseconstructorargument. When
True, wildcards whose keys are absent from the provided kwargsare preserved verbatim in the output (including any constraint suffix, e.g.
{subject,\d+}) rather than raising aKeyError. This enables incremental /multi-step formatting of BIDS path templates — a template can be partially
resolved in one pass and the remainder resolved in a subsequent pass.
Underscore-squelching state is handled correctly for missing wildcards:
__d__,_TAG_d_) suppress the leadingunderscore of the next wildcard, matching the behaviour when the wildcard is
present.
_TAG_) restore the leading underscore of the nextwildcard, again matching the present-wildcard behaviour.
SnakemakeFormatter: Stricter value validationA new
_validatemethod is called whenever a caller-supplied value is used(i.e. when the key is found in
kwargsor when an integer positional key isused). The following rules are enforced:
__d__""or"/"___""or"_"_TAG_d_(directory wildcards)"/""_"or"/"Why stricter validation?
SnakemakeFormatterdecides whether to insert a leading underscore before eachwildcard by inspecting the final character of the previous value in the
format string. If a caller-supplied value ends with
_or/, that characteris indistinguishable from the literal separators that the formatter uses to
suppress underscores. As a result, a single-pass
formatter.format(template, **all_values)and an incremental multi-pass format (resolving one wildcard at atime) would silently produce different output for the same inputs. This
inconsistency would be extremely confusing to diagnose, so the formatter now
rejects such values up front with a clear
ValueError.Internal refactor:
squelch_underscore→_underscoreThe boolean
squelch_underscoreattribute has been replaced by the string_underscore, which holds either""(squelched) or"_"(active). Thiseliminates a conditional at every use site and makes the intent more explicit.
squelch_underscorewas already considered an implementation detail; it is notpart of the public API.
Tests
TestValidation: new test class covering all_validateerror andacceptance cases.
TestAllowMissing: new test class covering preservation of missing wildcards(with and without constraints) and reset of internal state between
vformatcalls.
test_single_and_multistep_format_equivalent: property-based test assertingthat formatting all wildcards in a single call produces the same result as
resolving them one-by-one via
allow_missing=True.test_bids.py: simplified template construction (brace-escaping of mandatoryvalues is no longer required), added
test_match_format_roundtripto verifythat regex-extracted wildcard values round-trip correctly through
SnakemakeFormatter.