Skip to content

feat: partial template formatting and stricter wildcard value validation - #532

Merged
pvandyken merged 1 commit into
mainfrom
feat/allow-missing
Feb 28, 2026
Merged

pvandyken merged 1 commit into
mainfrom
feat/allow-missing

Conversation

@pvandyken

Copy link
Copy Markdown
Contributor

Changes

SnakemakeFormatter: Add allow_missing mode

SnakemakeFormatter now accepts an allow_missing: bool = False constructor
argument. When True, wildcards whose keys are absent from the provided kwargs
are preserved verbatim in the output (including any constraint suffix, e.g.
{subject,\d+}) rather than raising a KeyError. This enables incremental /
multi-step formatting of BIDS path templates — a template can be partially
resolved in one pass and the remainder resolved in a subsequent pass.

Underscore-squelching state is handled correctly for missing wildcards:

  • Missing directory wildcards (__d__, _TAG_d_) suppress the leading
    underscore of the next wildcard, matching the behaviour when the wildcard is
    present.
  • Missing dummy wildcards (_TAG_) restore the leading underscore of the next
    wildcard, again matching the present-wildcard behaviour.

SnakemakeFormatter: Stricter value validation

A new _validate method is called whenever a caller-supplied value is used
(i.e. when the key is found in kwargs or when an integer positional key is
used). The following rules are enforced:

Wildcard Accepted values
__d__ "" or "/"
___ "" or "_"
_TAG_d_ (directory wildcards) any string ending in "/"
all other entities any string not ending in "_" or "/"

Why stricter validation?

SnakemakeFormatter decides whether to insert a leading underscore before each
wildcard by inspecting the final character of the previous value in the
format string. If a caller-supplied value ends with _ or /, that character
is indistinguishable from the literal separators that the formatter uses to
suppress underscores. As a result, a single-pass formatter.format(template, **all_values) and an incremental multi-pass format (resolving one wildcard at a
time) would silently produce different output for the same inputs. This
inconsistency would be extremely confusing to diagnose, so the formatter now
rejects such values up front with a clear ValueError.

Internal refactor: squelch_underscore → _underscore

The boolean squelch_underscore attribute has been replaced by the string
_underscore, which holds either "" (squelched) or "_" (active). This
eliminates a conditional at every use site and makes the intent more explicit.
squelch_underscore was already considered an implementation detail; it is not
part of the public API.

Tests

  • TestValidation: new test class covering all _validate error and
    acceptance cases.
  • TestAllowMissing: new test class covering preservation of missing wildcards
    (with and without constraints) and reset of internal state between vformat
    calls.
  • test_single_and_multistep_format_equivalent: property-based test asserting
    that formatting all wildcards in a single call produces the same result as
    resolving them one-by-one via allow_missing=True.
  • test_bids.py: simplified template construction (brace-escaping of mandatory
    values is no longer required), added test_match_format_roundtrip to verify
    that regex-extracted wildcard values round-trip correctly through
    SnakemakeFormatter.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for incremental / partial Snakemake template formatting by preserving missing wildcards (allow_missing=True), and introduces stricter validation for wildcard values to prevent underscore/slash state ambiguities during multi-step formatting.

Changes:

  • Extend SnakemakeFormatter with allow_missing behavior and internal underscore-state refactor.
  • Add _validate() to reject wildcard values that could cause inconsistent single-step vs multi-step formatting.
  • Expand and adjust tests, including new validation/allow-missing suites and a BIDS match→format roundtrip test.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
src/snakebids/utils/snakemake_templates.py Implements allow_missing, underscore-state refactor, and stricter value validation.
tests/test_snakemake_templates/test_snakemake_formatter.py Adds/updates property-based tests for missing-wildcard preservation and validation rules.
tests/test_paths/test_bids.py Simplifies template construction, adds match→format roundtrip test, and changes active spec-version test suites.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/snakebids/utils/snakemake_templates.py
Comment thread src/snakebids/utils/snakemake_templates.py Outdated
Comment thread src/snakebids/utils/snakemake_templates.py Outdated
Comment thread tests/test_paths/test_bids.py Outdated
Comment thread src/snakebids/utils/snakemake_templates.py Outdated
@codecov

codecov Bot commented Feb 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.26%. Comparing base (ef96e61) to head (a564279).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #532      +/-   ##
==========================================
+ Coverage   95.18%   95.26%   +0.07%     
==========================================
  Files          56       56              
  Lines        2451     2492      +41     
==========================================
+ Hits         2333     2374      +41     
  Misses        118      118              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/snakebids/utils/snakemake_templates.py
Comment thread src/snakebids/utils/snakemake_templates.py Outdated
Comment thread src/snakebids/utils/snakemake_templates.py Outdated
Implement the `allow_missing` flag from the snakemake expand function,
allowing wildcards to be omitted from the function call.

To ensure a 1-step format with `allow_missing=False` would be equivalent
to a multistep format with `allow_missing=True`, validation was added to
prevent specially crafted values from stymieing the underscore
squelching behaviour
@pvandyken
pvandyken merged commit 59300e8 into main Feb 28, 2026
5 of 6 checks passed
@pvandyken
pvandyken deleted the feat/allow-missing branch February 28, 2026 02:35
Copilot AI added a commit that referenced this pull request Feb 28, 2026
- Replace snakemake_templates.py with PR #532's version:
  - Uses _current_field (not _field_constraints dict)
  - Uses _underscore: str (not squelch_underscore: bool)
  - Adds _validate() for entity value validation
  - Wraps get_value() to catch KeyError for allow_missing
- Replace test_snakemake_formatter.py with PR #532's version
- Update test_bids.py with latest main version
- expanding.py and datasets.py expand reimplementation unchanged

Co-authored-by: pvandyken <87136354+pvandyken@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants