Skip to content

test: cover change-signature regressions - #2048

Closed
vitali87 wants to merge 1 commit into
pr-split/split-pr1548-rebased/pr-7from
pr-split/split-pr1548-rebased/pr-9
Closed

vitali87 wants to merge 1 commit into
pr-split/split-pr1548-rebased/pr-7from
pr-split/split-pr1548-rebased/pr-9

Conversation

@vitali87

Copy link
Copy Markdown
Owner

Regression coverage for duplicate sources, override keywords, positional-only markers, and related failures.

Files changed

  • codebase_rag/tests/test_change_signature_regressions.py

Diff stats

+206 additions, -0 deletions (206 LOC)

Dependencies

This PR depends on: pr-7

Dependency graph

Merge in this order:

pr-1: feat: add change-signature types and constants
├── pr-2: feat: parse and render signature parameters
│   ├── pr-4: feat: rewrite signature definitions and call sites
│   │   └── pr-5: feat: plan change-signature graph edits
│   │       └── pr-6: feat: expose change-signature via API, CLI, and MCP
│   │           ├── pr-7: test: cover change-signature units
│   │           │   ├── pr-8: test: cover change-signature acceptance flows
│   │           │   └── pr-9: test: cover change-signature regressions  <-- this PR
│   │           └── pr-10: docs: document change-signature architecture
│   └── pr-5: feat: plan change-signature graph edits
│       └── pr-6: feat: expose change-signature via API, CLI, and MCP
│           ├── pr-7: test: cover change-signature units
│           │   ├── pr-8: test: cover change-signature acceptance flows
│           │   └── pr-9: test: cover change-signature regressions  <-- this PR
│           └── pr-10: docs: document change-signature architecture
├── pr-3: feat: map change-signature call arguments
│   ├── pr-4: feat: rewrite signature definitions and call sites
│   │   └── pr-5: feat: plan change-signature graph edits
│   │       └── pr-6: feat: expose change-signature via API, CLI, and MCP
│   │           ├── pr-7: test: cover change-signature units
│   │           │   ├── pr-8: test: cover change-signature acceptance flows
│   │           │   └── pr-9: test: cover change-signature regressions  <-- this PR
│   │           └── pr-10: docs: document change-signature architecture
│   └── pr-5: feat: plan change-signature graph edits
│       └── pr-6: feat: expose change-signature via API, CLI, and MCP
│           ├── pr-7: test: cover change-signature units
│           │   ├── pr-8: test: cover change-signature acceptance flows
│           │   └── pr-9: test: cover change-signature regressions  <-- this PR
│           └── pr-10: docs: document change-signature architecture
├── pr-4: feat: rewrite signature definitions and call sites
│   └── pr-5: feat: plan change-signature graph edits
│       └── pr-6: feat: expose change-signature via API, CLI, and MCP
│           ├── pr-7: test: cover change-signature units
│           │   ├── pr-8: test: cover change-signature acceptance flows
│           │   └── pr-9: test: cover change-signature regressions  <-- this PR
│           └── pr-10: docs: document change-signature architecture
└── pr-5: feat: plan change-signature graph edits
    └── pr-6: feat: expose change-signature via API, CLI, and MCP
        ├── pr-7: test: cover change-signature units
        │   ├── pr-8: test: cover change-signature acceptance flows
        │   └── pr-9: test: cover change-signature regressions  <-- this PR
        └── pr-10: docs: document change-signature architecture

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)
  • ^feat/.*
  • ^fix/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: vitali87/code-graph-rag/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 0ffd9f0c-e492-4745-9cb9-bbebb60865a2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

Safe to merge from a production-behavior perspective, but the new regression coverage should be tightened so it reliably detects the failures it is intended to prevent.

Findings

  1. P2 Broken calls can pass ▶
  2. P2 Refusal reason is unchecked ▶

Summary

This change adds useful signature-change regression coverage, but two assertions can pass while the behavior they are intended to protect is broken. Tightening the call-rewrite and positional-only refusal checks will make the new coverage dependable.

Reviews (1) · Last reviewed commit: "test: cover change-signature regressions"

assert report.applied, report.message
text = (root / "pkg/shapes.py").read_text()
# Both definitions are rewritten, so no caller may still say `factor`.
assert "factor=2" not in text, text

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Broken calls can pass

This assertion only verifies that factor=2 disappeared. It also passes if the call is removed or changed to an invalid keyword, so the regression test can succeed while the rewritten program is no longer callable. Assert the rewritten signature and Square().area(size=2), and preferably execute the rewritten fixture to check its result. This is a non-blocking test-coverage concern; the practical cost is that future regressions can silently ship.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Command output from the check

  • Executed the prepared uv pytest command against the changed regression module; all six tests passed, showing the current weak assertion is accepted.

Evidence from the check

  • Authored and executed the focused Python harness that invokes the real rewrite flow and tests two broken rewritten-call mutants, providing the reproducible validation method.

Command output from the check

  • Captured the command and complete contents of the exact mutation harness that was executed, making the oracle test reproducible.

Command output from the check

  • Executed the mutation harness: line 146 accepted the invalid-keyword and removed-call results while their runtime behavior was broken, confirming the test-oracle weakness.

View artifacts

T-Rex Ran code and verified through T-Rex

"def call():\n return target(1, 2, 3)\n",
)
store, updater = _index(root)
with pytest.raises(SignatureRefused):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Refusal reason is unchecked

The bare pytest.raises(SignatureRefused) accepts any refusal, including one caused by an unrelated malformed parameter specification. Capture the exception and assert the positional-only-boundary diagnostic so this test specifically proves that moving the boundary is rejected. This is a non-blocking test-coverage concern; the practical cost is that the intended guard could regress without this test detecting it.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Evidence from the check

  • Authored and executed script constructs the same temporary indexed repository as the changed test and runs the intended and malformed-spec requests, demonstrating what the bare assertion accepts.

Command output from the check

  • Executed repro with the exact changed-test request; it raises SignatureRefused with the positional-only boundary message, establishing the intended behavior.

Command output from the check

  • Executed repro adds malformed `invalid@`; it still raises SignatureRefused but with a parameter-spec message, proving the bare assertion lacks diagnostic specificity.

Command output from the check

  • Executed the changed test node under pytest; it passed while asserting only the broad exception class, confirming the weak assertion is active.

View artifacts

T-Rex Ran code and verified through T-Rex

@greptile-apps

greptile-apps Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

  • P2 Regression test accepts broken rewritten calls ▶

    • Bug
      • At codebase_rag/tests/test_change_signature_regressions.py:146, the test only checks that the old literal factor=2 is absent. The executed mutation harness proved that an invalid replacement sizee=2 and removal of the call both satisfy that assertion while respectively raising TypeError and returning None rather than the expected 8. The real current rewrite also showed factor remaining in the method body, yielding NameError, which this oracle does not observe.
    • Cause
      • The oracle is a negative raw-text check only; it does not assert both rewritten method signatures, the exact rewritten invocation, or execute use_square() to validate behavior.
    • Fix
      • Assert both rewritten definitions contain def area(self, size):, assert the call contains Square().area(size=2), and execute/import the rewritten fixture with assert use_square() == 8.
  • P2 Positional-only regression test accepts unrelated SignatureRefused failures ▶

    • Bug
      • At codebase_rag/tests/test_change_signature_regressions.py:198, pytest.raises(SignatureRefused) verifies only exception type. The executed repro showed that the intended request raises the positional-only-boundary message, but an independently malformed invalid@ parameter specification raises a different SignatureRefused and would satisfy that same assertion. Thus the test does not diagnose the specific guard it claims to cover.
    • Cause
      • The assertion neither captures excinfo nor matches/asserts the SIGNATURE_POSITIONAL_ONLY_MOVED diagnostic.
    • Fix
      • Capture the exception and assert a stable, specific diagnostic, e.g. with pytest.raises(SignatureRefused, match="positional-only"): or assert the relevant stable wording/constant against str(excinfo.value).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Strengthen assertions for keyword rewriting and positional-only marker placement, and correct the misleading fixture docstring.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 Medium severity · 1 Low severity

Open (3)
What changed in this PR

Adds regression tests for change_signature covering duplicate sources, override keyword rewriting, and positional-only parameters.

Changes:

  • Tests duplicate-source rejection and distinct mappings.
  • Covers override-specific keyword names.
  • Tests positional-only marker preservation and boundary validation.
File Description
codebase_rag/​tests/​test_change_signature_regressions.py Adds regression coverage for signature rewriting failures.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

assert report.applied, report.message
text = (root / "pkg/shapes.py").read_text()
# Both definitions are rewritten, so no caller may still say `factor`.
assert "factor=2" not in text, text
reingest=updater.reingest,
)
assert report.applied, report.message
assert "/" in (root / "pkg/only.py").read_text().split("\n")[0]
Comment on lines +53 to +56
Its own fixture, with no defaulted parameter: the shared one has
`b: str = 'x'` and a `helper(2)` site relying on it, which a rename leaves
possibly-missing and the postcondition rightly rolls back. That rollback is
the operation working correctly, so the shared fixture would be testing
@vitali87
vitali87 force-pushed the pr-split/split-pr1548-rebased/pr-7 branch from 603be17 to 8a3e861 Compare September 22, 2026 22:00
@vitali87
vitali87 force-pushed the pr-split/split-pr1548-rebased/pr-9 branch from 07334a1 to 8760ea7 Compare September 22, 2026 22:00
@vitali87

Copy link
Copy Markdown
Owner Author

Superseded: change_signature lands through the split of #1912, which replaces #1548, as the stack #2163 → #2161 → #2162 → #2166 → #2164 → #2169 → #2168 → #2165 → #2167 on main. Closing this split-pr1548 stack at the owner's direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants