Repository navigation
fix: validate exact error output in example checker - #7
Draft
1932819331hy-debug wants to merge 1 commit into
Draft
1932819331hy-debug wants to merge 1 commit into
1932819331hy-debug wants to merge 1 commit into
Conversation
Signed-off-by: YU HUANG <343001481+1932819331hy-debug@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #3.
The example checker currently accepts any missing/empty-secret stderr that
contains the expected message. A real Python child emitting that message
followed by an unexpected diagnostic passes the original checker.
This change requires the exact message and newline, while allowing only the
known
exit status 1trailer forgo run. Other prefixes, suffixes, exit codesand unexpected stdout are rejected. Failure messages still withhold child
output.
Adds standard-library regression tests, runs them in the existing maintenance
workflow, and documents the local command. The three-OS/nine-language matrix,
action pins, permissions and Sotto integration steps are unchanged.
Verification
Tested on Linux with Python 3.12.14 and Node.js 24.19.0:
python3 -m unittest discover -s tests -v: 10 tests pass. Before the fix,the new suite produced 49 failing subcases.
python3 scripts/check_examples.py python javascript: all eight real democases pass.
the patched checker rejects the same output without disclosure.
checker language selections, using synthetic subprocess results.
git diff --checkpass.Real Go and the other six non-Python/JavaScript runtimes were not exercised
locally; their real execution and macOS/Windows coverage remain for CI. No
Sotto installation, integration, keychain or credential operations were run.
This fixes validation in the examples repository, not the production secret
store. AI assistance was used for implementation, tests and independent review.