Repository navigation
Decode Python string-literal escapes (\x, \u, \U, octal, control letters) (0041) - #490
Merged
Merged
Conversation
…l letters) (0041)
CompileEscapes decodes what Python's unicode-escape does, keeps the backslash of an unknown
escape, and a malformed escape is a parse error. \N{name} is still rejected.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
…conversion (0041) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Ruslan · project thread
Before: a string literal decoded only
\n,\rand\t;'A\x42'renderedAx42,'\v'renderedv, and any unknown escape lost its backslash.After:
CompileEscapesdecodes what Python's unicode-escape does:\a \b \f \n \r \t \v \\ \' \", backslash-newline (removed), 1-3 digit octal,\xHH,\uHHHH,\UHHHHHHHH(UTF-8, UTF-16 or UTF-32 by character type). An unknown escape such as\qkeeps its backslash, as in Python. A truncated or out-of-range escape is a parse error (ErrorCode::UnexpectedTokenon the literal), also in adjacent-literal runs.How:
CompileEscapesgains anokout-parameter and is split into small helpers insrc/helpers.h;LexerHelper::IsValidStringlets the expression parser reject a bad literal before its value is built (only literals containing a backslash are scanned twice).Deliberate divergence:
\N{name}needs the Unicode name table and is rejected as malformed (literals.string_escape_named, kindrejects, still owned by 0041, which staysin-progress). Also addedwide.filters.format_ascii_reprto the filters allow-list under 0035: it was hidden becauseéused to render asu00e9; the real cause is the wide-to-UTF-8 loss in the C locale.Tests: 11 new corpus cases in
literals.py, 3 rows leave the allow-list, unit tests inhelpers_tests.cpp. Full suite and preflight (format, clang-tidy 22) pass locally.Roles: none (main session); 0 verifier rounds; 0 red pushes so far.
🤖 Generated with Claude Code
https://claude.ai/code/session_01H8Fwx6ZYnAu1uY3tb6fSWK
Generated by Claude Code