Skip to content

fix(user-input-fold): preserve supplementary characters at prose cutoff - #586

Merged
tt-a1i merged 1 commit into
openpi-dev:mainfrom
ooiuuii:fix/fold-unicode-boundary
Sep 20, 2026
Merged

tt-a1i merged 1 commit into
openpi-dev:mainfrom
ooiuuii:fix/fold-unicode-boundary

Conversation

@ooiuuii

@ooiuuii ooiuuii commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Problem

Fixes #585.

The first-line prose cutoff in folded user messages can split an emoji or supplementary CJK character, producing a replacement character when terminal output is encoded as UTF-8.

Value

Keep the preview faithful to complete input characters while retaining the existing preview size and folding behavior.

Approach

Move the existing cutoff back one code unit only when it falls between a matching high/low surrogate pair. No new helper, configuration or rendering framework; all other folding paths remain unchanged.

Validation

  • Actual finalized-user Markdown transformer: supplementary characters straddling the 1,200-unit boundary fail the UTF-8 round-trip assertion before the fix. Focused suite: 26 pass / 1 fail before, 27/27 pass after.
  • Controls cover a complete pair fitting the budget, BMP text, existing fold marker and unchanged streaming input.
  • node --experimental-strip-types --test tests/extensions/user-input-fold/index.test.ts, bun run check, git diff --check — pass.
  • Scoped structured review (autoreview --mode local --engine codex --model gpt-6-astra --thinking xhigh --no-web-search): helper exit0, no actionable findings.
  • Synthetic source-checkout fixtures, no private messages or provider calls. No installed-app E2E claim. Full local suite has separately reproduced host baseline timeout/Windows process-cleanup limitations; fresh CI is the broader gate.

Impact

  • User-visible: no broken supplementary character at this folded prose boundary.
  • Model-visible context and stored Session content: unchanged; original input remains intact.
  • Runtime/lifecycle, config and compatibility: no changes. Existing code-unit budget, thresholds, streaming behavior and fenced-block handling preserved.

@github-actions github-actions Bot added the area:ui Terminal UI, rendering, themes, or visual assets label Sep 19, 2026
@tt-a1i
tt-a1i merged commit a96746a into openpi-dev:main Sep 20, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ui Terminal UI, rendering, themes, or visual assets

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(user-input-fold): prose cutoff splits supplementary characters

2 participants