Skip to content

fix(clipboard): truncate the preview on a rune boundary - #35

Merged
Nadim147c merged 1 commit into
Nadim147c:mainfrom
WayneKent:fix/preview-utf8-truncation
Oct 29, 2026
Merged

Nadim147c merged 1 commit into
Nadim147c:mainfrom
WayneKent:fix/preview-utf8-truncation

Conversation

@WayneKent

Copy link
Copy Markdown
Contributor

Problem

The preview is capped at 8 KiB, but the cap was applied with a plain byte cut
(w.buf.Truncate(maxPreviewLength)). When that cut lands inside a multi-byte
rune, the preview ends in a partial UTF-8 sequence.

DuckDB rejects an invalid UTF-8 string bound as a VARCHAR, so

INSERT INTO events (primary_mime_type, time, preview) VALUES (?, ?, ?)

fails and the daemon only reports:

ERRO failed to insert data
  error=
  │ could not bind parameter

The transaction rolls back and the whole clipboard event is lost: yankd list
never shows it.

Because CJK runes are 3 bytes, 8192 % 3 == 2, so for Chinese content the cut
almost always splits a rune. ASCII copies are unaffected — that is why only some
copies were lost. Reported case: copying a long Chinese markdown document from a
web page.

Change

  • Add runeSafeLength(buf, n), which backs the cut off to the start of the last
    partial rune, so the preview is always valid UTF-8.
  • At most 3 bytes are dropped: the buffer only ever holds valid UTF-8, because
    only entries whose IsText is true are appended to it.
  • The preview is still capped at 8 KiB by bytes; only the cut point moved.

Tests

TestGeneratePreivewKeepsPreviewValidUTF8 covers:

  • 3-byte runes, where the cap lands mid-rune
  • 4-byte runes, where the cap lands mid-rune
  • 4-byte runes with an aligned cap, and pure ASCII — these two also fail if the
    fix ever over-trims by a fixed offset

The first two fail without the fix and pass with it.

Reproducing on the unfixed version

Only payloads larger than 8 KiB are affected, so any clipboard content above
that size with multi-byte characters reproduces it:

# 1. build a payload that is just over the cap
line='这是一段用于复现预览截断问题的中文内容,复制它会触发 yankd 的预览生成逻辑。'
yes "$line" | head -n 75 > /tmp/yankd-repro.md   # 8250 bytes

# 2. sanity check: the 8 KiB cap really does land inside a rune
head -c 8192 /tmp/yankd-repro.md | iconv -f UTF-8 -t UTF-8 >/dev/null \
  && echo "aligned — shift the payload by one ASCII byte and try again" \
  || echo "this payload hits it"

# 3. put it on the clipboard (needs wl-clipboard; copying the same
#    8250-byte text from any app works too)
wl-copy --type text/plain < /tmp/yankd-repro.md

With the unfixed build and the daemon running:

  • the daemon logs ERRO failed to insert data ... could not bind parameter
  • the entry never appears in yankd list

A browser copy of the same text also hits it — a browser additionally advertises
escaped \uXXXX variants of the text, which only moves the cut point, so if a
particular document happens to be aligned, add or remove one ASCII character and
copy again.

- back the preview cut off to the start of the last partial rune, so the
  preview is always valid UTF-8
- DuckDB rejects an invalid-UTF-8 VARCHAR bind, which failed the insert and
  dropped the whole clipboard event for any copy over 8 KiB containing
  multi-byte characters
- add TestGeneratePreivewKeepsPreviewValidUTF8: three- and four-byte runes,
  an aligned cut and pure ASCII; the last two also guard against
  over-trimming
@Nadim147c
Nadim147c merged commit 7442311 into Nadim147c:main Oct 29, 2026
1 check passed
@Nadim147c

Copy link
Copy Markdown
Owner

Thanks for the pull request!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants