Repository navigation
Support additional single-byte DBF codepages - #22
Merged
Merged
Conversation
Contributor
|
lgtm! Thoughts @jefflroberts ? |
RandomChirp
approved these changes
Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support additional single-byte DBF codepages
The DBF reader recognizes many language-driver markers but decodes most of them
as CP1252. This turns Greek, Cyrillic, Baltic and other text into incorrect
characters, and writes can replace representable characters with question marks.
Add mappings for Windows 1250–1257, Thai 874 and DOS
437/737/850/852/857/860/861/863/865/866, retaining the existing pages. Both
decoding and encoding use the selected mapping. Correct marker 0x98 to Mac Greek
10006, while documenting that Macintosh codecs remain unimplemented.
Tests compare all byte values with referenced Microsoft mappings, exercise
character/memo writes followed by reopening, preserve binary memos, test field
widths and CPCONVERT, and import synthetic captions through the rebuilt WASM.
No new dependencies or application fixtures.
Validation on macOS ARM64: 121 Rust tests, 32 importer tests, rebuilt WASM,
typecheck, lint and build pass. Windows remains untested.
This is DBF codec support. Existing CP1252 fallback for unimplemented pages,
session-locale assumptions, DBCS, RTL and string built-ins remain separate work.
In particular, ALLTRIM currently truncates non-Latin-1 Unicode codepoints; that
can still corrupt correctly decoded text and is documented for follow-up.
Developed with AI assistance; checked with synthetic regression tests.