Repository navigation
Writer fidelity: run text, escaping, reproducible bytes, paragraph and table properties - #7
Conversation
Run text went into a single <w:t> as is, so a "\t" or "\n" ended up inside it, where Word shows it as a space at best. The reader turns <w:tab/> into "\t", so a document read and written back lost its tabs, and the converter keeps the newlines of a <pre>, which then lost its line breaks. The writer now splits the text on "\t", "\r\n", "\r" and "\n" and writes <w:tab/> / <w:br/> between <w:t> segments in the same <w:r>, leaving out empty segments. This covers runs anywhere (body, headers, footers, hyperlinks, lists, cells) and the table caption. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
htmlspecialchars without ENT_SUBSTITUTE returns '' for a string with an invalid UTF-8 sequence, so the whole run text or attribute value was silently lost. And C0 control characters other than tab, newline and carriage return passed through, which makes the part malformed: Word refuses to open the file. text() and attr() now escape with ENT_SUBSTITUTE (an invalid sequence becomes U+FFFD) and remove every character outside the XML 1.0 Char production (C0 controls, U+FFFE, U+FFFF). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ZipArchive stamped every entry with the current time, so two writes of
the same document differed once a 2-second DOS tick passed. Every entry
now carries CoreProperties::$modified as a UTC wall-clock time, or
1980-01-01 00:00:00 when it is not set.
ZipArchive::setMtimeName() is not enough on its own: libzip converts the
Unix timestamp to a DOS date/time with the C localtime() and the
process's TZ (not PHP's date.timezone, and putenv('TZ=...') before
close() does not change it), so one timestamp still gave different
bytes on machines in different zones. ZipTimestamp writes the DOS
time/date fields into each local file header and central directory
record after close(). libzip writes no extended-timestamp extra field,
and the CRC does not cover these fields. An archive it cannot parse
(ZIP64, comment) is returned unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DocumentBuilder::build() never set Document::$coreProperties and the builder had no way to pass them, so docProps/core.xml (title, author, dates) was only reachable by building the Document by hand. coreProperties(CoreProperties) stores them and build() passes them on. The README documents it (in all four languages) along with the fixed entry time from the previous commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paragraph::$styleId (appended to the constructor) and ParagraphBuilder::styleId() write any paragraph style id as <w:pStyle>: Normal, Title, Quote, or a custom id registered with StyleRegistry::paragraph(). Before, only Heading1..6 could be referenced (through headingLevel). An explicit id wins over the heading level; list items keep ListParagraph. The id is XML-escaped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ParagraphStyle::$pageBreakBefore and ParagraphStyle::$widowControl (?bool: null writes nothing, false writes w:val="0") are written in CT_PPrBase order (after keepLines, before numPr), read back by OoxmlPropertyParser/StylesResolver, and available on ParagraphBuilder. The reader already parsed pageBreakBefore but dropped it on the way into the model, so a paragraph that started a new page lost its break on a read/write round trip. HTML: pageBreakBefore serializes as break-before: page; the converter parses break-before: page and page-break-before: always. widowControl has no HTML mapping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RunStyle::$fontFamilyEastAsia (appended to the constructor), with withFontFamilyEastAsia() and RunStyleBuilder::fontFamilyEastAsia(), is written as w:eastAsia on <w:rFonts>, in document runs and in style run properties (a shared RunFonts helper). <w:rFonts> is written when either font is set; with only fontFamily the output is unchanged. The reader keeps an explicit w:eastAsia in its own slot; an eastAsiaTheme reference is not resolved (the theme map holds the Latin faces only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
computeGridColumns() repeated a spanning cell's full width for each column it spans, so a 2000-twip cell with gridSpan 2 produced two 2000-twip gridCols and the grid was wider than the cell. The spanned columns now share the width (width / span). The grid was also taken from the first row only. A column now takes its width from the first row where an unspanned cell with a known width covers it, falling back to a spanning cell's share, then to 2000 twips. A title row spanning the whole table no longer decides the columns of the rows below it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TableStyle::$indentTwips and TableBuilder::indent() / indentMm() / indentPt() write <w:tblInd w:w=".." w:type="dxa"/> in CT_TblPr order (after jc, before tblBorders). TableReader parses a dxa tblInd back; nil and auto read as no indent. The HTML serializer writes it as margin-left on the <table> (unless the table is centred), and the converter reads a length margin-left back as the indent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The writer wrote every row height as w:hRule="atLeast". TableRow gains $heightRule (RowHeightRule::AtLeast, the default, or ::Exact), written as w:hRule, and TableRowBuilder::exactHeight(). TableReader reads hRule="exact" back; atLeast, auto and a missing hRule read as AtLeast, as before. The rule is carried through the reader's and the HTML converter's row-span reconstruction. The HTML serializer does not write row heights today, so the rule is not serialized either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Writer fidelity (from #7) with follow-up fixes
|
Thank you, @serpentblade — this was a careful, well-researched contribution. The libzip/ All ten of your commits are merged unchanged, with your authorship, through #8: GitHub would not let us push to this branch, so we carried them over to a branch here and added four follow-up commits on top. What we changed, and why:
A small request for next time: please tick "Allow edits by maintainers" when opening a pull request. For a fork owned by an organisation GitHub may still refuse, so a fork under your personal account works best. Then we can push small fixes straight into your branch and keep the conversation in one place. Merged through #8. Thanks again! |
Thanks for merging #3 and #4. This PR collects the rest of the gaps we found while building a DOCX exporter on the library. It is one PR instead of four. The commits are grouped by area, and each one passes the full suite and
composer stanon its own, so any group can be dropped or cherry-picked.fix: write tabs and newlines in run text as w:tab and w:brfix: XmlEscape keeps invalid UTF-8 text and drops non-XML charactersfix: deterministic ZIP output (fixed entry timestamps)feat: DocumentBuilder::coreProperties()feat: arbitrary paragraph style id (w:pStyle)feat: w:pageBreakBefore and w:widowControl on paragraphsfeat: East Asian font (w:eastAsia) on run stylesfix: split a spanning cell's width across its grid columnsfeat: table indent (w:tblInd)feat: exact row height (w:hRule="exact")All new constructor parameters are appended at the end, so positional callers are unaffected.
isEmpty()andcopy()cover the new fields. The CHANGELOG has one[Unreleased]section, and the README is updated in all four languages wherever the public API changed.1. Run text
Tabs and newlines.
renderRun()put all ofRun::$textinto one<w:t>, where Word shows a"\t"or"\n"as a space at best. The library hit this on its own round trips.BodyReaderreads<w:tab/>as"\t", so a document read and written back lost every tab.Converterkeeps a<pre>'s newlines in the run text, so preformatted HTML came out as one line. The writer now splits the text on"\t","\r\n","\r"and"\n", and writes<w:tab/>and<w:br/>between<w:t>segments in the same<w:r>:Empty segments are left out. Text without tabs or newlines, and an empty run, give the same output as before. The table caption uses the same helper.
Escaping.
XmlEscapehad two problems:htmlspecialchars()withoutENT_SUBSTITUTE, so a string with one invalid UTF-8 byte became''. The whole run text, hyperlink target or font name disappeared without a warning. The bad sequence now becomes U+FFFD, and the rest of the string is kept.Charproduction (C0 controls other than tab, LF and CR, plus U+FFFE and U+FFFF) were written as they were, and Word refused to open the file. They are now removed. They have no visible form, so a stray\x0Bfrom a paste shouldn't turn into a replacement glyph.This applies to
text()andattr()alike, so relationship targets, style names,lvlTextandcore.xmlare covered too.2. Reproducible bytes and core properties
Deterministic ZIP. Identical input produced identical XML, but ZipArchive stamped every entry with the current time. Two writes more than one 2-second DOS tick apart therefore differed, which breaks checksums, cache keys, snapshot tests and diffs of generated files. Every entry now carries
CoreProperties::$modified, read as a UTC wall-clock time. When that isn't set, it carries 1980-01-01 00:00:00, the earliest DOS time. Values outside 1980–2107 are clamped.ZipArchive::setMtimeName()alone does not make the bytes match across machines. libzip 1.11.4 converts the timestamp with Clocaltime(), which reads the process'sTZ. PHP'sdate.timezonemakes no difference, and neither doesputenv('TZ=UTC')beforeclose().setMtimeName($name, 315532800)writes:TZUTC0000/0021(1980-01-01 00:00)Asia/Tokyo4800/0021(09:00)America/New_York9800/019f(19:00 on an invalid "1980-12-31")libzip writes no extended-timestamp or NTFS extra fields. The DOS time and date in the local header and in the central directory record are the only places the time is stored, and nothing else in the headers varied between runs. So
Writer\ZipTimestamp::apply()overwrites those two fields for each entry afterclose(), walking the central directory from the end record. The CRC doesn't cover them, andZipArchive::open(..., CHECKCONS)accepts the result. An archive it can't read safely (ZIP64, a comment, an unexpected signature) is returned unchanged; the writer never produces one.sha1(DocumentBuilder::new()->paragraph('Hi')->toBytes())is now the same underTZ=UTC,Asia/TokyoandAmerica/New_York.ZipTimestampis a final class with one static method. It is public only because it lives in the writer's namespace, so it could be marked@internal.DocumentBuilder::coreProperties().build()never setDocument::$coreProperties, so builder users had to construct theDocumentby hand to get a title or author:A caller who wants real timestamps passes
modified: new DateTimeImmutable().3. Paragraphs and runs
Paragraph::$styleId. A paragraph could reference onlyHeading1..6, throughheadingLevel. Styles registered withStyleRegistry::paragraph()could be defined but never applied. The id is written as<w:pStyle>and XML-escaped. An explicit id wins overheadingLevel, and list items keepListParagraph. The reader doesn't fill it in: it resolves style formatting into the paragraph's own styles, and the writer's registry may not contain the source document's styles.pageBreakBefore. Until now the only page break was aPageBreakblock, which writes a paragraph of its own and leaves an empty line at the top of the new page. The reader already parsedw:pageBreakBeforebut dropped it before the model, so this also fixes a round-trip loss. HTML output isbreak-before: page, andbreak-before: pageandpage-break-before: alwaysare parsed back.widowControlis a?bool.nullwrites nothing,truewrites<w:widowControl/>, andfalsewritesw:val="0". Becausecopy()treatsnullas "keep", it cannot reset the value tonull. There is no CSS mapping, becausewidowsandorphansare counts, not an on/off switch.w:pProrder follows CT_PPrBase:pStyle,keepNext,keepLines,pageBreakBefore,widowControl,numPr,pBdr, …fontFamilyEastAsiais written asw:eastAsiaon<w:rFonts>, both in document runs and instyles.xml. With onlyfontFamilyset, the output is byte-identical to before. The reader reads an explicitw:eastAsia.eastAsiaThemeisn't resolved, because the theme map holds only the Latin fonts.4. Tables
Table::$gridColumnsTwips,computeGridColumns()repeated a spanning cell's full width for every column it spans. A 2000-twipgridSpan: 2cell next to a 2000-twip cell gave2000 / 2000 / 2000, i.e. 6000 twips for a 4000-twip row. Widths also came from the first row only, so a title row spanning the table set every column (5000 over 1500 + 3500 gave5000 / 5000). Now each column takes its width from the first row where an unspanned cell with a width covers it. Failing that, it takes a spanning cell's width divided by its span, and failing that, 2000 as before. The number of columns still comes from the first row. Tables with explicit grid columns are unaffected.TableStyle::$indentTwipsis written as<w:tblInd w:type="dxa">at its place in CT_TblPr, betweenjcandtblBorders. Negative values are allowed. The reader reads it back. In HTML it is written asmargin-lefton the<table>, except for a centred table, andTableStyleApplierparses a lengthmargin-leftback. As a result, HTML that already setmargin-lefton a table now gets an indented table in Word.TableRow::$heightRuleis a backed enum,RowHeightRule::AtLeast(the default) orRowHeightRule::Exact, likeVerticalAlign. Before, everyw:trHeightwasatLeast. The reader mapsexacttoExact, andatLeast,autoor a missing rule toAtLeast.TableRowBuilder::height()resets the rule toAtLeast, so whichever ofheight()andexactHeight()is called last wins. The serializer doesn't write row heights today, so the rule isn't serialized either.Not included
styles.xml) line spacing, caps andoutlineLvl, and a configurabledocDefaults(font, size,w:lang). These touch howStyleRegistryis designed, so we'd rather open an issue first.ParagraphStyle/RunStyleand drop several base fields (lineSpacingRule,keepWithNext,keepLines, caps).ThemeResolverstores the minor-font alias asminorHAnsi, butresolveFontlooks keys up in lowercase.Tests
composer install && vendor/bin/phpunit: OK (468 tests, 1255 assertions). Before: OK (420 tests, 1068 assertions).composer stan: no errors. Each commit passes on its own: 429, 434, 440, 442, 446, 451, 455, 458, 464 and 468 tests.Notable tests:
tests/Writer/RunTextTest.php, with a DocxReader → Word2007Writer round trip.tests/Writer/XmlEscapeTest.php:document.xmland.relsparse withDOMDocumenteven with\x01and invalid UTF-8 in the input.tests/Writer/DeterministicOutputTest.phpreads the DOS fields from the raw headers.statName()would convert through the local time zone and hide the difference. The test needs nosleep()and passes under all three time zones above.🤖 Generated with Claude Code