Skip to content

Update vendored usx.rng from tcdocs 3.1.2 - #427

Open
longbored wants to merge 1 commit into
Bridgeconn:masterfrom
longbored:refresh-usx-schema
Open

longbored wants to merge 1 commit into
Bridgeconn:masterfrom
longbored:refresh-usx-schema

Conversation

@longbored

@longbored longbored commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

User description

Refs #420

Picking up your note on that issue: "Update our current usx.rnc with latest one from tcdocs, test and conform."

The existing copy in schemas/ is from #207 (July 2023) and predates USX 3.1.2, so it rejects things the current spec allows (like efe in Footnote.style.enum, which is what blocks #423).

This takes grammar/usx.rng from the tcdocs 3.1.2 branch (as opposed to main), at commit b76e9f0. main has merged 3.2 and dropped the initial chapter requirement, so it accepts tests/introductions/bad/test1 ("a body paragraph is not allowed before the first chapter element", validated=fail), which the 3.1.2 schema correctly rejects. (Obviously we could just jump to 3.2, but that'd need to be handled.)

Test and conform. Against the two schema-validation tests that CI currently excludes (generated_usx_with_rnc_grammar, testsuite_usx_with_rnc_grammar), 59 failures become 43.

20 newly pass:

  • bugfixes/tcc, bugfixes/new-marker-ipc, bugfixes/new-marker-ta, bugfixes/new-marker-wl
  • advanced/figureInNote, biblica/CategoriesOnNotes, biblica/CrossRefWithPipe
  • specExamples/footnote, usfmjsTests/ts, usfmjsTests/ts_2, usfmjsTests/chunk_footnote

4 newly fail, and they're two fixtures counted twice: paratextTests/NoErrorsShort and special-cases/empty-book. Both are validated=fail in their metadata but overridden to "pass" in tests/__init__.py, one of them under your "Need to be fixed" heading. Each is only an \id and a \usfm line, so the newer schema is siding with the metadata against the override. I've left both alone rather than guess.

Everything else is green: tree-sitter test 76/76, and the CI python selection 3777 passed.

I also added schemas/readme.md recording the source URL, branch, commit and date, in the same spirit as tests/readme.md, with a one-line curl | diff so anyone can verify the copy against upstream. Happy to drop that file if you'd rather not carry it.

Scoped to the schema and that readme only. No fixture edits, no changes to the exclusion lists in tests/__init__.py, and no attempt to re-enable the excluded tests. Those felt like your calls once you can see the numbers.


CodeAnt-AI Description

Update USX validation to match the USX 3.1.2 specification

What Changed

  • USX validation now accepts constructs supported by USX 3.1.2, including extended footnotes such as efe
  • Validation recognizes the current rules for introductions, lists, paragraphs, character styles, figures, references, milestones, chapters, verses, and tables
  • Invalid structures such as missing book or chapter content continue to be rejected according to the 3.1.2 specification
  • The vendored schema now records its exact upstream source and revision for future verification

Impact

✅ Fewer valid USX files rejected
✅ Support for USX 3.1.2 footnotes and markup
✅ More consistent USX validation

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Summary by CodeRabbit

  • New Features

    • Expanded USX 3.1.2 validation to support additional Scripture and peripheral-book structures.
    • Improved validation for paragraphs, inline content, notes, references, tables, figures, milestones, sidebars, and metadata.
    • Added broader support for USFM styles, attributes, whitespace, embedded content, and character handling.
  • Documentation

    • Documented the schema’s source, version details, verification process, update history requirements, and the USJ JSON schema.

The copy in schemas/ dated from Bridgeconn#207 (July 2023) and predates USX 3.1.2,
so it rejected constructs the current spec allows. Most visibly it has no
efe in Footnote.style.enum, which blocks Bridgeconn#423.

Taken from the tcdocs 3.1.2 branch at b76e9f0, and adds schemas/readme.md
recording the source, branch, commit and date so the copy stays traceable
and diffable against upstream.

Against the schema-validation tests: 20 more pass, 4 fewer. The 4 are
paratextTests/NoErrorsShort and special-cases/empty-book, both marked
validated=fail and both consisting of only an \id and \usfm line, which
the newer schema correctly rejects.

Refs Bridgeconn#420
@codeant-ai

codeant-ai Bot commented Sep 8, 2026

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR 028c2cf Sep 08, 2026 · 05:34 05:36

@codeant-ai

codeant-ai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Sep 8, 2026
@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: e31e7cb5-ef48-4768-8b5e-d99476745d68

📥 Commits

Reviewing files that changed from the base of the PR and between 4ee1b91 and 028c2cf.

📒 Files selected for processing (2)
  • schemas/readme.md
  • schemas/usx.rng

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The PR updates the USX 3.1.2 Relax NG schema with expanded lexical rules, root models, peripheral-book support, richer content models, validation patterns, and style definitions. It also documents schema provenance, verification, and the usj.js schema.

Changes

USX schema validation

Layer / File(s) Summary
Lexical terminals and root contracts
schemas/usx.rng, schemas/readme.md
The schema adds shared whitespace, text, marker, identifier, verse, and reference matchers. Scripture and Peripheral roots use separate content models. Schema provenance and verification instructions are documented.
Book and peripheral parsing
schemas/usx.rng
Book and peripheral identification now use explicit marker, code, description, attribute, and newline parsing. Peripheral titles are optional and repeatable.
Document structures and tables
schemas/usx.rng
Header, paragraph, list, section, peripheral, and table models gain dedicated styles, paragraph matching, and expanded content choices.
Inline, character, and figure models
schemas/usx.rng
Character, note-character, attribute, figure, and milestone definitions are unified and expanded.
Verse, notes, and reference validation
schemas/usx.rng
Chapter, verse, footnote, cross-reference, sidebar, category, reference, break, remark, and cell-style validation is expanded.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: ⚪ Minimal · up to 028c2

The schema refresh broadens USX 3.1.2 validation and adds provenance documentation, with no actionable merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: updating the vendored usx.rng schema from tcdocs 3.1.2. It matches the pull request objectives and changed files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread schemas/usx.rng
Comment on lines +1526 to +1540
<value usfm:propattribs="src size ref alt? loc? copy?">fig</value>
</attribute>
<optional>
<ref name="char.closed"/>
</optional>
<optional>
<group>
<usfm:match match="PIPE" matchout="&#x27;|&#x27;" dump="true"/>
<oneOrMore>
<attribute usfm:fallback-from="style">
<usfm:matchpair first="/[^\\\n=&quot;]+/" between="&#x27;=&quot;&#x27;" second="/[^\\\n&quot;]*/" after="/&quot;${WSS}/" fallback="/[^\\\n=]+(?=\\)/"/>
<anyName>
<except>
<choice>
<name ns="">style</name>
<name ns="">closed</name>
</choice>
</except>
</anyName>
<data type="string"/>
</attribute>
</oneOrMore>
</group>
<ref name="char.closed" usfm:ignore="true"/>
</optional>
<group usfm:order="-1">
<optional>
<text>
<usfm:match match="TEXTATTRIB"/>
<usfm:match match="TEXTNOTATTRIB"/>
</text>
</optional>
</group>
<optional>
<ref name="Attributes"/>
</optional>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The active FigureThree rule makes all figure attributes optional, so <figure style="fig"/> validates despite missing required image source, size, and reference data. [incomplete implementation]

Assessment: 🟠 Major · 🔁 Occurrence: Sometimes

Use CodeAnt Skill Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** schemas/usx.rng
**Line:** 1526:1540
**Comment:**
	*Incomplete Implementation: The active `FigureThree` rule makes all figure attributes optional, so `<figure style="fig"/>` validates despite missing required image source, size, and reference data.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

@codeant-ai

codeant-ai Bot commented Sep 8, 2026

Copy link
Copy Markdown

CodeAnt Nitpicks

1 code suggestion

1. ipc appears twice in the introduction-style enum with different descriptions, creating contradictory schema documentation for the same value.

Code quality · schemas/usx.rng:677-678

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant