Skip to content

156 expand metacore schema - #157

Draft
LiamHobby wants to merge 38 commits into
mainfrom
156-expand-metacore-schema
Draft

LiamHobby wants to merge 38 commits into
mainfrom
156-expand-metacore-schema

Conversation

@LiamHobby

@LiamHobby LiamHobby commented Jun 6, 2026 •

Copy link
Copy Markdown
Collaborator

PR Summary: 156-expand-metacore-schema

What Changed

1. New MetacoreDefine R6 subclass (R/MetacoreDefine.R)

A new class MetacoreDefine inherits from Metacore and adds three extra tables needed to fully
describe a CDISC Define.xml document:

  • study_level: study-wide metadata (study name, protocol, standard, define version, language)
  • documents: referenced external documents (ID, title, href)
  • comments: comments cross-referenced from value_spec

The subclass overrides initialize(), validate(), and metacore_filter(). The filter method now
trims documents and comments to only those still referenced after subsetting to a single dataset.

2. Schema formalised into a single source of truth (R/metacore-schema.R)

Previously column names and types were scattered across initializer code. Now there are two
explicit schema functions:

  • base_column_schema: the original 7-table schema (backwards-compatible)
  • define_column_schema: extends the base with Define.xml-only columns and the 3 new tables

Alongside these, base_col_regex() and define_col_regex() centralise the regex patterns used to
locate and rename columns when reading Excel specs. Internal helpers fill_cols(), col_vars(),
reorder_by_schema(), and define_only_cols() use these schemas to keep initialisation and
validation in a single location.

3. Extended schema columns on existing tables

When define_fields = TRUE the following existing tables gain additional columns:

Table New Columns
ds_spec class, repeating, reference, purpose
ds_vars role
value_spec where_label, comment_id
derivations method_name, method_type, document_id, pages

4. define_fields opt-in parameter added to spec_to_metacore and all spec_type_to_* builders

All reader functions now accept define_fields = FALSE (the default). When FALSE the readers
produce the classic schema and return a plain Metacore object so existing code should remain valid. When
TRUE they read the extra columns, build the three new tables, and return a MetacoreDefine object.

5. Validators updated

Validators now inspect the active schema (base vs define) to decide which columns are required,
so they don't raise spurious errors about missing Define.xml columns on plain Metacore objects,
and vice versa.

6. metacore.R refactored to R6 naming convention

The core R6 class was renamed from mixed internal naming to MetaCore (internal R6 class name),
following proper R6 conventions. Public-facing constructor and active binding names are
unchanged. Shared initialisation logic was extracted into .metacore_init_base_tables() so both
MetaCore and MetaCoreDefine share it without duplication.

7. spec_type_to_comments and spec_type_to_documents builder functions added

Two new spec_type_to_* functions read the Comments and Documents sheets from Excel
specifications when define_fields = TRUE.

Why it Changed

The driving need is Define.xml round-trip support. The xml_builders.R layer can already write
Define.xml content from a Metacore object, but the base schema lacks the metadata fields
required by the CDISC Define-XML 2.0 standard (method names/types, document references,
comments, study-level attributes, etc.). Without those fields, a Define.xml produced by the
package is incomplete.

The implementation deliberately keeps this opt-in define_fields = FALSE (default) so that the
majority of current users who use metacore for existing ADaM/SDTM programming workflows are
unaffected. Only users who want to work with experimental features and build Define.xml documents need to pass
define_fields = TRUE.

@LiamHobby LiamHobby linked an issue Jun 6, 2026 that may be closed by this pull request
5 tasks
@LiamHobby LiamHobby self-assigned this Jun 6, 2026
@LiamHobby LiamHobby added enhancement New feature or request documentation Improvements or additions to documentation experimental labels Jun 6, 2026
@codecov

codecov Bot commented Jun 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.68879% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.60%. Comparing base (ad3f8d0) to head (204e9eb).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
R/spec_builder.R 89.54% 53 Missing ⚠️
R/MetacoreDefine.R 95.65% 4 Missing ⚠️
R/Metacore.R 98.75% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #157      +/-   ##
==========================================
+ Coverage   91.69%   93.60%   +1.90%     
==========================================
  Files           9       11       +2     
  Lines        1421     1861     +440     
==========================================
+ Hits         1303     1742     +439     
- Misses        118      119       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

define.xml documentation Improvements or additions to documentation enhancement New feature or request experimental

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhancement: Expand metacore schema to support Define.xml generation

1 participant