APxx datasets giving false positives, even when -v 3-4 is given.

Standard

CDISC SDTM/SDTMIG

Reference Rule ID(s)

CORE-00376 CORE-000575 CORE-000701 CORE-000575 CORE-000852 CORE-000902

Conformance Rule ID(s) (if published in CORE)

No response

JIRA Ticket

No response

CLI Command Used (if applicable)

python core.py -s SDTM -v 3.4 -d /path/to/datasets

Rule YAML

N/A

Attach any sample test data file(s)

When working on an SDTMIG-3.4 submission for a customer that contains APMH (which is based on 3.2 I presume), we got a number of false positives for the above CORE rules.
Some of the rules even treat it as a "custom domain".
Essentially, as I gave -v 3-4, APMH should not have been evaluated at all, as it is based on 3.2, and also as it is a very old standard even breaking some of the principles of SDTM-1.4 on which SDTMIG-3.2 is based.

For users not realizing this, these false positives can be very confusing, even tempting them to "correct" their datasets.

My proposal is to, when an APxx dataset is in the list of datasets submitted to the engine, then these are excluded from the validation.
In the Excel tab "Dataset Details" , these could be marked as "Not evaluated".

I presume this is a just 1 or a few lines in the Python code.

Attach any output report and/or log file(s)

N/A

Expected output

See above

Any Additional Information

No response

https://github.com/cdisc-org/cdisc-rules-engine/blob/main/cdisc_rules_engine/services/data_services/base_data_service.py#L240 this is the logic for AP-- and custom domains. If it thinks APMH is a custom domain, it means it is IDing one of these columns in the dataset to pin it to a specific class. If not, it checks if it is AP, grabs the MH off the dataset name, then goes to look for an MH domain submitted along with the APMH. It then likely IDs Events as the class off the MH domain.

I am not fond of skipping all AP-- datasets in engine and would like to resolve the issue at hand. Do you get a similar issue when running 3.2 as it sounds like the APMH is based on that version? If so, could you make some fake MH/APMH data I could reproduce this error with to resolve it?

Thanks! I will try with -v 3-2 tomorrow.
If that is what is needed, it means that in my application that runs on top of CORE, I will need to split off the validation of the APxx datasets using -v 3-2, as all the others usually are 3-4.

AP was a 3.2 (extension?) domain thar was never updated in 3.3 and 3.4.
It should have been ... but it looks as noone felt responsible.
I need to check whether it was added to 4.0. If not, I am going to make some trouble to the SDTM team ;-)

I redid with SDTMIG-3.2, and it now indeed looks much better.
For APMH, I still get one false positive for CORE-000852 (Variables are not in the correct order as shown in SDTM for the observation class): it looks as it expects APID to come before DOMAIN. But I also still get a lot of "rule execution error" with the message "Filename for domain doesn't exist". No idea why yet.
For SQAPMH (Suppqual for APMH), it is more problematic. I suspect it expects USUBJID as the identifier, but SQAPxx datasets of course has APID as the identifier.
I attach the Excel report, to which I added a column "Comments Jozef" in the tab "Issue Summary".
I unfortunately cannot provide you the XPT datasets themselves, as these are customer data.
If necessary, I can provide you some screenshots.

I cannot judge whether this should get any priority, just hoping that APxx will make it into SDTMIG-4.0. As far as I can see, it wasn't in the "public review" one.

CORE-Report-2026-09-01T08-18-09.xlsx

"Filename for domain doesn't exist" traces to _get_associated_persons_inherit_class

def _get_associated_persons_inherit_class(self, dataset_metadata):
    ap_suffix = dataset_metadata.ap_suffix
    ...
    datasets = self.get_datasets()
    if len(datasets) > 1:
        new_dataset_metadata = search_in_list(
            datasets, lambda item: item.domain == ap_suffix
        )
        if new_dataset_metadata:
            ...
            new_dataset = self.get_dataset(dataset_name=new_dataset_metadata.name)
        else:
            raise ValueError("Filename for domain doesn't exist")
    return self.get_dataset_class(new_dataset, new_dataset_metadata)

it appears to be looking for the parent of the AP-- in the submission to inherit the domain from it.

@JozefAerts

Section 1.4 of the SDTMIG 3.4 references the SDTMIG-AP, so version 3.4 of SDTM does indeed support the use of AP domains and the check should exist for -s SDTM -v 3.4. @JozefAerts is correct that the SDTM team need to maintain the IG for AP since it applies to SDTMIG versions >= 3.2.

MH does not need to exist for APMH to exist (there i s no parent/child relationship for AP-- domains). APMH is medical history for the associated persons only. Therefore, the presence of MH should not be checked if APMH is present. The same applies to other AP-- domains.

'Execution error' is shown for many rules but the variables references in the rules are not to AP domains but are meant for 'regular' SDMTIG domains.

CORE-000014,CORE-000015,CORE-000016,CORE-000018,CORE-000021,CORE-000023,CORE-000024,CORE-000030,CORE-000081,CORE-000089,CORE-000091,CORE-000092,CORE-000093,CORE-000094,CORE-000096,CORE-000098,CORE-000099,CORE-000100,CORE-000101,CORE-000117,CORE-000118,CORE-000133,CORE-000159,CORE-000160,CORE-000161,CORE-,000162,CORE-000163,CORE-000164,CORE-000184,CORE-000195,CORE-000196,CORE-000197,CORE-000198,CORE-,00199,CORE-000200

For CORE-000852, the order must be STUDYID,DOMAIN,APID,--SEQ. These variables with SREL are the only required variables.

In this case, the APMH may include variables that are used in MH (i.e., MHTERM, MHPRESP, MHOCCUR) but this is not mandatory.

CORE-000598 indicates that the 'Dataset name does not begin with DOMAIN value'. This is not a correct test for AP-- domains. If AP is the prefix, the 'suffix' should match a DOMAIN value.

For the SQAP-- domains, USUBJID will not be expected or permissable. APID is used with RDOMAIN and --SEQ to join the APMH and SQAPMH datasets.

Navigation Menu

Repository navigation

APxx datasets giving false positives, even when -v 3-4 is given. #1837

Standard

Reference Rule ID(s)

Conformance Rule ID(s) (if published in CORE)

JIRA Ticket

CLI Command Used (if applicable)

Rule YAML

Attach any sample test data file(s)

Attach any output report and/or log file(s)

Expected output

Any Additional Information

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

APxx datasets giving false positives, even when -v 3-4 is given. #1837

Description

Standard

Reference Rule ID(s)

Conformance Rule ID(s) (if published in CORE)

JIRA Ticket

CLI Command Used (if applicable)

Rule YAML

Attach any sample test data file(s)

Attach any output report and/or log file(s)

Expected output

Any Additional Information

Activity

SFJohnson24 commented on Aug 31, 2026

JozefAerts commented on Aug 31, 2026

JozefAerts commented on Sep 1, 2026

SFJohnson24 commented on Sep 9, 2026

pendingintent commented on Sep 10, 2026

pendingintent commented on Sep 10, 2026

pendingintent commented on Sep 10, 2026

pendingintent commented on Sep 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions