conformance - Compatibility Tests

This repo defines what "Ajar Compatible" means. Any implementation, open or closed source, may claim compatibility if it passes the published vectors and harness.

T1.14 harness

The T1.14 deliverable is an executable Python harness under harness/, runnable as:

python -m harness offline --ajar-dir ../ajar
python -m harness gateway-core --target https://host --ajar-dir ../ajar

The harness currently reads schemas, examples, and seed vectors from an ajar spec checkout via --ajar-dir while vector layout migration is handled by issue #2. It does not import the reference implementation or use private hooks.

Requirements

Python 3.12+ is expected. Install the single allowed dependency:

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

requirements.txt declares only jsonschema; everything else is Python standard library.

Modes

offline runs deterministic vector families that do not need a live server: schema examples, invalid examples, scope matching, extension policy, runtime decision vectors, manifest checklist vectors, and crypto signing vectors.

gateway-core probes a live CORE gateway through its public HTTP interface: manifest discovery, manifest schema/lifetime/sequence checks, view content negotiation, browser passthrough, probeable §10 headers, and problem+json error header behavior.

Reports

The harness prints a human table to stdout. Add --json report.json before the subcommand to write machine-readable results:

python -m harness --json report.json offline --ajar-dir ../ajar

The JSON report is an array of:

{
  "check_id": "offline.scope.scope-exact-commerce-purchase",
  "spec_section": "8.1",
  "vector_id": "scope-exact-commerce-purchase",
  "verdict": "pass",
  "detail": "scope decision was allow"
}

verdict is pass, fail, or skip. Skips are never silent: every skipped check includes a reason in detail.

Exit codes

Exit code 0 means all non-skipped checks passed. Exit code 1 means at least one check failed. CLI usage errors return argparse's standard non-zero exit.

Principles

1. Every spec MUST maps to at least one vector.
2. Vectors are data; harnesses are thin. Third parties in any language can consume vectors directly.
3. Adversarial parity: passing requires surviving the attack set and the ordinary success cases.
4. Reference implementations run the same public suite as everyone else.

License

Apache-2.0. See LICENSE.