diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 0000000..4f9138d --- /dev/null +++ b/.github/workflows/tests.yml @@ -0,0 +1,20 @@ +name: Tests + +on: + push: + pull_request: + +jobs: + tests: + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10", "3.11", "3.12", "3.13"] + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + - run: python -m unittest discover -s tests -q + - run: python -m json.tool .agents/plugins/marketplace.json >/dev/null + - run: python -m json.tool .codex-plugin/plugin.json >/dev/null diff --git a/AGENTS.md b/AGENTS.md index 1114909..ab47ca5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,14 +16,44 @@ templates for agent-assisted implementation. - Do not publish credentials, raw private logs, screenshots containing private information, or local runtime output. +## Repository Shape + +The portable layer is the eight-skill, six-template workflow and its docs; it +works without Python. The optional `unknowns_harness` layer is a local, +non-authoritative observer and does not grant authority or perform actions. + +## Frozen Contracts + +Do not change without a recorded decision: + +- package, module, function, and dataclass field names; +- console script names, flags, exit codes, JSON keys, and key ordering; +- typed error strings and rejection fields; +- the eight skills and six templates, including their filenames and content. + +The exact module map and platform boundary are in `docs/architecture.md`. + +## Input Validation + +`evidence_graph`, `consultation_budget`, and `semantic_fingerprint` use +`type(value) is not str` deliberately; `isinstance(True, int) is True`. + +## Non-Authoritative Boundary + +The harness observes local inputs only. It must not grant authority, issue +tokens or permissions, perform external actions, or add an authority layer. + ## Verification Before pushing, run: ```bash +python3 -m unittest discover -s tests -q +python3 -m json.tool .agents/plugins/marketplace.json >/dev/null +python3 -m json.tool .codex-plugin/plugin.json >/dev/null git diff --check -find . -type f -not -path './.git/*' -maxdepth 4 -print ``` When changing Markdown, inspect the rendered structure or at least review the -headings and links. +headings and links. Read `docs/architecture.md` for the complete map and test +coverage summary. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d2ce38e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,5 @@ +# Agent Instructions + +This repository's agent instructions live in [AGENTS.md](AGENTS.md). +Read [docs/architecture.md](docs/architecture.md) for the module map, +platform boundary, and test coverage summary. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..c980e46 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,26 @@ +# Contributing + +The portable workflow layer is the primary product and must remain usable +without Python. The optional harness is a local observer only: it does not +grant authority, issue tokens or permissions, or perform external actions. + +## Verification + +From the repository root, run: + +```bash +python3 -m unittest discover -s tests -q +python3 -m json.tool .agents/plugins/marketplace.json >/dev/null +python3 -m json.tool .codex-plugin/plugin.json >/dev/null +git diff --check +``` + +Public-facing changes receive independent review before push. Changes to the +skills or templates are contract changes and need a discussion before editing. + +`tests/golden/` pins the public CLI output; regenerating it is a contract change requiring a decision. +`tests/test_docs_consistency.py` fails if a new document or skill is not linked from the README. + +See [AGENTS.md](AGENTS.md) for repository rules and +[docs/architecture.md](docs/architecture.md) for the module map, platform +boundary, and test coverage. diff --git a/PROJECT_PROFILE.yaml b/PROJECT_PROFILE.yaml index 2109c35..ed007f8 100644 --- a/PROJECT_PROFILE.yaml +++ b/PROJECT_PROFILE.yaml @@ -17,8 +17,10 @@ authority: verification: default: + - python3 -m unittest discover -s tests -q + - python3 -m json.tool .agents/plugins/marketplace.json >/dev/null + - python3 -m json.tool .codex-plugin/plugin.json >/dev/null - git diff --check - - find . -type f -not -path './.git/*' -maxdepth 4 -print layout: skills: skills/ diff --git a/README.md b/README.md index ffd3812..827c3f5 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,35 @@ Most agent failures are not only coding failures. They are failures to notice that the agent was operating with the wrong map: missing domain context, implicit taste, unasked architecture questions, or hidden codebase constraints. +## Copy-Paste Prompt + +Use this when you want an agent to apply the workflow without installing the +skills first: + +```text +Before implementing, run an unknowns pass. + +Classify the work into: +- Known Knowns +- Known Unknowns +- Unknown Knowns +- Unknown Unknowns + +Declare familiarity as new, partial, or known. If the area is unfamiliar, +teach me the likely blindspots before planning. If the success criteria are +implicit, give me a few concrete options to react to before editing code. If +named decisions remain, ask only the questions that would change architecture, +scope, or user-visible behavior. + +During implementation, keep implementation notes. If the real codebase forces +you to deviate from the plan, choose the conservative option and log: +- Expected +- Found +- Chosen conservative option +- Why conservative +- Follow-up needed +``` + ## The 4-Way Split Use these labels before implementation: @@ -233,36 +262,13 @@ disagreement in each direction. The recorded dispatch remains `executed: false`. The lower-level capability preflight remains available as -`unknowns-harness preflight`. +`unknowns-harness preflight`. For a capability preflight check, use +`examples/capabilities-missing.json` as a sample manifest with +`unknowns-harness preflight --manifest examples/capabilities-missing.json +--require ledger`. -## Copy-Paste Prompt - -Use this when you want an agent to apply the workflow without installing the -skills first: - -```text -Before implementing, run an unknowns pass. - -Classify the work into: -- Known Knowns -- Known Unknowns -- Unknown Knowns -- Unknown Unknowns - -Declare familiarity as new, partial, or known. If the area is unfamiliar, -teach me the likely blindspots before planning. If the success criteria are -implicit, give me a few concrete options to react to before editing code. If -named decisions remain, ask only the questions that would change architecture, -scope, or user-visible behavior. - -During implementation, keep implementation notes. If the real codebase forces -you to deviate from the plan, choose the conservative option and log: -- Expected -- Found -- Chosen conservative option -- Why conservative -- Follow-up needed -``` +The `workflow` module and console script require POSIX `fcntl`; the preflight +and review commands remain cross-platform. ## Optional Automation Gates @@ -279,6 +285,20 @@ unknowns, resolve what matters, log deviations, and teach the result. Reviews can use the [proportional review contract](docs/review-contract.md) and its optional immutable-artifact provenance check. +## Documentation + +- [Architecture](docs/architecture.md) — entrypoints, module map, boundaries, + platforms, tests, and growth seams. +- [Contributing](CONTRIBUTING.md) — verification and contribution guidance. +- [Workflow](docs/workflow.md) — the portable workflow in detail. +- [Examples](docs/examples.md) — example artifacts and usage. +- [Automation gates](docs/gates.md) — optional checks and review hooks. +- [Review contract](docs/review-contract.md) — proportional review guidance. +- [Project notes](docs/project-notes.md) — scope and design notes. +- [日本語の紹介](docs/ja.md) — Japanese overview. +- [Implementation record](docs/implementation-record.md) — frozen historical + build and review record. + ## Status The original portable skills, templates, and manual workflow are accompanied diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..01106e3 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,141 @@ +# Architecture + +`unknowns-workflow` ships two layers that are useful together but have +different requirements. + +## Two layers + +The portable workflow is the primary product: eight installable skills under +`skills/`, six reusable templates under `templates/`, and the explanatory +prose in `README.md` and `docs/`. It works without Python and does not depend +on the optional harness. + +The optional local harness is an installable Python package named +`unknowns_harness`. It observes and demonstrates the workflow locally. It +grants no authority, issues no token or permission, and performs no external +action. + +## Entrypoints and module map + +The public entrypoints are: + +| Entrypoint | Implementation | +| --- | --- | +| `unknowns-harness` | `unknowns_harness.cli:main` | +| `workflow` | `unknowns_harness.workflow_cli:main` | +| `python -m unknowns_harness` | `unknowns_harness.__main__` -> `cli:main` | + +The import and dependency direction is: + +```text +cli.py -> preflight, review_provenance +workflow_cli.py -> workflow +workflow.py -> preflight, evidence_graph, consultation_budget +evidence_graph.py -> stdlib only; imports fcntl +preflight.py -> stdlib only +consultation_budget.py -> stdlib only +review_provenance.py -> stdlib only +telemetry.py -> stdlib only; no in-package consumer +semantic_fingerprint.py -> stdlib only; no in-package consumer +__init__.py -> preflight only +``` + +Module purposes are intentionally small: + +- `cli.py` parses the `unknowns-harness` commands and prints preflight or + review-provenance results. +- `workflow_cli.py` parses the two deterministic `workflow` scenarios. +- `workflow.py` runs the portable-kernel demonstrations and produces their + stable observational reports. +- `preflight.py` checks a caller-supplied capability manifest and returns + typed local rejections. +- `evidence_graph.py` validates and appends non-authoritative JSONL evidence + events under an explicit root. +- `consultation_budget.py` computes deterministic blocker, evidence, and + attempt fingerprints and local consultation decisions. +- `review_provenance.py` validates an optional review record against its + caller-supplied artifact, route, and independence requirements. +- `telemetry.py` builds in-memory observational events with stable JSON. +- `semantic_fingerprint.py` hashes only explicitly named artifacts and paths. +- `__init__.py` exposes the small preflight API without importing POSIX-only + modules eagerly. +- `__main__.py` routes `python -m unknowns_harness` to `cli:main`. + +`telemetry` and `semantic_fingerprint` have no consumer inside the package. +They remain public API by intent: callers and tests import them directly, so +their absence would be a compatibility break rather than dead-code cleanup. + +## Platform boundary + +`evidence_graph.py` imports POSIX-only `fcntl`. Consequently, +`unknowns_harness.evidence_graph`, `unknowns_harness.workflow`, and the +`workflow` console script are POSIX-only. `import unknowns_harness`, +`unknowns-harness preflight`, and `unknowns-harness review` do not import +`fcntl` and remain cross-platform. + +## Input validation convention + +`evidence_graph.py`, `consultation_budget.py`, and +`semantic_fingerprint.py` deliberately use `type(value) is not str` at their +closed string-input boundaries instead of `isinstance(value, str)`. The exact +check rejects string subclasses and preserves the numeric-field rule that +`isinstance(True, int) is True`; a future cleanup must not weaken these +contracts. `preflight.py` and `review_provenance.py` use `isinstance` where +subclass acceptance is harmless. + +## Non-authoritative boundary + +The harness is an observational local checker. Its outputs describe declared +capabilities, review records, evidence, telemetry, fingerprints, and demo +results; they do not authorize effects, issue permissions or tokens, or +perform external actions. The boundary is not a universal `authoritative` +field: `PreflightResult`, review-provenance JSON, and telemetry JSON do not +carry that field and never did. + +The boundary's mechanical coverage is deliberately limited. The workflow +tests use a recursive closed-output check; the evidence path has a closed +`_FIELDS` set plus injection tests; telemetry has a complete JSON pin; and the +remaining serializers use frozen dataclass shapes. No single check enforces +the property globally. The remainder is a semantic boundary held by code +review, not a keyword filter presented as a proof of authority absence. + +## Tests and verification + +Tests live under `tests/` and run with the standard library: + +```bash +python3 -m unittest discover -s tests -q +``` + +The test modules cover these contracts: + +| Test module | Coverage | +| --- | --- | +| `test_cli.py` | malformed, rejected, and ready capability CLI results | +| `test_preflight.py` | manifest shape, capability values, and deterministic JSON | +| `test_review_provenance.py` | review record schema, routes, identity, and exit codes | +| `test_evidence_graph.py` | path safety, JSONL durability, schema, locking, and adjudication | +| `test_consultation_budget.py` | fingerprints, closed records, and consultation limits | +| `test_semantic_fingerprint.py` | explicit path bounds, escapes, and stable hashes | +| `test_telemetry.py` | stable in-memory event serialization | +| `test_workflow.py` | phase order, demos, kernel calls, determinism, and boundary assertions | + +The two plugin descriptors are validated as JSON with: + +```bash +python3 -m json.tool .agents/plugins/marketplace.json >/dev/null +python3 -m json.tool .codex-plugin/plugin.json >/dev/null +``` + +## Growth seams + +`run_workflow` is intentionally linear while there are two scenarios. If a +third scenario is added, split the function at its scenario branch so each +scenario can be read and tested independently. Do not add a registry or +strategy layer before that condition exists. + +`evidence_graph.py` currently keeps five concerns together: path safety, +canonical encoding, event schema validation, file locking, and exposure +adjudication. If the module grows further and a real change becomes difficult +to make, those concerns are the candidate split lines. This is a future seam, +not work to perform now. diff --git a/implementation-notes.md b/docs/implementation-record.md similarity index 99% rename from implementation-notes.md rename to docs/implementation-record.md index 0a357af..a7c11c3 100644 --- a/implementation-notes.md +++ b/docs/implementation-record.md @@ -1,3 +1,6 @@ +> This is the frozen historical record of how the harness was built and reviewed. +> It is not user documentation and is not maintained going forward. + # Unknowns Workflow Harness — Implementation Record ## Goal diff --git a/docs/ja.md b/docs/ja.md index c51b061..224c32d 100644 --- a/docs/ja.md +++ b/docs/ja.md @@ -4,6 +4,15 @@ AIにコードを書かせるとき、失敗の原因は「AIの実装力が低い」だけではありません。そもそも依頼文や計画の中に、まだ言語化されていない前提、未確認の仕様、コードベース特有の落とし穴が残っていることが多いです。 +## オプションのローカルハーネス + +このリポジトリには、ワークフローをローカルで観察・実演する任意の +Python ハーネスも含まれています。ハーネスは利用必須ではなく、 +ポータブルなスキルとテンプレートは Python なしで利用できます。 +ハーネスの概要は [README の説明](../README.md#what-the-harness-adds) を、 +モジュール構成と対応プラットフォームは +[アーキテクチャの説明](architecture.md) を参照してください。 + このワークフローは、そこを実装前に見つけるためのものです。 ## 何を解決するものか diff --git a/docs/project-notes.md b/docs/project-notes.md index e831344..c15ff01 100644 --- a/docs/project-notes.md +++ b/docs/project-notes.md @@ -8,7 +8,8 @@ for agent-assisted implementation. - Installable skill folders live under `skills/`. - Reusable artifact templates live under `templates/`. - Public workflow explanation lives under `docs/` and `README.md`. -- No CLI is included in the initial release. +- An optional local harness ships the `unknowns-harness` and `workflow` CLI + scripts; the portable workflow still works without Python. ## Design Choices diff --git a/pyproject.toml b/pyproject.toml index 007065b..a7e2081 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -8,6 +8,19 @@ version = "0.1.0" description = "A non-authoritative local harness for Unknowns Workflow checks." requires-python = ">=3.10" license = "MIT" +readme = "README.md" +dependencies = [] +classifiers = [ + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Operating System :: POSIX", +] + +[project.urls] +Homepage = "https://github.com/rasokiwayami/unknowns-workflow" +Repository = "https://github.com/rasokiwayami/unknowns-workflow" [project.scripts] unknowns-harness = "unknowns_harness.cli:main" diff --git a/tests/golden/adjudication.json b/tests/golden/adjudication.json new file mode 100644 index 0000000..8bad03e --- /dev/null +++ b/tests/golden/adjudication.json @@ -0,0 +1 @@ +{"adjudication":{"agreement_count":2,"agreement_rate":0.5,"authoritative":false,"disagreement_directions":{"allow->deny":1,"deny->allow":1},"exposure_count":4,"schema_version":1},"authoritative":false,"consultation":{"action":"consult","authoritative":false,"reason":"consultation_available"},"dispatch":{"authoritative":false,"executed":false,"mode":"local_record_only","record":"public-demo-dispatch"},"phases":[{"asset_check":{"assets":[{"authoritative":false,"identifier":"skill:finding-unknowns","paths":[{"authoritative":false,"path":"skills/finding-unknowns/SKILL.md","present":true},{"authoritative":false,"path":"templates/ledger.md","present":true}],"ready":true},{"authoritative":false,"identifier":"skill:unknowns-explainer","paths":[{"authoritative":false,"path":"skills/unknowns-explainer/SKILL.md","present":true},{"authoritative":false,"path":"templates/explainer.md","present":true}],"ready":true},{"authoritative":false,"identifier":"skill:unknowns-impl-notes","paths":[{"authoritative":false,"path":"skills/unknowns-impl-notes/SKILL.md","present":true},{"authoritative":false,"path":"templates/implementation-notes.md","present":true}],"ready":true}],"authoritative":false,"completed":true,"ready":true},"authoritative":false,"phase":"unknowns_intake","stable_identifiers":["skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"]},{"authoritative":false,"phase":"capability_preflight","ready":true,"ready_capabilities":["skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"],"rejections":[],"required_capabilities":["skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"],"summary":{"authoritative":false,"ready":3,"required":3}},{"authoritative":false,"executed":false,"mode":"local_record_only","phase":"bounded_local_dispatch","record":"public-demo-dispatch"},{"authoritative":false,"events":[{"authoritative":false,"baseline_decision":null,"decision":null,"dedup_key":"4a83c022676df5332d007f8b91c3d477174738e0f3c6e9c8f570e5cf54d71f0b","depends_on":[],"event_id":"public-adjudication-demo:4a83c022676df5332d007f8b","kind":"observation","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":1,"shadow_decision":null,"stream_id":"public-adjudication-demo","subject":"bounded-local-dispatch-recorded"},{"authoritative":false,"baseline_decision":null,"decision":"allow","dedup_key":"4ecb67fc6cbfcfbef9344ee91fcdef0567060f5124245567c902831e7e08efe9","depends_on":["public-adjudication-demo:4a83c022676df5332d007f8b"],"event_id":"public-adjudication-demo:4ecb67fc6cbfcfbef9344ee9","kind":"decision","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":2,"shadow_decision":null,"stream_id":"public-adjudication-demo","subject":"local-preflight-observed"}],"phase":"evidence_events"},{"authoritative":false,"decision":{"action":"consult","authoritative":false,"reason":"consultation_available"},"phase":"consultation_budget_check"},{"authoritative":false,"events":[{"authoritative":false,"baseline_decision":"allow","decision":null,"dedup_key":"5b27a576ef406c9a6e27546284af43f794df96eac89555914e1ee984f75d6132","depends_on":["public-adjudication-demo:4ecb67fc6cbfcfbef9344ee9"],"event_id":"public-adjudication-demo:5b27a576ef406c9a6e275462","kind":"comparable_exposure","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":3,"shadow_decision":"allow","stream_id":"public-adjudication-demo","subject":"case-1"},{"authoritative":false,"baseline_decision":"deny","decision":null,"dedup_key":"488020101d4dbffa3b287f0a18faa72969727e5ac442435706959d46a428021c","depends_on":["public-adjudication-demo:4ecb67fc6cbfcfbef9344ee9"],"event_id":"public-adjudication-demo:488020101d4dbffa3b287f0a","kind":"comparable_exposure","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":4,"shadow_decision":"deny","stream_id":"public-adjudication-demo","subject":"case-2"},{"authoritative":false,"baseline_decision":"allow","decision":null,"dedup_key":"6b1aa32f36ec747109b135bff39a3d88d12a8c7b4e90029a251d2b2e4de66c2a","depends_on":["public-adjudication-demo:4ecb67fc6cbfcfbef9344ee9"],"event_id":"public-adjudication-demo:6b1aa32f36ec747109b135bf","kind":"comparable_exposure","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":5,"shadow_decision":"deny","stream_id":"public-adjudication-demo","subject":"case-3"},{"authoritative":false,"baseline_decision":"deny","decision":null,"dedup_key":"8fc2284e9f93c0cc01221dc12eec58d2814d819bc446797754656c2892da885d","depends_on":["public-adjudication-demo:4ecb67fc6cbfcfbef9344ee9"],"event_id":"public-adjudication-demo:8fc2284e9f93c0cc01221dc1","kind":"comparable_exposure","produced_at":"2025-01-01T00:00:00+00:00","producer":"workflow-demo","schema_version":1,"sequence":6,"shadow_decision":"allow","stream_id":"public-adjudication-demo","subject":"case-4"}],"phase":"comparable_exposures"},{"authoritative":false,"phase":"adjudication_report","report":{"agreement_count":2,"agreement_rate":0.5,"authoritative":false,"disagreement_directions":{"allow->deny":1,"deny->allow":1},"exposure_count":4,"schema_version":1}}],"scenario":"adjudication","status":"completed"} diff --git a/tests/golden/typed-rejection.json b/tests/golden/typed-rejection.json new file mode 100644 index 0000000..c8e53d5 --- /dev/null +++ b/tests/golden/typed-rejection.json @@ -0,0 +1 @@ +{"authoritative":false,"phases":[{"asset_check":{"assets":[{"authoritative":false,"identifier":"skill:finding-unknowns","paths":[{"authoritative":false,"path":"skills/finding-unknowns/SKILL.md","present":true},{"authoritative":false,"path":"templates/ledger.md","present":true}],"ready":true},{"authoritative":false,"identifier":"skill:unknowns-explainer","paths":[{"authoritative":false,"path":"skills/unknowns-explainer/SKILL.md","present":true},{"authoritative":false,"path":"templates/explainer.md","present":true}],"ready":true},{"authoritative":false,"identifier":"skill:unknowns-impl-notes","paths":[{"authoritative":false,"path":"skills/unknowns-impl-notes/SKILL.md","present":true},{"authoritative":false,"path":"templates/implementation-notes.md","present":true}],"ready":true}],"authoritative":false,"completed":true,"ready":true},"authoritative":false,"phase":"unknowns_intake","stable_identifiers":["skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"]},{"authoritative":false,"phase":"capability_preflight","ready":false,"ready_capabilities":["skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"],"rejections":[{"authoritative":false,"capability":"bounded_local_dispatch","code":"missing_capability","remedy":"Configure the bounded local dispatch recorder locally, then rerun."}],"required_capabilities":["bounded_local_dispatch","skill:finding-unknowns","skill:unknowns-explainer","skill:unknowns-impl-notes"],"summary":{"authoritative":false,"ready":3,"required":4}}],"scenario":"typed-rejection","status":"rejected"} diff --git a/tests/test_docs_consistency.py b/tests/test_docs_consistency.py new file mode 100644 index 0000000..be5dd50 --- /dev/null +++ b/tests/test_docs_consistency.py @@ -0,0 +1,65 @@ +import re +import subprocess +import unittest +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parents[1] +LINK_PATTERN = re.compile(r"\[[^]]+\]\(([^)]+)\)") + + +def _tracked_markdown_files() -> list[Path]: + result = subprocess.run( + ["git", "ls-files", "--", "*.md"], + cwd=REPO_ROOT, + check=True, + capture_output=True, + text=True, + ) + return [ + REPO_ROOT / path + for path in result.stdout.splitlines() + if (REPO_ROOT / path).is_file() + ] + + +class DocsConsistencyTests(unittest.TestCase): + def test_every_doc_is_linked_from_readme(self) -> None: + readme_targets = { + target.split("#", 1)[0] + for target in LINK_PATTERN.findall((REPO_ROOT / "README.md").read_text()) + } + docs = sorted( + path.relative_to(REPO_ROOT).as_posix() + for path in (REPO_ROOT / "docs").glob("*.md") + ) + self.assertTrue(all(doc in readme_targets for doc in docs)) + + def test_every_relative_markdown_link_resolves(self) -> None: + links_resolve = True + for source in _tracked_markdown_files(): + for raw_target in LINK_PATTERN.findall(source.read_text(encoding="utf-8")): + target = raw_target.split("#", 1)[0] + if not target or target.startswith(("http://", "https://")): + continue + resolved = (source.parent / target).resolve() + if not resolved.is_relative_to(REPO_ROOT) or not resolved.is_file(): + links_resolve = False + self.assertTrue(links_resolve) + + def test_readme_skills_match_skill_directories(self) -> None: + section = (REPO_ROOT / "README.md").read_text().split("## Skills", 1)[1] + section = section.split("## ", 1)[0] + listed = sorted(re.findall(r"^- `([^`]+)`:", section, re.MULTILINE)) + directories = sorted( + path.name + for path in (REPO_ROOT / "skills").iterdir() + if path.is_dir() + ) + self.assertEqual(listed, directories) + + def test_every_skill_directory_has_skill_markdown(self) -> None: + directories = [ + path for path in (REPO_ROOT / "skills").iterdir() if path.is_dir() + ] + self.assertTrue(all((path / "SKILL.md").is_file() for path in directories)) diff --git a/tests/test_public_contract.py b/tests/test_public_contract.py new file mode 100644 index 0000000..235a498 --- /dev/null +++ b/tests/test_public_contract.py @@ -0,0 +1,43 @@ +# Protects JSON key names, key ordering, separators, phase names and order, and +# exit codes. Regenerating a golden file is a contract change, not a fix. + +import tempfile +import unittest +from contextlib import redirect_stdout +from io import StringIO +from pathlib import Path + +from unknowns_harness.workflow_cli import main + + +REPO_ROOT = Path(__file__).resolve().parents[1] + + +class PublicContractTests(unittest.TestCase): + def _assert_scenario( + self, scenario: str, golden_name: str, expected_exit_code: int + ) -> None: + with tempfile.TemporaryDirectory() as runtime_root: + output = StringIO() + with redirect_stdout(output): + exit_code = main( + [ + scenario, + "--assets-root", + str(REPO_ROOT), + "--runtime-root", + runtime_root, + ] + ) + + golden = (REPO_ROOT / "tests" / "golden" / golden_name).read_text( + encoding="utf-8" + ) + self.assertEqual(output.getvalue(), golden) + self.assertEqual(exit_code, expected_exit_code) + + def test_typed_rejection_output_and_exit_code(self) -> None: + self._assert_scenario("typed-rejection", "typed-rejection.json", 2) + + def test_adjudication_output_and_exit_code(self) -> None: + self._assert_scenario("adjudication", "adjudication.json", 0) diff --git a/unknowns_harness/__init__.py b/unknowns_harness/__init__.py index ea1defb..9b7ccbe 100644 --- a/unknowns_harness/__init__.py +++ b/unknowns_harness/__init__.py @@ -1,4 +1,13 @@ -"""Non-authoritative local checks for the portable Unknowns Workflow.""" +"""Non-authoritative local checks for the portable Unknowns Workflow. + +Public entrypoints are ``unknowns-harness`` through ``cli:main``, ``workflow`` +through ``workflow_cli:main``, and ``python -m unknowns_harness`` through +``__main__``. The package modules are split into capability preflight and +review validation, deterministic workflow demonstrations, evidence and +consultation decisions, plus the public telemetry and semantic-fingerprint +helpers. POSIX-only ``evidence_graph`` and ``workflow`` are intentionally not +imported here, so the package itself remains importable cross-platform. +""" from .preflight import PreflightResult, Rejection, check_capabilities diff --git a/unknowns_harness/cli.py b/unknowns_harness/cli.py index ad234bb..23b2f7e 100644 --- a/unknowns_harness/cli.py +++ b/unknowns_harness/cli.py @@ -1,4 +1,4 @@ -"""Command-line entrypoint for local harness checks.""" +"""Command-line entrypoint for capability and review-provenance checks.""" from __future__ import annotations diff --git a/unknowns_harness/consultation_budget.py b/unknowns_harness/consultation_budget.py index 974ec20..da695c0 100644 --- a/unknowns_harness/consultation_budget.py +++ b/unknowns_harness/consultation_budget.py @@ -1,4 +1,8 @@ -"""Pure, non-authoritative consultation budget decisions.""" +"""Pure, non-authoritative consultation budget decisions. + +Exact built-in type checks are deliberate at closed string and record +boundaries; they reject subclasses and preserve the input contract. +""" from __future__ import annotations diff --git a/unknowns_harness/evidence_graph.py b/unknowns_harness/evidence_graph.py index c71c24e..a0ea72b 100644 --- a/unknowns_harness/evidence_graph.py +++ b/unknowns_harness/evidence_graph.py @@ -1,4 +1,11 @@ -"""Append-only, non-authoritative evidence records under an explicit root.""" +"""Append-only, non-authoritative evidence records under an explicit root. + +This module handles path safety, canonical encoding, event schema validation, +file locking, and exposure adjudication. + +Exact built-in type checks are deliberate at closed event boundaries; they +reject subclasses and preserve the input contract. +""" from __future__ import annotations @@ -124,6 +131,33 @@ def _dedup_key(event: Mapping[str, object]) -> str: return hashlib.sha256(_canonical(identity)).hexdigest() +def _check_kind_decision_shape( + kind: object, + decision: object, + baseline_decision: object, + shadow_decision: object, +) -> None: + if kind == "observation": + if any( + value is not None + for value in (decision, baseline_decision, shadow_decision) + ): + raise EvidenceGraphError("schema_invalid") + elif kind == "decision": + if ( + decision not in DECISIONS + or baseline_decision is not None + or shadow_decision is not None + ): + raise EvidenceGraphError("decision_closed") + elif ( + decision is not None + or baseline_decision not in DECISIONS + or shadow_decision not in DECISIONS + ): + raise EvidenceGraphError("decision_closed") + + def _validate_event( event: Mapping[str, object], *, @@ -165,21 +199,12 @@ def _validate_event( for field in ("decision", "baseline_decision", "shadow_decision"): if event[field] is not None and type(event[field]) is not str: raise EvidenceGraphError("schema_invalid") - kind = event["kind"] - if kind == "observation": - if any(event[field] is not None for field in ("decision", "baseline_decision", "shadow_decision")): - raise EvidenceGraphError("schema_invalid") - elif kind == "decision": - if event["decision"] not in DECISIONS or any( - event[field] is not None for field in ("baseline_decision", "shadow_decision") - ): - raise EvidenceGraphError("decision_closed") - elif ( - event["decision"] is not None - or event["baseline_decision"] not in DECISIONS - or event["shadow_decision"] not in DECISIONS - ): - raise EvidenceGraphError("decision_closed") + _check_kind_decision_shape( + event["kind"], + event["decision"], + event["baseline_decision"], + event["shadow_decision"], + ) if event["dedup_key"] != _dedup_key(event): raise EvidenceGraphError("dedup_key_mismatch") if any(item["event_id"] == event["event_id"] for item in previous): @@ -288,22 +313,9 @@ def append_event( for value in (decision, baseline_decision, shadow_decision): if value is not None and type(value) is not str: raise EvidenceGraphError("schema_invalid") - if kind == "observation" and any( - value is not None for value in (decision, baseline_decision, shadow_decision) - ): - raise EvidenceGraphError("schema_invalid") - if kind == "decision" and ( - decision not in DECISIONS - or baseline_decision is not None - or shadow_decision is not None - ): - raise EvidenceGraphError("decision_closed") - if kind == "comparable_exposure" and ( - decision is not None - or baseline_decision not in DECISIONS - or shadow_decision not in DECISIONS - ): - raise EvidenceGraphError("decision_closed") + _check_kind_decision_shape( + kind, decision, baseline_decision, shadow_decision + ) produced = datetime.now(timezone.utc) if produced_at is None else _parse_time(produced_at) if produced > datetime.now(timezone.utc): raise EvidenceGraphError("future_event") diff --git a/unknowns_harness/preflight.py b/unknowns_harness/preflight.py index 3af71ae..621b645 100644 --- a/unknowns_harness/preflight.py +++ b/unknowns_harness/preflight.py @@ -1,4 +1,4 @@ -"""Deterministic, non-authoritative capability checks.""" +"""Deterministic, non-authoritative checks for declared capabilities.""" from __future__ import annotations diff --git a/unknowns_harness/review_provenance.py b/unknowns_harness/review_provenance.py index 254248c..cf81645 100644 --- a/unknowns_harness/review_provenance.py +++ b/unknowns_harness/review_provenance.py @@ -1,4 +1,4 @@ -"""Pure validation for an optional review-provenance record.""" +"""Pure validation for an optional caller-supplied review record.""" from __future__ import annotations diff --git a/unknowns_harness/semantic_fingerprint.py b/unknowns_harness/semantic_fingerprint.py index 857bfb3..b4a09c5 100644 --- a/unknowns_harness/semantic_fingerprint.py +++ b/unknowns_harness/semantic_fingerprint.py @@ -1,4 +1,8 @@ -"""Deterministic fingerprints for explicitly named public inputs.""" +"""Deterministic fingerprints for explicitly named public inputs. + +Exact built-in type checks are deliberate for closed artifact name and content +inputs; they reject subclasses where the public contract requires exact types. +""" from __future__ import annotations diff --git a/unknowns_harness/telemetry.py b/unknowns_harness/telemetry.py index 6b1f51f..b8b2447 100644 --- a/unknowns_harness/telemetry.py +++ b/unknowns_harness/telemetry.py @@ -1,4 +1,4 @@ -"""In-memory observational events with deterministic serialization.""" +"""Build in-memory observational events with deterministic serialization.""" from __future__ import annotations diff --git a/unknowns_harness/workflow.py b/unknowns_harness/workflow.py index 832861e..4e16ef9 100644 --- a/unknowns_harness/workflow.py +++ b/unknowns_harness/workflow.py @@ -1,4 +1,4 @@ -"""Deterministic, non-authoritative Unknowns Workflow demonstrations.""" +"""Run deterministic, non-authoritative Unknowns Workflow demonstrations.""" from __future__ import annotations diff --git a/unknowns_harness/workflow_cli.py b/unknowns_harness/workflow_cli.py index 372ad45..98c8eff 100644 --- a/unknowns_harness/workflow_cli.py +++ b/unknowns_harness/workflow_cli.py @@ -1,4 +1,4 @@ -"""Console entrypoint for deterministic Unknowns Workflow demos.""" +"""Console entrypoint for deterministic workflow demonstrations.""" from __future__ import annotations