Repository navigation
feat(shadow-ai): add cMCP audit adapter with trusted session bindings #322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
carloshvp
wants to merge
1
commit into
main
Choose a base branch
from
shadow-ai-cmcp-adapter
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+712
−11
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| name: Shadow AI tests | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: ['shadow-ai/**', 'noxfile.py', 'requirements/validate.txt', '.github/workflows/shadow-ai-tests.yml'] | ||
| push: | ||
| branches: [main] | ||
| paths: ['shadow-ai/**', 'noxfile.py', 'requirements/validate.txt', '.github/workflows/shadow-ai-tests.yml'] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| python-version: ['3.11', '3.12', '3.13'] | ||
| cmcp-version: ['0.2.0', '0.7.0'] | ||
| env: | ||
| PYTHONPATH: shadow-ai/src | ||
| SHADOW_AI_REQUIRE_CMCP: '1' | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| - name: Install tests | ||
| run: pip install --require-hashes -r requirements/validate.txt | ||
| - name: Install released cMCP audit API | ||
| # AuditChain and SqliteAuditStore use only the standard library. This | ||
| # exercises their wire format, not a gateway/TEE or its dependencies. | ||
| run: pip install --no-deps "cmcp-runtime==${{ matrix.cmcp-version }}" | ||
| - name: Test scanner and cMCP adapter | ||
| run: python -m pytest shadow-ai/tests -q |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| """Attribute cMCP audit records using operator-supplied session bindings. | ||
|
|
||
| Bindings and the audit source identifier are trusted policy inputs. This module | ||
| does not authenticate logs, attest a gateway, or derive identity from its catalog. | ||
| """ | ||
| from __future__ import annotations | ||
|
|
||
| from dataclasses import dataclass | ||
| from pathlib import Path | ||
| from types import MappingProxyType | ||
| from typing import Any, Iterable, Iterator | ||
|
|
||
| from shadow_ai_discovery import ( | ||
| IdentityUnavailable, | ||
| ScanReport, | ||
| ShadowAIScanner, | ||
| _json_type_name, | ||
| ) | ||
|
|
||
|
|
||
| def _require_identifier(value: str, field: str) -> None: | ||
| if not isinstance(value, str) or not value.strip(): | ||
| raise ValueError(f"{field} must be a non-blank string") | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class SessionAgentBinding: | ||
| """One authenticated session's identity, supplied by an operator/control plane. | ||
|
|
||
| Use a source_id that identifies the gateway and its session namespace/epoch. | ||
| Do not construct these from audit fields or a tool catalog. Identifiers are | ||
| exact strings, with no trimming, case folding or session-prefix matching. | ||
| """ | ||
|
|
||
| source_id: str | ||
| session_id: str | ||
| agent_id: str | ||
|
|
||
| def __post_init__(self) -> None: | ||
| for field in ("source_id", "session_id", "agent_id"): | ||
| _require_identifier(getattr(self, field), field) | ||
|
|
||
|
|
||
| class CmcpAuditAdapter(ShadowAIScanner): | ||
| """The scanner's four scan methods, with explicit cMCP session attribution. | ||
|
|
||
| source_id comes from the trusted caller's audit-source selection, never a | ||
| record. trusted_bindings are a snapshot: subsequent caller mutations cannot | ||
| rebind a session. Any duplicate source/session key is a configuration error. | ||
|
|
||
| Input records stay unchanged. timestamp retains the scanner's existing | ||
| behavior; timestamp_utc mapping/normalization is deliberately separate. | ||
| """ | ||
|
|
||
| def __init__( | ||
| self, | ||
| catalog_path: str | Path, | ||
| *, | ||
| source_id: str, | ||
| trusted_bindings: Iterable[SessionAgentBinding], | ||
| ) -> None: | ||
| _require_identifier(source_id, "source_id") | ||
| bindings: dict[tuple[str, str], str] = {} | ||
| for binding in trusted_bindings: | ||
| if not isinstance(binding, SessionAgentBinding): | ||
| raise TypeError("trusted_bindings must contain SessionAgentBinding objects") | ||
| key = (binding.source_id, binding.session_id) | ||
| if key in bindings: | ||
| raise ValueError("duplicate source/session binding; attribution is ambiguous") | ||
| bindings[key] = binding.agent_id | ||
| self._source_id = source_id | ||
| self._bindings = MappingProxyType(bindings) | ||
| super().__init__(catalog_path) | ||
|
|
||
| def _classify( | ||
| self, located_records: Iterable[tuple[int, dict[str, Any]]], location_kind: str | ||
| ) -> ScanReport: | ||
| unavailable: list[IdentityUnavailable] = [] | ||
|
|
||
| def attributed() -> Iterator[tuple[int, dict[str, Any]]]: | ||
| for location, record in located_records: | ||
| tool_name = record.get("tool_name") | ||
| if not tool_name: | ||
| continue | ||
| if not isinstance(tool_name, str) or not tool_name.strip(): | ||
| raise ValueError(f"invalid tool_name at {location_kind} {location}") | ||
| reason, observed_type = self._session_reason(record) | ||
| if reason is None: | ||
| agent_id = self._bindings[(self._source_id, record["session_id"])] | ||
| # An extra claim in a record cannot replace the trusted | ||
| # binding or conceal a conflicting identity assertion. | ||
| if "agent_id" in record and record["agent_id"] != agent_id: | ||
| reason = "agent_conflict" | ||
| if reason is not None: | ||
| unavailable.append(IdentityUnavailable( | ||
| location_kind, location, tool_name, record.get("timestamp", ""), | ||
| reason, observed_type, | ||
| )) | ||
| continue | ||
| yield location, {**record, "agent_id": agent_id} | ||
|
|
||
| report = super()._classify(attributed(), location_kind) | ||
| diagnostics = sorted((*unavailable, *report.unavailable), key=lambda item: item.location) | ||
| return ScanReport(report.findings, tuple(diagnostics)) | ||
|
|
||
| def _session_reason(self, record: dict[str, Any]) -> tuple[str | None, str | None]: | ||
| if "session_id" not in record: | ||
| return "session_missing", None | ||
| session_id = record["session_id"] | ||
| if session_id is None: | ||
| return "session_null", None | ||
| if not isinstance(session_id, str): | ||
| return "session_non_string", _json_type_name(session_id) | ||
| if not session_id.strip(): | ||
| return "session_blank", None | ||
| if (self._source_id, session_id) not in self._bindings: | ||
| return "session_unmapped", None | ||
| return None, None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| """Produce real cMCP SQLite payloads using an installed, released audit API.""" | ||
| from __future__ import annotations | ||
|
|
||
| import json | ||
| import sqlite3 | ||
| import sys | ||
| import tempfile | ||
| from importlib.metadata import version | ||
| from pathlib import Path | ||
|
|
||
| from cmcp_runtime.audit.chain import AuditChain | ||
| from cmcp_runtime.audit.store import SqliteAuditStore | ||
|
|
||
|
|
||
| def write_audit(path: Path) -> list[dict]: | ||
| """Exercise AuditChain and the exact payload serialization in SqliteAuditStore. | ||
|
|
||
| No gateway, network, TEE or caller authentication is exercised here. Session | ||
| identity comes from the separate operator bindings supplied by the tests. | ||
| """ | ||
| if version("cmcp-runtime") not in {"0.2.0", "0.7.0"}: | ||
| raise ValueError("fixture producer requires cmcp-runtime 0.2.0 or 0.7.0") | ||
| with tempfile.TemporaryDirectory() as directory: | ||
| database = Path(directory) / "audit.db" | ||
| store = SqliteAuditStore(database) | ||
| try: | ||
| for session in ("session-a", "session-b", "session-c"): | ||
| chain = AuditChain(session, store=store) | ||
| chain.append("tool_call", call_id=session + "/1", tool_name="echo", policy_decision="allow") | ||
| if session == "session-a": | ||
| chain.append("tool_call", call_id=session + "/2", tool_name="delete_all", policy_decision="deny") | ||
| finally: | ||
| store.close() | ||
| with sqlite3.connect(database) as connection: | ||
| payloads = [row[0] for row in connection.execute("SELECT payload FROM audit_entries ORDER BY rowid")] | ||
| path.write_text("\n".join(payloads) + "\n", encoding="utf-8") | ||
| return [json.loads(payload) for payload in payloads] | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| write_audit(Path(sys.argv[1])) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
with sqlite3.connect(...)commits but never closes, so the TemporaryDirectory cleanup hits WinError 32 on Windows in both interop tests.contextlib.closing(sqlite3.connect(database))fixes it.