feat(agents,trace): one Langfuse trace per Agent Protocol run

.cursor/rules/ai-systems-engineer.mdc

@@ -13,7 +13,7 @@ You are the AI Systems Integration Lead. Your domain covers the configuration, r
 1. **Gateway Standard:** All LLM API calls must route through Envoy AI Gateway. Never connect an agent directly to OpenAI, Anthropic, or Bedrock.
 2. **Observability:** Ensure Langfuse tracing is injected into all agent executions and gateway requests. 
 3. **Orchestration (Aegra):** Ensure asynchronous multi-agent communication leverages Valkey as the message broker. 
-4. **Tool Execution:** All tool calls must be routed through the MCP (Model Context Protocol) Gateway to prevent credential leakage.
+4. **Tool Execution:** All tool calls must be routed through the MCP (Model Context Protocol) Gateway to prevent credential leakage. Native tools and customer extra backends share that gateway (`mcp.extraBackends`). Do not ship vendor MCP images in the platform chart. Do not give agents a second egress path to SaaS APIs.
 
 ## Output Format
 - Provide production-ready configuration files, Python/TypeScript agent logic, or integration code.

.cursor/rules/component-versions.mdc

@@ -0,0 +1,57 @@
+---
+description: Pin latest stable third-party versions; anchor on Langfuse/Aegra compatibility; match operators to datastore pins.
+globs: **/install.sh, **/values*.yaml, **/Chart.yaml, **/Dockerfile, **/requirements.txt, charts/**, profiles/**, images/**
+alwaysApply: true
+---
+
+# Component Version Policy
+
+## Default
+
+1. **Latest stable at release time:** Pick the current **stable** release of each third-party component (no beta, RC, or `-latest` tags). Verify it is actively maintained and not EOL.
+2. **Pin exactly:** Record semver in `charts/zelkor-platform/values.yaml`, `install.sh`, Dockerfiles, and operator chart versions. Use image tags with **major.minor.patch** (or digest in production overlays).
+3. **Hold until next Zelkor release:** Keep pins fixed between Zelkor semver releases unless a **security patch** or explicit dependency-bump PR updates them. Re-baseline the full stack together when bumping.
+
+## Product-anchored selection
+
+Do **not** pick datastore versions from Docker Hub alone. **Langfuse**, **Aegra**, and other runtime products define what they support.
+
+1. Read each product's official compatibility docs (Postgres, ClickHouse, Valkey/Redis, etc.).
+2. Choose the **highest stable version supported by every consumer** (compatibility intersection).
+3. If products disagree, use the **lowest common supported major** or upgrade the constraining product first — never deploy a DB version one product does not certify.
+
+**Example:** Langfuse v4 requires Postgres ≥16; Aegra supports Postgres ≥14 → pin **latest Postgres 16.x patch** both can use.
+
+## Operators (Gate B / production)
+
+When `databases.mode: operator-cr` or external operators are used:
+
+- **CloudNativePG**, **ClickHouse Operator**, **Valkey Operator**, **Qdrant Helm** (and future operators) must be the **latest stable operator release that officially supports the pinned datastore version**.
+- Gate A (`in-cluster-basic`) uses the **same datastore version pins**; only the deployment mechanism differs (StatefulSet vs CR).
+
+## Bootstrap / gateway stack
+
+- **Envoy Gateway** and **Envoy AI Gateway:** latest stable Helm/OCI release; document `--version` in `install.sh` (or env override). Apply upstream compatibility notes (EG version ↔ AI Gateway version).
+- Use upstream `envoy-gateway-values.yaml` / `extensionManager` requirements from the AI Gateway docs — do not hand-roll partial EG config.
+
+## Forbidden pins
+
+- Floating tags: `:latest`, major-only (`langfuse:2`, `postgres:16-alpine` without patch).
+- Unpinned Helm installs without `--version`.
+- Download URLs with `release/latest` (e.g. gVisor) — pin to a specific release artifact.
+
+## Where pins live
+
+| Layer | Location |
+|-------|----------|
+| Kind bootstrap (EG, AI Gateway, gVisor) | `install.sh` |
+| Platform workloads & datastores | `charts/zelkor-platform/values.yaml` |
+| Local-only overlay | `profiles/values-local.yaml` (secrets/hosts; same version pins unless documented exception) |
+| First-party images | `images/*/Dockerfile`, `values.yaml` `*.image.tag` / `digest` |
+| Living compatibility matrix | `internal/plan/component_compatibility_matrix.md` (multi-root workspace) |
+
+## When changing versions
+
+1. Update the compatibility matrix and revision history in `internal/plan/phase_1_requirements.md`.
+2. Bump all related pins (datastore + operator + bootstrap if needed) in one logical change.
+3. Validate on the test server via `internal/dev/` before merge.

.cursor/rules/example-charts.mdc

@@ -0,0 +1,18 @@
+---
+description: Example charts consume the platform via overlays; do not patch the platform chart to boot a demo
+globs: examples/**
+alwaysApply: false
+---
+
+# Example / Demo Charts
+
+Each demo is a standalone Helm release under `examples/<name>/chart/`.
+
+- Put agent logic, seed jobs, HTTPRoutes, domain schema, and E2E tests here. Agent Python belongs in a container image, or in `chart/files/` loaded with `.Files.Get` — never inlined in ConfigMap YAML.
+- Configure the platform through this chart's values and in-cluster DNS. Do not edit `charts/zelkor-platform/` to hard-wire this demo.
+- If the platform is missing a knob, stop. Add a generic value on the platform chart first, then set it here. SaaS MCP (ServiceNow, …) is `mcp.extraBackends: []` on the platform overlay plus **this** chart’s ClusterIP MCP Deployment — do not add a vendor MCP to `charts/zelkor-platform`.
+- Do not add this demo as a toggle inside the platform chart.
+- `install.sh` may deploy this release when `INSTALL_EXAMPLES=true`; that is orchestration, not a reason to change platform defaults.
+- Serve the demo graph from **this** release’s **ClusterIP** Deployment (`FROM zelkor-aegra`). Do not mount `aegra.json` onto platform Aegra or set `aegra.graphs` on the platform chart. Prefer clients hitting platform Aegra (`graph_id`); a demo HTTPRoute is TTFV-only, not the production exposure pattern.
+
+Tenant isolation handlers stay in platform `agents/`, not in the demo.

.cursor/rules/helm-python-packaging.mdc

@@ -0,0 +1,36 @@
+---
+description: Do not inline Python in ConfigMaps; ship images, or Helm files/ via .Files.Get
+alwaysApply: true
+---
+
+# Python Packaging (Helm / Kubernetes)
+
+Do **not** paste Python source into ConfigMap (or Secret) templates. Inline `server.py: |` / `import os` in YAML is forbidden in platform and example charts.
+
+## Preference
+
+1. **Required for app modules:** Put the module in a container image under `images/`. The Deployment runs `ghcr.io/devopssquaddev/zelkor-*` (pin `digest` when possible).
+2. **Allowed for config:** Keep SQL, Colang, or JSON under `chart/files/` and load it with `.Files.Get` or `.Files.Glob`. ConfigMaps may **mount** that file; they must not **contain** application Python in the template.
+3. **Forbidden:** Python (including `__init__.py` stubs) written under `data:` in `templates/*.yaml`.
+
+```yaml
+# BAD — source lives in the template
+data:
+  server.py: |
+    import os
+    from fastapi import FastAPI
+```
+
+```yaml
+# GOOD — config lives in files/; template only references it
+data:
+  init.sql: |
+{{ .Files.Get "files/init.sql" | indent 4 }}
+```
+
+```yaml
+# REQUIRED for app modules — image already has the Python
+# spec.containers[].image: ghcr.io/devopssquaddev/zelkor-mcp:dev
+```
+
+Edit and lint Python in `.py` files (or image build context), never inside Helm YAML.

.cursor/rules/necessary-length.mdc

@@ -0,0 +1,30 @@
+---
+description: Write only as much as the task needs; chat replies and new docs stay short by default
+alwaysApply: true
+---
+
+# Necessary length
+
+Write the shortest text that still does the job. Length tracks task complexity — not a default essay.
+
+Grounded in: [Anthropic verbosity](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5) (“Skip non-essential context, keep examples minimal”); [llmbestpractices](https://llmbestpractices.com/prompt-engineering/best-practices) (cut what does not change the answer; “be concise” is weak — bound length); Claude Code / Copilot CLI (direct answer, no preamble/postamble).
+
+## Chat
+
+- Lead with the answer. Stop when the user can act.
+- Prefer 1–3 short sentences or a small list. Expand only if they ask or the task is multi-step.
+- No preamble (“Great question”, “Here’s a comprehensive…”), no closing recap of what you just said.
+- One example beats a paragraph describing the format. Skip a second example unless it shows a different case.
+- State a rule once. Do not rephrase it.
+
+## Documents (`plan/`, `requirements/`, `docs/`, new `.md`)
+
+- Open with the decision or contract. Then acceptance criteria. Then a revision-history row if the living-spec rule applies.
+- Do not add Overview/Background/Rationale sections that restate the title or the decision.
+- Prefer a table or a short numbered list over narrative. Cut any paragraph the spec still works without.
+- New spec: aim at one screen (~40–80 lines) unless the user asked for a full ADR. Split later; do not pad now.
+- Do not invent extra sections “for completeness.”
+
+## Test
+
+Would a colleague with the title and this file still know what to do if you deleted the last third? If yes, delete it.

.cursor/rules/no-dev-defaults-in-platform.mdc

@@ -0,0 +1,52 @@
+---
+description: Never bake kind/dev hosts, unsigned auth, local secrets, or Docker Desktop endpoints into the production platform
+alwaysApply: true
+---
+
+# No Dev Defaults in the Platform
+
+`charts/zelkor-platform/`, `images/`, and `agents/` ship to customers. Kind/dev convenience lives only in `profiles/values-local.yaml` (and example `values-local.yaml`).
+
+## What must not appear as a shipped default
+
+| Kind of value | Examples | Allowed only in |
+|---|---|---|
+| Local hosts | `*.localhost`, `127.0.0.1`, `host.docker.internal` | local profile, tests env, quickstart (kind path) |
+| Unsigned / local auth | `dev-key`, `dev:` prefix, `devTokens.enabled: true`, `trustTenantHeader: true` | local profile |
+| Local secrets | `zelkor-dev-password`, `admin@zelkor.local`, baked Langfuse `pk-lf-zelkor-dev-*` | local profile |
+| Kind-only topology | `hostNetwork: true` as chart default, `kind-zelkor` | local profile / install.sh overlay |
+| Demo / lab tenants | `Bank_Alpha`, `Squad_Alpha`, fixture org ids | overlay or tests |
+
+Knobs are fine (`auth.devTokens.enabled: false`, `gateway.hosts.*: ""`). **Literals that make local kind work without an overlay are not.**
+
+## While implementing
+
+1. New Helm values default to empty, `false`, or a generic production-safe string — never a laptop/kind value.
+2. Templates: no `| default "ai-gateway.localhost"` (or any row above). Use `required` or omit the field when empty.
+3. Image / agent Python: `os.getenv("X", "")` — not `os.getenv("X", "http://mcp.localhost")` or `"dev-key"`.
+4. `install.sh` applies `profiles/values-local.yaml`; it does not write those values into the chart.
+5. Enabling unsigned tokens, trusting `X-Tenant-ID`, or pointing at Docker Desktop is an **overlay**, not a chart or image default.
+
+```yaml
+# BAD — kind host in the production chart
+hosts:
+  aiGateway: "ai-gateway.localhost"
+# BAD
+Host: {{ .Values.gateway.hosts.aiGateway | default "ai-gateway.localhost" }}
+
+# GOOD
+hosts:
+  aiGateway: ""
+```
+
+## After each substantial implementation
+
+Diff `values.yaml`, `templates/`, `files/`, `images/`, `agents/` and ask: **would this default still be correct on a customer cluster with no kind overlay?**
+
+- [ ] No new `*.localhost` / `host.docker.internal` / `127.0.0.1` as chart or image default
+- [ ] `devTokens` / `trustTenantHeader` still default off; no `dev-key` / `dev:` in chart or images
+- [ ] No new passwords, Langfuse lab keys, or `admin@zelkor.local` in `values.yaml`
+- [ ] New knobs are empty/off; local values landed in `profiles/values-local.yaml` only
+
+Sibling rule (pytest-only resources): `tests-do-not-shape-platform.mdc`.
+Spec: `internal/requirements/dev/examples_and_demos.md`

.cursor/rules/platform-chart-agnostic.mdc

@@ -0,0 +1,19 @@
+---
+description: Platform Helm chart stays generic; no example schemas, seeds, or demo-shaped defaults
+globs: charts/zelkor-platform/**
+alwaysApply: false
+---
+
+# Platform Chart
+
+This tree is Community Edition platform only.
+
+- Do not add example toggles (`*.enabled` for a demo name) to `values.yaml`. Local kind overlays live in `profiles/values-local.yaml`, not in this chart.
+- Do not seed demo databases, collections, prompts, or eval datasets here. Platform init may create platform-owned DBs only (e.g. `aegra`, `langfuse`).
+- Do not point MCP, NeMo, Langfuse, or Aegra defaults at a specific example's database, collection, project id, or product copy.
+- Do not import or template from `examples/`.
+- New knobs get generic defaults. Example-specific values belong in `examples/<name>/chart/`. Customer SaaS MCP registers via `mcp.extraBackends: []` (empty default) — do not add `servicenow` (or any vendor) as a chart default or Deployment.
+- Do not add `| default "*.localhost"`, `dev-key`, fixture tenants, or pytest-only Services/Routes so `tests/` pass. Tests adapt via env; kind hosts live in `profiles/values-local.yaml` (see `tests-do-not-shape-platform.mdc`).
+- Do not inline Python in templates. Prefer images; otherwise `files/` + `.Files.Get` (see `helm-python-packaging.mdc`).
+
+Customer production installs this chart with no `examples/` release.

.cursor/rules/platform-demo-boundary.mdc

@@ -0,0 +1,41 @@
+---
+description: Keep the platform chart domain-agnostic; demos overlay values and never patch platform defaults
+alwaysApply: true
+---
+
+# Platform vs Demo Boundary
+
+Demos live under `zelkor-platform/examples/<name>/` as **separate Helm charts**. They are not part of `charts/zelkor-platform`.
+
+## Placement test
+
+Before editing a file, ask: if every demo under `examples/` were deleted, would this still belong in the platform for a customer who never deploys examples?
+
+- **Yes** → platform (`charts/zelkor-platform/`, `agents/`, `tests/`)
+- **No** → that demo's chart, values, seed jobs, or `examples/<name>/tests/`
+
+## Invariant (no keyword lists)
+
+`INSTALL_EXAMPLES=false ./install.sh` then `pytest tests/` must be a valid, useful install. Platform tests must not assume any example chart, schema, collection, or seed data exists.
+
+`charts/zelkor-platform/` must not reference `examples/`. Examples may depend on the platform.
+
+## Split work
+
+If the task names both layers, do **platform first** and stop. Do not edit `examples/` until platform tests would pass without examples. Then implement the demo as an overlay only.
+
+If a demo needs a capability the platform lacks: add a **generic** Helm value with a **generic** default, then set the demo-specific value in `examples/<name>/`. Do not change platform defaults to make one demo boot.
+
+Do **not** register demo graphs onto the platform Aegra Deployment (`aegra.graphs`, ConfigMap `graphModules`). Each demo/customer agent is its own image and **ClusterIP** Deployment. Platform Aegra stays the **default public** front door with empty `graphs` (route by `graph_id`). Do not publish a public HTTPRoute per agent by default.
+
+## Defaults vs overlays
+
+The platform chart may expose knobs (`databaseUrl`, `collection`, refusal text, tenant mappings, `mcp.extraBackends: []`). It must not ship demo-shaped defaults, vendor MCP names (ServiceNow, …), or local secrets in `values.yaml`. Kind/dev hosts, passwords, and unsigned auth live in `profiles/values-local.yaml`. Each demo chart (or `install.sh` example overlay) supplies its own values. Customer SaaS MCP Deployments are the customer’s GitOps (or a demo chart), not `charts/zelkor-platform`.
+
+Platform tests create their own fixtures and use opaque tenant ids (`tenant-a`). Demo E2E lives only under `examples/<name>/tests/`. Tests must not bake kind hosts or pytest-only resources into the platform chart (see `tests-do-not-shape-platform.mdc`).
+
+## Stop and ask
+
+Do not edit `charts/zelkor-platform/` from a demo-only task except to add a generic knob. If unsure, ask.
+
+Spec: `internal/requirements/dev/examples_and_demos.md`

.cursor/rules/platform-engineer.mdc

@@ -16,6 +16,8 @@ You are the Platform Engineering Agent. Your primary responsibility is translati
 4. **BYOC & On-Prem Parity:** Ensure infrastructure code clearly delineates between AWS BYOC (e.g., EKS, Aurora, ElastiCache) and On-Premise deployments.
 5. **Sandboxing Enforcement:** Default untrusted workloads to `RuntimeClass: gVisor`. Use `kata-clh` for strict hardware-level VM isolation when requested.
 6. **GitOps Compliance:** All deployments must be declarative and compatible with ArgoCD or Flux. No manual `kubectl apply` commands should be recommended.
+7. **No inline Python in ConfigMaps:** Do not paste Python under `data:` in Helm templates. Prefer a container image; allowed fallback is `files/**/*.py` loaded with `.Files.Get` / `.Files.Glob`.
+8. **Component Version Policy:** Pin latest **stable** third-party releases; hold pins until the next Zelkor release. Select Postgres/ClickHouse/Valkey/etc. from **Langfuse + Aegra compatibility** (intersection), then pin operators to the latest stable release that supports those datastore versions. See `component-versions.mdc`.
 
 ## Output Format
 - Provide valid, linted Terraform (HCL) or Kubernetes manifests (YAML/Helm).

.cursor/rules/requirements-sync.mdc

@@ -39,9 +39,17 @@ Update governing requirement and specification documents whenever:
 
 ## 4. Mapping Matrix
 - **Platform Features & Helm Profiles:** `internal/plan/phase_1_requirements.md` (or relevant phase doc)
+- **Drop-in agent contract (intercept / wrap / MCP):** `internal/plan/requirements_drop_in_agent_contract.md`
+- **Agent Langfuse trace (one run = one waterfall):** `internal/plan/requirements_agent_trace.md`
+- **Deep Agents + `zelkor` CLI (Phase 2):** `internal/plan/requirements_deepagents_zelkor.md`
+- **Native MCP / BYO extra backends:** `internal/plan/requirements_native_mcp_servers.md` (§2.3 extra backends)
+- **Native Postgres / Qdrant MCP implementation:** `internal/plan/architecture_native_mcp_oss_wrap.md` (thin Postgres; Qdrant pip-wrap)
+- **Component pins & compatibility matrix:** `internal/plan/component_compatibility_matrix.md`
 - **Multi-Tier Roadmap:** `internal/plan/roadmap.md`
 - **FinServe Demo & Workload Scenarios:** `internal/requirements/demo/finserve_ai_demo.md`, `internal/requirements/dev/examples_and_demos.md`
 - **Developer & CI Gates:** `internal/requirements/dev/agent_code_review.md`, `internal/requirements/dev/agent_sdlc_and_tools.md`
+- **Test-server fail-stop (fix one test at a time):** `internal/.cursor/rules/test-server-fail-stop.mdc`
+- **Platform vs demo + tests-must-not-shape-chart:** `internal/requirements/dev/examples_and_demos.md`
 - **Quickstart & User Documentation:** `internal/requirements/quickstart/`, `zelkor-platform/docs/quickstart.md`
 
 ---

.cursor/rules/tdd-ai-enforcer.mdc

@@ -13,6 +13,7 @@ You are the enforcer of Test-Driven AI. In a compliance-heavy environment, AI mu
 1. **Tests First:** Before implementing any logic for Aegra orchestration, Envoy AI Gateway routing, or LangGraph agents, you must verify that a corresponding test suite (e.g., Pytest) or Langfuse Evaluation dataset exists.
 2. **Halt on Missing Specs:** If the user asks you to build a feature but no tests or OpenAPI specs exist for it, you must STOP and write the tests first. Ask the user to approve the tests before you write the implementation.
 3. **Execution:** Once tests are approved, you must write the code and iteratively run the tests until they pass. Do not declare a task finished if tests are failing.
+4. **Tests do not shape the platform:** Make tests pass by implementing the product or by env/overlay in the test — never by baking kind hosts, fixture literals, or pytest-only resources into `charts/zelkor-platform/` (see `tests-do-not-shape-platform.mdc`).
 
 ## Output Format
 - Output Pytest fixtures, Langfuse synthetic datasets, or OpenAPI specs.